NVIDIA Time Series Forecasting

And time series forecasting is a frequently useful need in anaytics.  NVIDIA provides a means, have not examined closely to see how complete this is. 

 Time Series Forecasting with the NVIDIA Time Series Prediction Platform and Triton Inference Server

By Kyle Kranen, Pawel Morkisz and Carl (Izzy) Putterman.  NVIDIA

n this post, we detail the recently released NVIDIA Time Series Prediction Platform (TSPP), a tool designed to compare easily and experiment with arbitrary combinations of forecasting models, time-series datasets, and other configurations. The TSPP also provides functionality to explore the hyperparameter search space, run accelerated model training using distributed training and Automatic Mixed Precision (AMP), and deploy and run inference on accelerated model formats on the NVIDIA Triton Inference Server.

Accurately forecasting future time series values using previous values has proven pivotal in understanding and managing complex systems, including but not limited to power grids, supply chains, and financial markets. In these forecasting applications, single-digit percentage improvements in predictive accuracy can have vast financial, ecological, and social impacts. In addition to needing to be accurate, forecasting models also must be able to function on real-time timescale.  ... '

Building TimeSeries Databases

Spent much of our careers working with timeseries data in supply chain analysis...

Not enough metadata considered?    

Always-on Time-Series Database: Keeping Up Where There's No Way to Catch Up  By Theo Schlossnagle, Justin Sheehy, Chris McCubbin

Communications of the ACM, July 2021, Vol. 64 No. 7, Pages 50-56  10.1145/3442518

In all likelihood, you have never given so much as a thought to what it might take to produce your own database. And you will probably never find yourself in a situation where you need to do anything of the sort.

But, if only as a thought exercise, consider this for a moment: What if, as a core business requirement, you found you needed to provide for the capture of data from disconnected operations, such that updates might be made by different parties at the same time—or in overlapping time—without conflicts? And what if your service called for you to receive massive volumes of data almost continuously throughout the day, such that you couldn't really afford to interrupt data ingest at any point for fear of finding yourself so far behind present state that there would be almost no way to catch up? Given all that, are there any commercially available databases out there you could use to meet those requirements?

Right. So, where would that leave you? And what would you do then? We wanted to explore these questions with Theo Schlossnagle, who did, in fact, build his own time-series database. As the founder and CTO of Circonus, an organization that performs telemetry analysis on an already large and exponentially growing number of IoT (Internet of Things) devices, Schlossnagle had good reason to make that investment.

Justin Sheehy, the chief architect of global performance and operations for Akamai, asks Schlossnagle about the thinking behind that effort and some of the key decisions made in the course of building the database, as well as what has been learned along the way. On behalf of ACM, Chris McCubbin, a senior applied scientist with Amazon Web Services, contributes to the discussion.

JUSTIN SHEEHY: As someone who once made the dubious decision to write my own database, I know it can prove to be the right thing to do, but—for most companies—I don't think it turns out that way. This isn't just a business question, but one that also has some interesting engineering dimensions to it. So, Theo, why did you feel the need to write your own time-series database?  .... 

Google AI Describes AutoML for Time Series

 Most our careers in the big enterprise involved working with time series.   Sales, Shipments delivered, Advertising Dollars,  marketing spends ... forecast plans and predictions.   A favorite quote was 'the forecast is wrong', but how wrong?   And Why?  And what are the risks involved?  So if we could do forecasts better, more data and intelligence based?  How might we do it?     

Using AutoML for Time Series Forecasting      In the GoogleBlog.

Friday, December 4, 2020

Posted by Chen Liang and Yifeng Lu, Software Engineers, Google Research, Brain Team

Time series forecasting is an important research area for machine learning (ML), particularly where accurate forecasting is critical, including several industries such as retail, supply chain, energy, finance, etc. For example, in the consumer goods domain, improving the accuracy of demand forecasting by 10-20% can reduce inventory by 5% and increase revenue by 2-3%. Current ML-based forecasting solutions are usually built by experts and require significant manual effort, including model construction, feature engineering and hyper-parameter tuning. However, such expertise may not be broadly available, which can limit the benefits of applying ML towards time series forecasting challenges.

To address this, automated machine learning (AutoML) is an approach that makes ML more widely accessible by automating the process of creating ML models, and has recently accelerated both ML research and the application of ML to real-world problems. For example, the initial work on neural architecture search enabled breakthroughs in computer vision, such as NasNet, AmoebaNet, and EfficientNet, and in natural language processing, such as Evolved Transformer. More recently, AutoML has also been applied to tabular data.

Today we introduce a scalable end-to-end AutoML solution for time series forecasting, which meets three key criteria:

Today we introduce a scalable end-to-end AutoML solution for time series forecasting, which meets three key criteria:

Fully automated: The solution takes in data as input, and produces a servable TensorFlow model as output with no human intervention.

Generic: The solution works for most time series forecasting tasks and automatically searches for the best model configuration for each task.

High-quality: The produced models have competitive quality compared to those manually crafted for specific tasks.

We demonstrate the success of this approach through participation in the M5 forecasting competition, where this AutoML solution achieved competitive performance against hand-crafted models with moderate compute cost... .' 

Tutorial: Random Forest for Time Series

 Am a long  time proponent of ensemble methods.  Here Jason Brownlee provides a nice tutorial on an often powerful method everyone should know.  As usual, well done, minimal tech.

Random Forest for Time Series Forecasting   by Jason Brownlee

by Jason Brownlee  in Time Series

Random Forest is a popular and effective ensemble machine learning algorithm.

It is widely used for classification and regression predictive modeling problems with structured (tabular) data sets, e.g. data as it looks in a spreadsheet or database table.

Random Forest can also be used for time series forecasting, although it requires that the time series dataset be transformed into a supervised learning problem first. It also requires the use of a specialized technique for evaluating the model called walk-forward validation, as evaluating the model using k-fold cross validation would result in optimistically biased results.

In this tutorial, you will discover how to develop a Random Forest model for time series forecasting.

After completing this tutorial, you will know:

Random Forest is an ensemble of decision trees algorithms that can be used for classification and regression predictive modeling.  Time series datasets can be transformed into supervised learning using a sliding-window representation.How to fit, evaluate, and make predictions with an Random Forest regression model for time series forecasting.

Let’s get started.   ....  

Astronomy Methods for Business Analytics?

Brought to attention by some of my astro colleagues, the effort is considerable.  Could businesses also construct such a 'survey' of how they operate?   Which could lead to a determination of where data might be used, needed?

The Vera C. Rubin Observatory, currently under construction in Chile, will conduct a vast astronomical survey of our dynamic Universe starting in 2022. They plan to collect 500 petabytes of image data by observing the skies continuously for 10 years and produce nearly instant alerts for objects that change in position or brightness every night. In addition to astronomical data, their dataset will include DevOps, IoT, and real-time monitoring data.

In this latest Data Science Central webinar, Dr. Angelo Fausti will demonstrate:

●     How a time-series database has the versatility to address their needs
●     How they created a solution to enhance visibility across their organization and improve actionable insights
●     How they pull software development and sensor data from their telescope, camera and observatory IoT devices

Speaker:
Dr. Angelo Fausti, Software Engineer - Vera C. Rubin Observatory

Hosted by:
Sean Welch, Host and Producer - Data Science Central

--- --------------------------------------------------------------------------

LSST Project Mission Statement
LSST’s mission is to build a well-understood system that provides a vast astronomical dataset for unprecedented discovery of the deep and dynamic universe. .... 

MIT Model Forecasts Business Financials

If real, here is something of considerable value ... Note 'Belief Propagation' .... which we tested for things like predicting future product sales in varying context.  Looking more closely.

Model Beats Wall Street Analysts in Forecasting Business Financials
MIT News
By Rob Matheson

Massachusetts Institute of Technology (MIT) researchers have developed an automated model that significantly outperforms humans in predicting business sales using only anonymized weekly credit card transactions and three-month earnings reports. The researchers used the model to predict quarterly earnings of more than 30 companies, and found that it outperformed the combined estimates of expert Wall Street analysts on 57% of predictions. The human analysts had access to any available private or public data and other machine learning models, while the MIT model used a very small dataset of the two data types. The researchers used a variation of the standard inference algorithm, called Kalman filtering or Belief Propagation. This technique uses data measurements observed over time, containing noise inaccuracies, to generate a probability distribution for unknown variables over a designated timeframe .... " 

Deep Learning for Time Series and Signal Processing

We researched the topic of using deep learning to replace forecasting time series applications in the enterprise,  for faster and more accurate results, but were never successful.    The need to segment many small  time steps and solve the resulting large neural nets was too difficult.   Here a useful piece on the current related approaches with machine learning.   Reading and will comment further.  From DSC.   Join the group

Deep learning: the final frontier for signal processing and time series analysis?
Posted by Andrea Manero-Bastin  
This article was written by Alexandr Honchar.

People use deep learning almost for everything today, and the “sexiest” areas of applications are computer vision, natural language processing, speech and audio analysis, recommender systems and predictive analytics. But there is also one field that is unfairly forgotten in terms of machine learning — signal processing (and, of course, time series analysis). In this article, I want to show several areas where signals or time series are vital, after I will briefly review classical approaches and will move on to my experience with applying deep learning for biosignal analysis in Mawi Solutions and for algorithmic trading. I already gave a couple of talks on this topic in Barcelona and Lviv, but I would like to make the materials a bit more accessible.

I am sure, that not only people working with time series data will benefit from this article. Computer vision specialists will learn how similar their domain expertise is to signal processing, NLP people will get some insights about sequential modeling and other professionals can have interesting takeaways as well. Enjoy!  ... '

Time Series and Deep Learning

A Considerable look at Deep learning and time series problems.   We did many of these kinds of problems in the enterprise.  And we did not consider DL because it seemed inefficient.   In O'Reilly.

3 reasons to add deep learning to your time series toolkit
The most promising area in the application of deep learning methods to time series forecasting is in the use of CNNs, LSTMs, and hybrid models.

By Francesca Lazzeri:

The ability to accurately forecast a sequence into the future is critical in many industries: finance, supply chain, and manufacturing are just a few examples. Classical time series techniques have served this task for decades, but now deep learning methods—similar to those used in computer vision and automatic translation—have the potential to revolutionize time series forecasting as well.

Due to their applicability to many real-life problems—such as fraud detection, spam email filtering, finance, and medical diagnosis—and their ability to produce actionable results, deep learning neural networks have gained a lot of attention in recent years. Generally, deep learning methods have been developed and applied to univariate time series forecasting scenarios, where the time series consists of single observations recorded sequentially over equal time increments. For this reason, they have often performed worse than naïve and classical forecasting methods, such as exponential smoothing (ETS) and autoregressive integrated moving average (ARIMA). This has led to a general misconception that deep learning models are inefficient in time series forecasting scenarios, and many data scientists wonder whether it’s really necessary to add another class of methods—such as convolutional neural networks or recurrent neural networks—to their time series toolkit.

In this post, I'll discuss some of the practical reasons why data scientists may still want to think about deep learning when they build time series forecasting solutions. ... "

AWS Time-Series Database

This says it is an architecture better adapted for time series and related metadata.  Most of what we did in supply chain analysis used time series prediction.

AWS Launches Time-Series Database  by Alex Woodie in Datanami

And details in AWS.

AWS threw its hat into the nascent ring for time-series databases yesterday with the launch of AWS TimeStream, a managed time-series database that AWS says can handle trillions of events per day.

Time-series databases have emerged as a best-in-class approach for storing and analyzing huge amounts of data generated by users and IoT devices. While relational and NoSQL databases are sometimes used for time-stamped and time-series data – such as clickstream data from Web and mobile devices, log data from IT gear, and data generated by industrial machinery — today’s massive data volumes from the IoT have outstripped the capability of those databases to keep up.

As the high-end time-series use cases piled up, AWS decided it was time to take action and make its entry into the still-specialized field, much as it did with last year’s launch of Neptune, a graph database, which is another specialized database field that’s emerging.   ... "

AWS says its new Timestream database organizes data by time intervals, which reduces the amount of data that needs to be scanned to answer a query. It minimizes storage needs and costs by automatically applying rollups, retention, tiering, and compression of data. AWS is delivering the services (it’s still in a technical preview) as a serverless product, which means there’s no underlying server on AWS to manage.

Timestream features what AWS calls an “adaptive query processing engine,” which it says can adapt to different time scales, like milliseconds, microseconds, and nanoseconds. All told, AWS claims Timestream can deliver 1,000 faster query performance at one-tenth the cost of a relational database. ... "

Patents for Anomaly Detection

 There are many ways to do this kind of anomaly detection.  Done them for years.  Anomaly detection alone is not new or patentable, but perhaps as a larger process?

Anodot Gains Patents for Anomaly Detection By George Leopold in Datanami

Anodot, which focuses on using machine learning techniques to spot anomalies in time-series data, announced a pair of U.S. patent awards this week covering its autonomous analytics framework.

The analytics vendor said Thursday (Oct. 11) it has been granted two U.S. patents for algorithms that allow users to apply machine learning-base anomaly detection. The algorithms are designed specifically to quickly identify the source of anomalies in large data sets, then perform root-cause analysis. The approach is promoted as faster than traditional business intelligence tools or dashboards.

The first patent award covers a method for identifying and analyzing data anomalies by comparing them with previous incidents to “determine their sensitivity,” the company said. Anodot trains its machine learning algorithms based on human behavior rather than using statistical analysis tools.

“By leveraging machine learning and artificial intelligence capabilities, we’re able to tap into human perception and identify business incidents that other BI tools would never find,” claimed Ira Cohen, Anodot’s co-founder and chief data scientist.

The second patent award is for an algorithm used to identify “seasonal trends,” including daily and weekly patterns that could be used to improve detection of data anomalies. Anodot said the technology can be used to provide autonomous analytics alerts to business customers as incidents are detected..... "

Deep Learning for Time Series Forecasting

Like Jason's style of clear motivations  and short tutorials.  You can get free samples of his writing below.

Jason Brownlee's New Book: 
Deep Learning for Time Series Forecasting
Predict the Future with MLPs, CNNs and LSTMs in Python
Deep Learning for Time Series Forecasting
$37 USD

Deep learning methods offer a lot of promise for time series forecasting, such as the automatic learning of temporal dependence and the automatic handling of temporal structures like trends and seasonality.

In this new Ebook written in the friendly Machine Learning Mastery style that you’re used to, finally cut through the math, research papers and patchwork descriptions about time series forecasting with deep learning algorithms.

With clear explanations, standard Python libraries, and step-by-step tutorial lessons you’ll discover how to develop deep learning models for your own time series forecasting projects.

About this Ebook:

Read on all devices: PDF format Ebook, no DRM.
Tons of tutorials: 5 parts, 25 step-by-step lessons, 575 pages.
Real-world projects: 2 large end-to-end tutorial projects.
Many datasets: Univariate, multivariate, multi-step, and more.
Working code: 131 Python (.py) code files included.
Clear, Complete End-to-End Examples.
Convinced? ....  "

What and Why are ARCH and GARCH?

When you do time series forecasting you almost always get changes in variance over time.  Sometimes enough to invalidate your decisions and conclusions.  We used these  methods in key ways to produce better results over time.  Somehow I rarely hear these methods mentioned recently.  Here Jason Brownlee provides a good Python based intro.  Fairly non-technical, but coding based.

How to Model Volatility with ARCH and GARCH for Time Series Forecasting in Python by Jason Brownlee   in Time Series

A change in the variance or volatility over time can cause problems when modeling time series with classical methods like ARIMA.

The ARCH or Autoregressive Conditional Heteroskedasticity method provides a way to model a change in variance in a time series that is time dependent, such as increasing or decreasing volatility. An extension of this approach named GARCH or Generalized Autoregressive Conditional Heteroskedasticity allows the method to support changes in the time dependent volatility, such as increasing and decreasing volatility in the same series.

In this tutorial, you will discover the ARCH and GARCH models for predicting the variance of a time series.

After completing this tutorial, you will know:

The problem with variance in a time series and the need for ARCH and GARCH models.
How to configure ARCH and GARCH models.
How to implement ARCH and GARCH models in Python.
Let’s get started.   .... "

Times Series with RNN Neural Nets

Been re-examining neural networks for time series models and forecasting.  In the long ago work of modeling with neural nets we had determined it was not useful, but new architectures of recurrent Neural nets RNN make it worth another look.  Here is an examination with Tensorflow

Building Recurrent Neural Networks in Tensorflow

Posted by Ahmet Taspinar  in DSC

Recurrent Neural Nets (RNN) detect features in sequential data (e.g. time-series data). Examples of applications which can be made using RNN’s are anomaly detection in time-series data, classification of ECG and EEG data, stock market prediction, speech recogniton, sentiment analysis, etc.

This is done by unrolling the data into N different copies of itself (if the data consists of N time-steps) .
In this way, the input data at the previous time steps t_n - 1, t_n - 2, t_n - 3, ... , t_0 can be used when the data at timestep t_n is evaluated. If the data at the previous time steps is somehow correlated to the data at the current time step, these correlations are remembered and otherwise they are forgotten.

By unrolling the data, the weights of the Neural Network are shared across all of the time steps, and the RNN can generalize beyond the example seen at the current timestep, and beyond sequences seen in the training set. .... "

Sequencing Problems for Natural Language Processing

Good piece by William Vorhies

Temporal Convolutional Nets (TCNs) Take Over from RNNs for NLP Predictions   Posted by William Vorhies in DSC

Summary: Our starting assumption that sequence problems (language, speech, and others) are the natural domain of RNNs is being challenged.  Temporal Convolutional Nets (TCNs) which are our workhorse CNNs with a few new features are outperforming RNNs on major applications today.  Looks like RNNs may well be history.

It’s only been since 2014 or 2015 when our DNN-powered applications passed the 95% accuracy point on text and speech recognition allowing for whole generations of chatbots, personal assistants, and instant translators.

Convolutional Neural Nets (CNNs) are the acknowledged workhorse of image and video recognition while Recurrent Neural Nets (RNNs) became the same for all things language.

One of the key differences is that CNNs can recognize features in static images (or video when considered one frame at a time) while RNNs excelled at text and speech which were recognized as sequence or time-dependent problems.  That is where the next predicted character or word or phrase depends on those that came before (left-to-right) introducing the concept of time and therefore sequence.

Actually RNNs are good at all types of sequence problems, including speech/text recognition, language-to-language translation, handwriting recognition, sequence data analysis (forecasting), and even automatic code generation in many different configurations. .... " 

Recalculating Patterns in Real Time

Intriguing approach for constantly changing time series data.

Novel Algorithm Enables Statistical Analysis of Time Series Data 

MIT News  By Sara Cody

Researchers at the Massachusetts Institute of Technology (MIT) have developed state-space multitaper time-frequency analysis (SS-MT), a unique algorithm they say delivers time series dataset analysis in real time. The team notes SS-MT enables scientists to work in a more informed manner with large, nonstationary datasets so they can not only measure the fluid properties of data but also make formal statistical comparisons between arbitrary data segments. "The algorithm functions similarly to the way a [global-positioning system] calculates your route when driving," says MIT professor Emery Brown. The team tested SS-MT by first analyzing electroencephalogram readings from patients receiving general anesthesia for surgery. The program produced a de-noised spectrogram defining changes in power across frequencies over time, and the researchers also applied SS-MT's inference paradigm to compare different levels of unconsciousness in terms of the differences in the spectral properties of these behavioral states. "The SS-MT analysis produces cleaner, sharper spectrograms," Brown says.  .... "

Time Series Insights in Azure

Time Series Insights PREVIEW
Instantly explore and analyze time-series data in IoT solutions
Azure Time Series Insights is a fully managed analytics, storage, and visualization service that makes it simple to explore and analyze billions of IoT events simultaneously. It gives you a global view of your data, letting you quickly validate your IoT solution and avoid costly downtime to mission-critical devices by helping you discover hidden trends, spot anomalies, and conduct root-cause analyses in near real-time.  .... " 

(Start for free at link, taking a look at the root cause example, always a great analytics place to start, because it concerns everyone) 

Grafana for Time Series

Brought to my attention. Grafana.

" ... The leading open source software for time series analytics ... Grafana is an open source metric analytics & visualization suite. It is most commonly used for visualizing time series data for infrastructure and application analytics but many use it in other domains including industrial sensors, home automation, weather, and process control.    .... " 

Facebook Releases Open Source Prophet

Doing time series forecasting well is important to every business.  Good to see this open sourcing underway by the Facebook Incubator.    Though the history of Facebook and the nature of its data has always made me worried about privacy

" .. Prophet is a procedure for forecasting time series data. It is based on an additive model where non-linear trends are fit with yearly and weekly seasonality, plus holidays. It works best with daily periodicity data with at least one year of historical data. Prophet is robust to missing data, shifts in the trend, and large outliers.

Prophet is open source software released by Facebook’s Core Data Science team. It is available for download on CRAN and PyPI. ... " 

Random Walks for Time Series Forecasting

Nicely done, fairly gentle.

A Gentle Introduction to the Random Walk for Times Series Forecasting with Python   by Jason Brownlee on January 20, 2017 in Time Series

How do you know if your time series problem is predictable?

This is a difficult question with time series forecasting. There is a tool called a random walk that can help you understand the predictability of your time series forecast problem.

In this tutorial, you will discover the random walk and its properties in Python.
After completing this tutorial, you will know:

What the random walk is and how to create one from scratch in Python.
How to analyze the properties of a random walk and recognize when a time series is and is not a random walk.
How to make predictions for a random walk.  .... " 

Testing Time Series Data

Very well done, and thoughtful and technical piece on testing a time series. Have heard of backtesting, but not hindcasting.   And more specifically its about making  forecasts in time.   What is generally known as forecasting.

 How To Backtest Machine Learning Models for Time Series Forecasting
by Jason Brownlee on December 19, 2016 in Time Series

The goal of time series forecasting is to make accurate predictions about the future.

The fast and powerful methods that we rely on in machine learning, such as using train-test splits and k-fold cross validation, do not work in the case of time series data. This is because they ignore the temporal components inherent in the problem.

In this tutorial, you will discover how to evaluate machine learning models on time series data with Python. In the field of time series forecasting, this is called backtesting or hindcasting ... "