Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
There are different kinds of machine learning algorithms, both univariate and multivariate, that are used for time series forecasting: for example ARIMA, VAR or AR.
Why is it harder (compared to classical models like ARIMA) to achieve good results using neural network based algorithms (like ANN and RNN) for multi step time series forecasting?
ANNs & RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of hyper-parameter optimization. Because neural networks are so flexible, it is not clear, at the outset, which arrangement of neurons will be most effective to solve a given problem. It is also not clear how fast the network should learn from new signals, what sorts of activation functions to use in the different layers of the network, and which of several possible regularization methods might be best. Making these decisions well requires either years of practice and experience, or a lot of trial and error (or, maybe both!).
In contrast, a regression-based method like ARMA will typically have just a couple of simple hyperparameters, each of which has a clear, intuitive, meaning. This means that an untrained practitioner can probably get an ARMA result that is close to the result of a trained practitioner using ARMA.
Essentially: neural networks are brittle and sensitive to the choice of hyper-parameters, while regression generally is not.
Given the (usual) higher architectural complexity of ML models compared to more classical forecasting models, ML models might also require more data, otherwise they might just overfit the training dataset.
Furthermore, online learning (or training) of a neural network using stochastic gradient descent (that is, one example at a time) might also be numerically unstable and statistical inefficient (so convergence might be slow). See Towards stability and optimality in stochastic gradient descent for more details and a solution (AI-SGD).
