0
$\begingroup$

In time Series prediction, we have a stream of vectors. There are different approaches for accounting for the temporal patterns between these vectors.

There's two that I'm considering. An LSTM or augmenting the feature space. What's the difference between the two? The most obvious to me is that an LSTM is more expressive and can get superior accuracy if modelled properly.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

LSTM is a neural network which learns for an input x an output y. In additional to CNNs or MLPs it considers a hiddenstate h (which is influenced by prvious inputs) when your next input x is feed into the network.

Augmenting the feature Space is a technique which you do previous training your LSTM (to augment your data set in order to generatre more data and let the LSTM better generalize to new data). In the field of image recognition you can rotate your images by 40 degree to generate a new one. This process is known as data augmentation. Such methods also appliable to time series.

In summary: first, you start with augmenting your input feature space in order to improve prediction accuracy and then training your LSTM with the augmeneted training data set.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I don't think this answers the question. $\endgroup$
    – echo
    Apr 9, 2018 at 19:20
  • 1
    $\begingroup$ This answer first distinguish and relates the two terms LSTM and feature space augmentation. For a more detailed answer please give more information (for instance a paper to each term). $\endgroup$
    – user3352632
    Apr 10, 2018 at 7:16
  • 1
    $\begingroup$ Everything @user3352632 says is correct and makes sense though :p $\endgroup$
    – Andreas Storvik Strauman
    Apr 11, 2018 at 7:01
0
$\begingroup$

I just read this in a recent Bengio paper and it's pretty obvious. He says that there are zero differences between a short-term memory and an augmented feature space. However, if you want to capture long-term dependencies without blowing up the feature space, you'd want to use an LSTM because traditional approaches can't dynamically learn what to "remember".

Improve this answer
$\endgroup$
12
  • $\begingroup$ Correct! However, the Elman-RNN (vanilla) can in theory learn long term dependencies as well as an LSTM, as Bengio also has a paper on. The problem is that it is very sensitive to the hyper parameters and thus they are very hard to tune. $\endgroup$
    – Andreas Storvik Strauman
    Apr 8, 2018 at 21:12
  • $\begingroup$ No, wait what? What do you mean by "augmented feature space"? $\endgroup$
    – Andreas Storvik Strauman
    Apr 8, 2018 at 21:13
  • $\begingroup$ Vector concatenation: <Vector T>.<Vector T-1>.<Vector T-2> $\endgroup$
    – echo
    Apr 8, 2018 at 21:49
  • 1
    $\begingroup$ No. Maybe. I don't understand how you can compare concatenating vectors and LSTM? $\endgroup$
    – Andreas Storvik Strauman
    Apr 11, 2018 at 6:02
  • 1
    $\begingroup$ By LSTM you mean long short term memory recurrent neural network, right? $\endgroup$
    – Andreas Storvik Strauman
    Apr 11, 2018 at 7:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .