# Experiment shows that LSTM does worse than Random Forest… Why?

LSTM is supposed to be the right tool to capture path-dependency in time-series data.

I decided to run a simple experiment (simulation) to assess the extent to which LSTM is better able to understand path-dependency.

The setting is very simple. I just simulate a bunch (N=100) paths coming from 4 different data generating processes. Two of these processes represent a real increase and a real decrease, while the other two fake trends that eventually revert to zero.

The following plot shows the simulated paths for each category:

The candidate machine learning algorithm will be given the first 8 values of the path ( t in [1,8] ) and will be trained to predict the subsequent movement over the last 2 steps.

In other words:

• the feature vector is X = (p1, p2, p3, p4, p5, p6, p7, p8)

• the target is y = p10 - p8

I compared LSTM with a simple Random Forest model with 20 estimators. Here are the definitions and the training of the two models, using Keras and scikit-learn:

# LSTM
model = Sequential()
history = model.fit(train_X_LS, train_y_LS, epochs=100, validation_data=(vali_X_LS, vali_y_LS), verbose=0)

# Random Forest
RF = RandomForestRegressor(random_state=0, n_estimators=20)
RF.fit(train_X_RF, train_y_RF);


The results are the summarized by the following scatter plots:

As you can see, the Random Forest model is clearly outperforming the LSTM. The latter seems to be not able to distinguish between the real and the fake trends.

### How would you modify the LSTM model to make it better at this problem?

Some remarks:

• The data points are divided by 100 to make sure gradients do not explode
• I tried to increase the sample size, but I noticed no differences
• I tried to increase the number of epochs over which the LSTM is trained, but I noticed no differences (the loss becomes stagnant after a bunch of epochs)
• You can find the code I used to run the experiment here
• With my limited knowledge I belive you need to set a larger amount of neurons for the LSTM network as they are currently set to 1 which would produce a 1-dimensional space with limited learning capacity regardless of inputs / outputs required – benbyford Apr 9 '19 at 15:03