Price Movement Forecasting Issue

Question

I am working on a project for price movement forecasting and I am stuck with poor quality predictions.

At every time-step I am using an LSTM to predict the next 10 time-steps. The input is the sequence of the last 45-60 observations. I tested several different ideas, but they all seems to give similar results. The model is trained to minimize MSE.

For each idea I tried a model predicting 1 step at a time where each prediction is fed back as an input for the next prediction, and a model directly predicting the next 10 steps(multiple outputs). For each idea I also tried using as input just the moving average of the previous prices, and extending the input to input the order book at those time-steps. Each time-step corresponds to a second.

These are the results so far:

1- The first attempt was using as input the moving average of the last N steps, and predict the moving average of the next 10. At time t, I use the ground truth value of the price and use the model to predict t+1....t+10

This is the result:

Predicting moving average:

On closer inspection we can see what's going wrong:

Prediction seems to be a flat line. Does not care much about the input data:

2. The second attempt was trying to predict differences, instead of simply the price movement. The input this time instead of simply being X[t] (where X is my input matrix) would be X[t]-X[t-1]. This did not really help. The plot this time looks like this:

Predicting differences:

But on close inspection, when plotting the differences, the predictions are always basically 0.

At this point, I am stuck here and running our of ideas to try. I was hoping someone with more experience in this type of data could point me in the right direction.

Am I using the right objective to train the model? Are there any details when dealing with this type of data that I am missing? Are there any "tricks" to prevent your model from always predicting similar values to what it last saw? (They do incur in low error, but they become meaningless at that point).

At least just a hint on where to dig for further info would be highly appreciated.

UPDATE

Here is my config

{
    "data": {
        "sequence_length":30,
        "train_test_split": 0.85,
        "normalise": false,
        "num_steps": 5
    },
    "training": {
        "epochs":200,
        "batch_size": 64
    },
    "model": {
        "loss": "mse",
        "optimizer": "adam",
        "layers": [
            {
                "type": "lstm",
                "neurons": 51,
                "input_timesteps": 30,
                "input_dim": 101,
                "return_seq": true,
                "activation": "relu"
            },
            {
                "type": "dropout",
                "rate": 0.1
            },
            {
                "type": "lstm",
                "neurons": 51,
                "activation": "relu",
                "return_seq": false
            },
            {
                "type": "dropout",
                "rate": 0.1
            },
            {
                "type": "dense",
                "neurons": 101,
                "activation": "relu"
            },
            {
                "type": "dense",
                "neurons": 101,
                "activation": "linear"
            }
        ]
    }
}

Notice the last layer with 101 neurons. It is not an error. We just want to predict the features as well as the price. In other words, we want to predict the price for time t+1 and use the features predicted to predict the price and new features at time t+2, ...

Answer 1

Given a model that takes in a price and a second value, such as a moving average of the price, the following configuration is my recommendation. This is based on training on a history of 45 input time steps and forecasting out 10 steps in the future. I have assumed that your 'num_steps' is a stride through the training data. Note that I have not tested this and you will most likely need to tweak a parameter or two:

{
    "data": {
        "sequence_length": 55,
        "train_test_split": 0.85,
        "normalise": true,
        "num_steps": 5
    },
    "training": {
        "epochs":200,
        "batch_size": 64
    },
    "model": {
        "loss": "mse",
        "optimizer": "adam",
        "layers": [
            {
                "type": "lstm",
                "neurons": 100,
                "input_timesteps": 45,
                "input_dim": 2,
                "return_seq": true,
                "activation": "relu"
            },
            {
                "type": "dropout",
                "rate": 0.2
            },
            {
                "type": "lstm",
                "neurons": 100,
                "activation": "relu",
                "return_seq": false
            },
            {
                "type": "dropout",
                "rate": 0.2
            },
            {
                "type": "dense",
                "neurons": 1,
                "activation": "linear"
            }
        ]
    }
}

Try this out and plot your model's loss and accuracy by epoch. See Display Deep Learning Model Training History in Keras for some information and code in this area. Let me know if you are not using Keras and I will point you to another article/code solution.

Answer 2

Remember that any machine learning model works good only when there is a "rule" or a correlation between modeling data and modeled data. When there is not, even the best algorithm will not predict/classify correctly. I am not saying that this must be the case, but probably you have come pretty close to the physical limit of what can be achieved using this data.

Maybe you should consider adding or extracting new features to your model.