1
$\begingroup$

I’m looking for some help with my neural network. I’m working on a binary classification on a recurrent neural network that predicts stock movements (up and down) Let’s say I’m studying Eur/Usd, I’m using all the data from 2000 to 2017 to train et I’m trying to predict every day of 2018.

The issue I’m dealing with right now is that my program is giving me different answers every time I run it even without changing anything and I don’t understand why?

The accuracy during the train from 2000 to 2017 is around 95% but I’ve noticed another issue. When I train it with 1 new data every day in 2018, I thought 2 epochs was enough, like if it doesn’t find the right answer the first time, then it knows what the answer is since the problem is binary, but apparently that doesn’t work.

Do you guys have any suggestion to stabilize my NN?

Improve this question
$\endgroup$
2
  • $\begingroup$ do you think stock movements is a mathematical function? $\endgroup$
    – user8426627
    Commented Jun 19, 2019 at 0:05
  • $\begingroup$ I’m obviously not gonna get 80% of accuracy, and that’s not my goal anyway, I’m just trying to stabilise the prediction for now $\endgroup$
    – neomatriciel
    Commented Jun 19, 2019 at 7:53

2 Answers 2

Reset to default
1
$\begingroup$

Firstly, dealing with the issue that the program gives different answers every time without making any changes can be due to a couple of things.

  • Assigning random values to weights and bias. This can be solved by setting a seed manually at the start of the program.
  • Make sure you have set the model to the testing mode after training. For some frameworks, this has to be done manually.

Secondly, regarding your expected results.

To generate a proper accuracy metric, you will have to sample your dataset into training and testing data, making sure there is now overlap between them. This might be an issue as you have stated training on data till 2017 and then again training on data of 2018.

Lastly, don't expect that the model will know that the output is wrong and directly change it because it's binary classification. This is not how neural networks work. The model fits the solution better by gradually updating its weights and biases over a number of iterations. So it will take a number of epochs to learn new trends in the data for 2018.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I thought about the random weights but I don't think that's the problem here, let me explain : I ran my code 15 times in a row without changing anything with 150 epochs for the big training until 2017, I checked everything on tensorboard and the loss is decreasing going slowly to 0 while the accuracy is increasing going slowly to 1, results are that I get 20% of variation on my correct predictions Then I ran the same test with only 15 epochs, around 0.55 accuracy and a big loss and I only got around 1% of variation on my correct predictions $\endgroup$
    – neomatriciel
    Commented Jun 20, 2019 at 10:42
  • $\begingroup$ I tried 100 epochs for the trainings on 2018's everyday data and still, some of them get 0 accuracy $\endgroup$
    – neomatriciel
    Commented Jun 20, 2019 at 10:44
  • $\begingroup$ As you have realized, it's true that the accuracy increases as the model is trained over a number of epochs. This is true till the model starts to overfit. $\endgroup$
    – skillsmuggler
    Commented Jun 20, 2019 at 20:10
  • $\begingroup$ Regarding the training for 2018 data, the most probable cause of this loss in accuracy is the fact that there is not enough data for the model to train on. In the first case, you have 356 days * 18 years data points, but of 2018 you seem to have only 365 days data points. $\endgroup$
    – skillsmuggler
    Commented Jun 20, 2019 at 20:13
  • $\begingroup$ Would it be a better idea to train a single NN for every day of 2018 ? So I would have to train « 365 » ? $\endgroup$
    – neomatriciel
    Commented Jun 25, 2019 at 9:04
0
$\begingroup$

This may not be directly answering your question, but predicting market movement based on past prices is probably not very sensible.

Assuming that future samples are drawn from the same populations as the past samples basically violates the founations of AIML and statistics quite frankly. See relevant figure below.

As far as accuracy goes, its is all relative. If you have a cpu which is right 0.999 of the time you have a useless piece of silicon, but if you have a 0.501 accuracy on stock market ID then you are the richest man in the world. That said, stock historical data is just not a phenomenon that repeats itself based on its own underlying distribution.

Always remember that when it comes to markets, past performance is not a good predictor of future returns—looking in the rear-view mirror is a bad way to drive. Machine learning, on the other hand, is applicable to datasets where the past is a good predictor of the future.

— Deep learning with python, Francois Chollet

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Rather than an image, can you please copy and paste the relevant section of the book that you want to show or emphasize? $\endgroup$
    – nbro
    Commented Mar 15, 2021 at 11:30
  • $\begingroup$ @nbro sorry, unless this is a personal request can you please refer me to relevant SE policy rule on this? $\endgroup$
    – hH1sG0n3
    Commented Mar 15, 2021 at 12:13
  • 1
    $\begingroup$ Please, see this, this and this. Essentially, in your case, text is preferable because it improves e.g. the searchability of your post. $\endgroup$
    – nbro
    Commented Mar 15, 2021 at 12:30
  • $\begingroup$ Text is preferable to improve the searchability of the post sounds a like a reasonable suggestion for post improvement, will do asap. $\endgroup$
    – hH1sG0n3
    Commented Mar 15, 2021 at 12:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .