I am trying to deploy a machine learning solution online into an application for a client. One thing they requested is that the solution must be able to learn online because the problem may be non-stationary and they want the solution to track the non-stationarity of the problem. I thought about this problem a lot, would the following work?

  1. Set the learning rate (step-size parameter) for the neural network at a low fixed value so that the most recent training step is weighted more.
  2. Update the model only once per day, in a mini-batch fashion. The mini-batch will contain data from the day, mixed with data from the original data set to prevent catastrophic interference. By using a mini batch update, I am not prone to biasing my model to the latest examples, and completely forgetting the training examples from months ago.

Would this set-up be "stable" for online/incremental machine learning? Also, should I set up the update step so it samples data from all distributions of my predicted variable uniformly so it gets an "even" update (i.e., does not overfit to the most probabilistic predicted value)?

  • $\begingroup$ Is your model going to be a neural network? $\endgroup$
    – nbro
    Apr 16 '19 at 10:02
  • $\begingroup$ Ideally, I wanted it to be because the system is quite complex. Is it easier to do for polynomial / linear models? $\endgroup$
    – Rui Nian
    Apr 16 '19 at 13:36
  • $\begingroup$ Ideally, the model should be trained only with new examples. This might help - datascience.stackexchange.com/questions/12761/…. $\endgroup$ Apr 17 '19 at 11:26
  • $\begingroup$ Hi Supratim, thanks for the reply. However, training with only the new examples cause catastrophic interference, especially for time series processes where states at time t and t + 1 are very similar. Imagine a robot having to learn 3 different tasks, and each task takes 1 week. Whenever the robot is learning one task, because it does it for a straight week and if it learns all new examples only, the model will completely be overfit to the newest data and forget the other tasks. I am trying to build an algorithm similar to adaptive resonance theory, except with cheaper maintenance cost. $\endgroup$
    – Rui Nian
    Apr 17 '19 at 13:43
  • $\begingroup$ Hi Rui, that's a good point. It's an interesting question, and I'm watching it too and hope to see an well-explained answer soon. $\endgroup$ Apr 17 '19 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.