I want to build a reinforcement learning model, which takes a camera picture as input, that learns online (in terms of machine learning). Based on the position of an object on the camera, I want the model to output an action. That action would be a stepper motor, that either moves to the right or left. This process would be repeated until a given goal/position is reached.

I can't go to the lab at the moment, so I wrote a virtual environment and let the agent live in that.

I am trying a neural network with the cross-entropy function. For small environments, this works fine. However, when I increase the size of the environment, the computation becomes really really slow and the model needs a lot of data input until it starts to learn. Also, it only learns offline. But what I would rather want is a model that learns online and only takes a few tries until it understands the underlying pattern. That isn't a problem for the virtual environment, since I can easily get thousands of data samples. But in the real environment, it would take ages this way.

  • Is there an online reinforcement algorithm that could help me out (instead of training the neural network with the cross-entropy loss function)?
  • $\begingroup$ You may want to look for Heirarchical Q Learning. And of course sending input as a series of frame rather than a single frame would be better. Yet, another improvement can be made by improving the input by removing unnecessary features. For example, your problem might not require colored image as an input so convert your image into grayscale before you move on to learn from it. $\endgroup$ May 14, 2020 at 13:35


You must log in to answer this question.