# LSTM in reinforcement learning

Please tell me that is the LSTM network for the problem of reinforcement learning, as I explain to her what she will get the reward of a prediction, because the output will contain only actions?

Well, well, let's say at first I can play and upload my actions to training so that she sees which actions are right, that is, which she should strive for, but how to make her learn relatively independently?

• Hi Alex and welcome to this community! We're glad to help. However, it is not clear what you're asking. Can you please write your question clearly in your native language and then use e.g. https://www.deepl.com/translator or Google Translate to translate your question to English, then edit your current post with that translation? – nbro Aug 13 '19 at 21:22
• Also useful to translate, re-translate back, then re-retranslate to determine how much of the content is being translated accurately. – DukeZhou Aug 14 '19 at 19:39

You can use LSTM in reinforcement learning, of course. You don't give actions to the agent, it doesn't work like that.

The agent give actions to your MDP and you must return proper reward in order to teach the agent. For example if you implement trading bot, the policy(policy=the agent, which is your LSTM network) will say that at step T it is going to have action 34, which means something to your MDP and you return reward for example -0.03 or +0.05 or whatever depending what that actions is doing at the moment T.

So I get the question like you want to do a supervised learning on a reinforcement learning environment.

You can mimic supervised learning as well, but the idea of reinforcement learning is not that.

Here is how to mimic:

Scenario: you are at step T, lets say you have 3 possible actions -1,0,+1;

In a supervised learning you must give the desired action to the learning process. In reinforcement learning you must give reward based on if you are happy or not from the agent's action.

So you must have predefined that for -1 you are not happy and you give reward 0.0, for action 0 you are not happy and you give reward 0.0 and for action +1 you are happy and you give reward +100;

I hope this makes things clear.

• Here's a question on your penultimate paragraph, I can't understand one thing, lstm gives out 3 actions, and how to me reward to give the neural network, the fourth output parameter or something? – alex-rudenkiy Sep 20 '19 at 18:27
• Its not that simple. You need a framework that makes everything for you. Otherwise you will need to reinvent the wheel. First value of the output is called "value" and should be with activation "identity" function. This is how the reinforcement learning works. Sometimes it can be made of many networks, not just one. I use computational graph from dl4j framework. Many reinforcement learning introduce the notion of value-function which often denoted as V(s) . The value function represent how good is a state for an agent to be in. – Borislav Markov Sep 21 '19 at 22:25
• I see you are interested in keras and python. Take a look at this framework :github.com/keras-rl and the answer to your question is at this row, look how the author has put first output as V(s;theta) and then comes all the actions as one hot vector: github.com/keras-rl/keras-rl/blob/master/rl/agents/dqn.py#L122 – Borislav Markov Sep 21 '19 at 22:34