The initial environment state is 0.25. Each time step the agent performs a discrete
1. If action is
1, then the new state will be
state + 0.1. If action is
0, the new state will be
state - random() * 0.2. The reward is
state - 0.5, however if
state > 0.98 (or
state < 0) the agent dies (with no reward).
First question: How do I teach the agent not to be too greedy? How to verify that the agent learned?
Main question: How to reduce the number of trials (i.e. the number of episodes) before the agent learns?
I would also appreciate any relevant references.
It took 1000 episodes of max 2000 timesteps, which is unacceptable for me (I wish to drastically reduce the number of episodes and timesteps).
The behavior is far from optimal. Ideally, the agent should choose action
0only if the state is larger than 0.88 (or something below that and within a small interval such as 0.01). [Edit] However, the threshold is 0.75, that forces the agent to choose
0even if it could safely choose
1, e.g. following 0.8 -> 0.76 -> 0.75 -> 0.74 trajectory before choosing