To answer your question, the specifics of some of the OpenAI Gym environments can be found on their wiki:
The episode ends when you reach 0.5 position, or if 200 iterations are reached.
There is a deeper question in what you asked, though:
My initial understanding was that an episode should end when the Car reaches the flagpost.
The environment certainly ...
The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the _max_episode_steps attribute of the environment.
There is a really small mistake in here that causes the problem:
for index, (current_state, action, reward, next_state, done) in enumerate(minibatch):
if not done:
new_q = reward + DISCOUNT * np.max(future_qs_list) #HERE
new_q = reward
# Update Q value for given ...
This is a case of overfitting the Q function leading to compounding errors when selecting actions.
You have been training your neural network as function approximator for too long on the same data distribution, so the neural network loses it's ability to generalize and slowly starts overfitting, i.e. learns the data exactly as it is or at least very closely....
What I was looking for is multi-agent RL, where I have multiple RL agents, each controlling actions of one user. All RL agents/user make an action in each environment step and each get their own reward.
I represent my RL agents' actions as dict, containing the RL agent ID as key and its action as value. The different agents may either use the same or a ...
After checking the Internet, you will probably find several resources such as
Try to understand the principles first (see above). After some reasonable amount of coding you can adapt OpenAI gym.
The question is conceptually wrong, because of misunderstanding of area. Explanation: The idea is to replace open ai gym by something different. For example: web-site or computer game. There is no way to create an environment based on image. If you want to use implemented algorithm for open ai gym and want to change environment for your own, could do ...
I had to change the actions selection function for this and tune some hyper-parameters. Here's what I did to make it converge:
Sampled the noise from a standard normal distribution instead of sampling randomly.
Changed the polyak constant (tau) from 0.99 to 0.001 (I didn't have an idea of what it should be, so I had just set it randomly in the first try)
I don't recommend changing the rules of the environment.
What you could do:
Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation.
e.g. Say I do have a continuous state space with one variable in range $[-\infty,\infty]$
I guess it would always be better if you can reuse existing environments to make it work for yourself. Since most of the environment codes is anyway opensourced, you can always edit it to your liking.
If you want a custom environment, you can add an environment to gym like this.