Hot answers tagged

8

You can try the actions yourselves, but if you want another reference, check out the documentation for ALE at GitHub. In particular, 0 means no action, 1 means fire, which is why they don't have an effect on the racket. Here's a better way: env.unwrapped.get_action_meanings()


5

OpenAI's Gym is a standardised API, useful for reinforcement learning, applied to a range of interesting environments many of which you can then access for free with little effort. It is very simple to use, and IMO worth learning if you want to practice RL using Python to any depth at all. You could use it to ensure you have good understanding of basic ...


4

You can try to figure out what exactly does an action do using such script: action = 0 # modify this! o = env.reset() for i in xrange(5): # repeat one action for five times o = env.step(action)[0] IPython.display.display( Image.fromarray( o[:,140:142] # extract your bat ).resize((300, 300)) # bigger image, easy for visualization ) ...


3

To answer your question, the specifics of some of the OpenAI Gym environments can be found on their wiki: The episode ends when you reach 0.5 position, or if 200 iterations are reached. There is a deeper question in what you asked, though: My initial understanding was that an episode should end when the Car reaches the flagpost. The environment certainly ...


3

The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the _max_episode_steps attribute of the environment.


3

There is a really small mistake in here that causes the problem: for index, (current_state, action, reward, next_state, done) in enumerate(minibatch): if not done: new_q = reward + DISCOUNT * np.max(future_qs_list) #HERE else: new_q = reward # Update Q value for given ...


3

You may be very interested to know that there was a bug in the v2 Lidar tracing, making the agent think there were phantom objects, and sometimes intersecting with its own legs: https://github.com/openai/gym/pull/1789 Finding this bug makes me even more impressed anyone has solved BipedalWalkerHardcore-v2 - it seems the observations from lidar have been ...


3

There seems to be no difference between 2 & 4 and 3 & 5. The inconsistency mentioned by Icyblade is due to the mechanics of the Pong environment. "Each action is repeatedly performed for a duration of k frames, where k is uniformly sampled from {2,3,4}" So the action is just repeated a different number of times due to randomness


3

This is a case of overfitting the Q function leading to compounding errors when selecting actions. You have been training your neural network as function approximator for too long on the same data distribution, so the neural network loses it's ability to generalize and slowly starts overfitting, i.e. learns the data exactly as it is or at least very closely....


2

There is no way to tell via the Gym API, and to any RL-based learning agent this is entirely unimportant, discrete actions are just arbitrary labels, and their effects are learned by trial and error in terms of reward and changes to state. If you need to know, you can read the code for each environment. Often it is described in a comment, or as constants. E....


2

I believe if you run a single agent in multiple parallel environments many times you will get similar actions in similar states, the reason behind multiple agents is that you will have different agents with different parameters and you can also have different explicit exploration policies so your exploration will be better and you will learn more from ...


2

It is your choice. This can even be different between training and target system. The approach called "exploring starts" chooses a random start state (and action if you are assessing a deterministic policy for action values). In general, if you don't have a reason to pick exploring starts, you should aim for your env.reset() function to put the environment ...


2

It depends on the task the agent is trying to learn and of course on the environment constrains. In an Atari game agents have a pre-fixed starting point because that's part of the games rules, so I would say that this is enough of a justification to make each simulation start from that starting point. Moreover, you have to pay attention to the kind of ...


2

One way to handle an arbitrarily large sequence is by adding a STOP signal as one possible token in the sequence, just like LSTM. So you could divide your game in turns: What you now call a single action (composed by multiple sub-actions) would become a turn. Now, you can have as many actions you'd like inside a turn. Each action is simply a list ...


2

Since the environment has some randomness in it, purely memorizing a trajectory to victory will not work. You will have to memorize every single trajectory for that to work, and there are an infinite number of them. So, you will need to add some sort of bias to your learning model - i.e., what to do when the observations in your pickle file don't match the ...


2

What's exactly the point of time.sleep() in this code? I don't really understand it, you're simply stopping the execution of the program for $0.01$ seconds, how will that affect the simulator in any way ? It's not running in parallel, it does one step of the simulation when you call env.step function and returns the next state and reward. Calling sleep ...


2

What I was looking for is multi-agent RL, where I have multiple RL agents, each controlling actions of one user. All RL agents/user make an action in each environment step and each get their own reward. I represent my RL agents' actions as dict, containing the RL agent ID as key and its action as value. The different agents may either use the same or a ...


2

Yes, it is the state of the memory; this would mainly involve variables, since the code would be in ROM. Since it is only 128 bytes in size, the screen memory would also not be included in this. The idea is that all information relevant to the game is captured in these 128 bytes; they represent the state of the game world at any given time. Movements of the ...


2

After checking the Internet, you will probably find several resources such as https://github.com/mohammadasghari/dqn-multi-agent-rl https://rlss.inria.fr/files/2019/07/RLSS_Multiagent.pdf https://arxiv.org/abs/2011.00583 and others. Try to understand the principles first (see above). After some reasonable amount of coding you can adapt OpenAI gym. Good ...


1

The question is conceptually wrong, because of misunderstanding of area. Explanation: The idea is to replace open ai gym by something different. For example: web-site or computer game. There is no way to create an environment based on image. If you want to use implemented algorithm for open ai gym and want to change environment for your own, could do ...


1

I had to change the actions selection function for this and tune some hyper-parameters. Here's what I did to make it converge: Sampled the noise from a standard normal distribution instead of sampling randomly. Changed the polyak constant (tau) from 0.99 to 0.001 (I didn't have an idea of what it should be, so I had just set it randomly in the first try) ...


1

I don't recommend changing the rules of the environment. What you could do: Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation. e.g. Say I do have a continuous state space with one variable in range $[-\infty,\infty]$ ...


1

I guess it would always be better if you can reuse existing environments to make it work for yourself. Since most of the environment codes is anyway opensourced, you can always edit it to your liking. If you want a custom environment, you can add an environment to gym like this.


1

Model your problem as an MDP To solve a problem with reinforcement learning, you need to model your problem as a Markov decision process (MDP), so you need to define the state space, the action space, and the reward function of the MDP. Understand your problem and the goal To do define these, you need to understand your problem and define it as a goal-...


1

I've actually implemented this game before using deep reinforcement learning. You are dealing with a dynamic action space here, where the action space may change at each time step of the game (or more generally the MDP). First, let's discuss the actual action spaces in each one of the two phases of Crib (or Cribbage) and formalize the question. Phase 1: The ...


1

The renowned book Reinforcement Learning: An Introduction (2nd edition), by Sutton and Barto, provides a different update rule than your first update rule for policy evaluation. Their update rule is more similar to your second update rule. See section 4.1. They also provide the pseudocode for policy evaluation on page 75 of the book. You can also find the ...


1

I thought about my input-layer. I had the 500 states one hot encoded. So 499 of every input node would be 0. And 0 is very bad in an neural network. I tried the same code with the "CardPole-v0" and it worked. So think about your input guys


1

I'm not sure why you need a continuing environment, but actually you can make most (if not all) OpenAI Gym environments continuing. When you perform a step, you receive information about the next state, the reward, a termination signal and a dictionary with additional information. Simply ignore the termination signal if you want the episodes to continue ...


1

It's not on your end, as a creator of flight simulator, to worry about what action should get the credit for the reward that happened some time after the action was taken. You should return the reward when the actual event happens not when the action that caused it happened. It's the job of the reinforcement learning agent to figure that out. For example if ...


1

https://github.com/openai/retro Current list of machines is Atari Atari2600 (via Stella) TurboGrafx-16/PC Engine (via Mednafen/Beetle PCE Fast) Game Boy/Game Boy Color (via gambatte) Game Boy Advance (via mGBA) Nintendo Entertainment System (via FCEUmm) Super Nintendo Entertainment System (via Snes9x) GameGear (via Genesis Plus GX) Genesis/Mega Drive (via ...


Only top voted, non community-wiki answers of a minimum length are eligible