# Tag Info

4

OpenAI's Gym is a standardised API, useful for reinforcement learning, applied to a range of interesting environments many of which you can then access for free with little effort. It is very simple to use, and IMO worth learning if you want to practice RL using Python to any depth at all. You could use it to ensure you have good understanding of basic ...

3

Parameters is a synonym for weights, which is the term most people use for a neural networks parameters (and indeed in my experience it is a term that machine learners will use in general whereas parameters is more often found in statistics literature). Batch size, learning rate etc. are hyper-parameters which basically means they are user specified, whereas ...

3

No, there is no file type associated with AI projects in general. Your examples of Photoshop and Excel are specific corporate branded products. These store bespoke data that only works with those products (plus maybe a few converters that can read the files for competitor products). Even more general examples such as .jpg for images or .txt for text ...

3

You may be very interested to know that there was a bug in the v2 Lidar tracing, making the agent think there were phantom objects, and sometimes intersecting with its own legs: https://github.com/openai/gym/pull/1789 Finding this bug makes me even more impressed anyone has solved BipedalWalkerHardcore-v2 - it seems the observations from lidar have been ...

2

Since the environment has some randomness in it, purely memorizing a trajectory to victory will not work. You will have to memorize every single trajectory for that to work, and there are an infinite number of them. So, you will need to add some sort of bias to your learning model - i.e., what to do when the observations in your pickle file don't match the ...

2

What's exactly the point of time.sleep() in this code? I don't really understand it, you're simply stopping the execution of the program for $0.01$ seconds, how will that affect the simulator in any way ? It's not running in parallel, it does one step of the simulation when you call env.step function and returns the next state and reward. Calling sleep ...

2

There is a really small mistake in here that causes the problem: for index, (current_state, action, reward, next_state, done) in enumerate(minibatch): if not done: new_q = reward + DISCOUNT * np.max(future_qs_list) #HERE else: new_q = reward # Update Q value for given ...

2

The main point in GPT-3 and already in 2 was the observation that performance was steadily increasing with increasing model size (As seen in Figure 1.2 in your linked paper). So it seems that while all progress made in NLP was definitely useful, it also is important to just scale up the model size. This may not seem like surprising point, but it actually ...

1

Is it training at all? Or is agent performance not improving over time? Q learning can be pretty unstable. I would recommend logging the sum of rewards received by the agent at the end of each episode and the model loss to help in the debugging process. The sum of rewards will show you if the agent is improving over time and the model loss will give you a ...

1

When it comes to GPU usage, nvidia-smi shows the usage at the time it was executed. You should try running watch -n0.01 nvidia-smi to see the usage of GPU every 0.01 second. It should output some small usage for current model, like 5%. You could try to increase you model, to e.g. self.d1 = Dense(1024, input_shape=(input_size,), activation="relu") ...

1

The question is conceptually wrong, because of misunderstanding of area. Explanation: The idea is to replace open ai gym by something different. For example: web-site or computer game. There is no way to create an environment based on image. If you want to use implemented algorithm for open ai gym and want to change environment for your own, could do ...

1

I don't recommend changing the rules of the environment. What you could do: Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation. e.g. Say I do have a continuous state space with one variable in range $[-\infty,\infty]$ ...

1

The video you linked is not using reinforcement learning (RL). It is using genetic algorithms (GA). GA is designed around using multiple agents and picking the best performing to move forward to next generation. With this approach, it is common to want to only view the best performing agents, as the learning mechanism uses the same selection process - the ...

1

I've actually implemented this game before using deep reinforcement learning. You are dealing with a dynamic action space here, where the action space may change at each time step of the game (or more generally the MDP). First, let's discuss the actual action spaces in each one of the two phases of Crib (or Cribbage) and formalize the question. Phase 1: The ...

1

In the DRL nanodegree in Udacity, the instructor says it is possible to combine on- and off-policy learning and suggests the following paper where this has been done: Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic (ICLR 2017). Citing the paper: The core idea is to use the first-order Taylor expansion of the critic as a control ...

Only top voted, non community-wiki answers of a minimum length are eligible