New answers tagged reinforcement-learning
0
votes
Where would I start if I wanted to create an AI agent to play a 2d game?
Theory
Learn the basic Reinforcement Learning (RL) concepts: Agent, Environment, Action, Reward, Return, Episode, Exploration. Maybe look at Q-Learning, but you don't need to dive deep yet.
Check out ...
- 126
0
votes
Where would I start if I wanted to create an AI agent to play a 2d game?
You can start with Monte Carlo Tree Search to play a 2D game. It is a pretty common algorithm in Reinforcement Learning. If you are comfortable enough with Neural Networks, you can use AlphaZero to ...
0
votes
Where would I start if I wanted to create an AI agent to play a 2d game?
Basic knowledge in machine learning does typically not involve reinforcement learning, which is an ideal choice to make an agent in a video game. Reinforcement learning is famous for beating the ...
- 176
2
votes
Accepted
Why is soft actor critic an off policy scheme?
SAC is an off-policy method because it learns from a replay buffer, which contains experiences collected by the agent over time from potentially different versions of the policy. This means the agent ...
1
vote
Why is Soft Q Learning not an Actor Critic method?
Indeed SQL is very similar to actor-critic method which has a soft Q-function critic network with parameter $\theta$ and an actor policy network with parameter $\phi$, and in fact the paper "...
- 561
1
vote
How does the discount factor (gamma) and memory replay work in Deep Q-learning?
Let's try to understand what actually happens when learning. You start with random values for Q(s,a), the function that estimates the reward of an action given a state s, and you then get an immediate ...
- 111
0
votes
A2C: Why do episode rewards reset?
I ran into this and learned that not only is observation normalization is as important as reward normalization.
The y-axis is showing this env reward is much higher than 1.0
Try rescaling your env ...
- 101
1
vote
It is not clear why sequential improvement is violated in the constructed rollout algorithm
The first sentence simply asserts the base heuristic accidentally produces an optimal control sequence, otherwise the rollout algo starting from initial state $x_0$ would not be strictly inferior to ...
- 561
2
votes
Accepted
Time index in TD(0) return
TD(0) is essentially an iterative online algo and your above one-step return TD target is used for TD error to further update value estimation of state $s_t$ at time step $t$, thus the one-step return ...
- 561
1
vote
Confusion in subscript for n-step TD(0)
n-step TD(0) is an extension of TD(0) that uses a sequence of forward n rewards and estimated state values to update the value estimate of the current state $x_k$, and TD(0) and Monte Carlo prediction ...
- 561
2
votes
Accepted
What is the difference between "base heuristic" and a "rollout algorithm" based on this base heuristic?
As illustrated under your above flowchart figure 2.4.2, base heuristic/policy is just a simple heuristic that is used to help make a quick decision about what action to take in a given state $x_k$ via ...
- 561
1
vote
Are platformer games, with the camera centred on the character, examples of egocentric vision?
At the time, this probably referred to what we call an POV or first person view. It was applied to head mounted cameras. This term was developed a long time ago to describe something that didn't have ...
- 111
0
votes
DQN with experience history to learn from already saved - which reward should I take?
More specifically than off-policy RL, you are looking at offline reinforcement learning techniques. In offline RL, all training data is known beforehand (stationary), which is in stark contrast to the ...
- 1,122
0
votes
Accepted
How to structure the input data for non-vision deep reinforcement learning?
Does it make a difference how I structure that data?
Since your gym environment creates two values, t and h at one single step, ...
- 1,007
0
votes
How the proof of the contraction of variance for distributional Bellman operator follows
The confusing part is how the variance is defined in terms of $P^{\pi}Z(x, a)$, which includes three sources of randomness, $X', A', Z$. However, if we define the variance as
\begin{align*}
\mathbb{V}(...
- 41
0
votes
How the proof of the contraction of variance for distributional Bellman operator follows
This is a common trick in reinforcement learning literature which uses the law of large numbers to use the sampled variables $X$ and $A$ instead of $x$ and $a$. Let's say we know the probability $p(x)$...
- 287
0
votes
What should I do, reinforcement learning agent gives different result on every train?
I suggest you should try fixing the seeds. Here's the code how to do it, if you are using torch and the environment is inherited from gym
...
- 1,007
0
votes
Why does $\alpha=1$ mean batch MC Learning?
I did not read the link but I am giving a standard derivation as it can be found in Sutton-Barto for instance.
The average return formula can be reformulated:
\begin{align}
V_{n+1}(s) &= \frac{1}{...
- 287
Top 50 recent answers are included