# Tag Info

### Where would I start if I wanted to create an AI agent to play a 2d game?

Theory Learn the basic Reinforcement Learning (RL) concepts: Agent, Environment, Action, Reward, Return, Episode, Exploration. Maybe look at Q-Learning, but you don't need to dive deep yet. Check out ...
• 126

### Where would I start if I wanted to create an AI agent to play a 2d game?

You can start with Monte Carlo Tree Search to play a 2D game. It is a pretty common algorithm in Reinforcement Learning. If you are comfortable enough with Neural Networks, you can use AlphaZero to ...

### Where would I start if I wanted to create an AI agent to play a 2d game?

Basic knowledge in machine learning does typically not involve reinforcement learning, which is an ideal choice to make an agent in a video game. Reinforcement learning is famous for beating the ...
• 176
Accepted

### Why is soft actor critic an off policy scheme?

SAC is an off-policy method because it learns from a replay buffer, which contains experiences collected by the agent over time from potentially different versions of the policy. This means the agent ...
1 vote

### Why is Soft Q Learning not an Actor Critic method?

Indeed SQL is very similar to actor-critic method which has a soft Q-function critic network with parameter $\theta$ and an actor policy network with parameter $\phi$, and in fact the paper "...
• 561
1 vote

### How does the discount factor (gamma) and memory replay work in Deep Q-learning?

Let's try to understand what actually happens when learning. You start with random values for Q(s,a), the function that estimates the reward of an action given a state s, and you then get an immediate ...
• 111

### A2C: Why do episode rewards reset?

I ran into this and learned that not only is observation normalization is as important as reward normalization. The y-axis is showing this env reward is much higher than 1.0 Try rescaling your env ...
• 101
1 vote

### It is not clear why sequential improvement is violated in the constructed rollout algorithm

The first sentence simply asserts the base heuristic accidentally produces an optimal control sequence, otherwise the rollout algo starting from initial state $x_0$ would not be strictly inferior to ...
• 561
Accepted

### Time index in TD(0) return

TD(0) is essentially an iterative online algo and your above one-step return TD target is used for TD error to further update value estimation of state $s_t$ at time step $t$, thus the one-step return ...
• 561
1 vote

### Confusion in subscript for n-step TD(0)

n-step TD(0) is an extension of TD(0) that uses a sequence of forward n rewards and estimated state values to update the value estimate of the current state $x_k$, and TD(0) and Monte Carlo prediction ...
• 561
Accepted

### What is the difference between "base heuristic" and a "rollout algorithm" based on this base heuristic?

As illustrated under your above flowchart figure 2.4.2, base heuristic/policy is just a simple heuristic that is used to help make a quick decision about what action to take in a given state $x_k$ via ...
• 561
1 vote

### Are platformer games, with the camera centred on the character, examples of egocentric vision?

At the time, this probably referred to what we call an POV or first person view. It was applied to head mounted cameras. This term was developed a long time ago to describe something that didn't have ...
• 111

### DQN with experience history to learn from already saved - which reward should I take?

More specifically than off-policy RL, you are looking at offline reinforcement learning techniques. In offline RL, all training data is known beforehand (stationary), which is in stark contrast to the ...
• 1,122
Accepted

### How to structure the input data for non-vision deep reinforcement learning?

Does it make a difference how I structure that data? Since your gym environment creates two values, t and h at one single step, ...
• 1,007

### How the proof of the contraction of variance for distributional Bellman operator follows

The confusing part is how the variance is defined in terms of $P^{\pi}Z(x, a)$, which includes three sources of randomness, $X', A', Z$. However, if we define the variance as \begin{align*} \mathbb{V}(...

### How the proof of the contraction of variance for distributional Bellman operator follows

This is a common trick in reinforcement learning literature which uses the law of large numbers to use the sampled variables $X$ and $A$ instead of $x$ and $a$. Let's say we know the probability $p(x)$...
• 287