Search Results

Rather than flattening your sub-action spaces into an action space that consists of very many primitive actions and having a DQN provide an output value for the many different possibilities, instead you … Where a DQN would be a function $f : \mathcal{S} \rightarrow \mathbb{R}^{|\mathcal{A}|}$, the decoupled network would be $N$ functions $f_i : \mathcal{S} \rightarrow \mathbb{R}^{|\mathcal{A}_i|}$. …

A reinforcement learning algorithm is considered model based if it uses estimates of the environments dynamics to help learn. For instance, in the Tabular Dyna-Q algorithm, every time you visit a stat …

As per OP's request in the comments, here is a snippet of code I used for Car-pool. class DQN: def __init__(self, observation_space, action_space): self.exploration_rate = epsilon_max self.observation_space …

for the sampled experience from a replay buffer in DQN. … Assuming we sample uniformly at random as in vanilla DQN, then the probability of the point never being seen is 0.368. …

I would recommend looking at Deep Q-Learning.

In any reinforcement learning problem, not just Deep RL, then there is an upper bound for the cumulative reward, provided that the problem is episodic and not continuing. If the problem is episodic an …

The way the states are used is as follows: Typically your $Q$-network will state a state as input and output scores over the action space. I.e. $Q : \mathcal{S} \rightarrow \mathbb{R}^{|\mathcal{A}|}$ …

It is not so much the problem of using Reinforcement Learning to train the neural networks, it is the assumptions made about the data given to standard Neural Networks. They are not capable of handlin …

I will first explain briefly to you the difference between supervised learning and reinforcement learning to make sure that you don't have any misunderstandings. In supervised learning you are provide …

In the Human Level Control paper where the DQN gained its popularity, the network is a little different to the tabular function. …

The loss function that you minimise in DQN is $$ L(\theta) = \mathbb{E}_{(s,a,r,s')\sim U(D)}\left[\left( r + \gamma \max_{a'}Q(s', a'; \theta^-) - Q(s, a; \theta)\right)^2 \right]\;$$ where $U(D)$ denotes …

As you say, the output of a $Q$ network is typically a value for all actions of the given state. Let us call this output $\mathbf{x} \in \mathbb{R}^{|\mathcal{A}|}$. To train your network using the sq …

The overestimation comes from the random initialisation of your Q-value estimates. Obviously these will not be perfect (if they were then we wouldn't need to learn the true Q-values!). In many value b …

I'm not familiar with the ins and outs of self-driving cars, but I imagine that the action space is not discrete. For instance, the car may want to decide what angle it needs to turn (rather than left …

In DQN that was presented in the original paper the update target for the Q-Network is $\left(r_t + \max_aQ(s_{t+1},a;\theta^-) - Q(s_t,a_t; \theta)\right)^2$ were $\theta^-$ is some old version of the …