Questions tagged [dqn]

For questions related to the deep Q-network (DQN), which is a deep neural network (e.g. a convolutional neural network) trained with a variant of Q-learning. The expression was coined in the paper "Playing Atari with Deep Reinforcement Learning" (2013) by Google's DeepMind.

Filter by
Sorted by
Tagged with
0
votes
0answers
27 views

What are the implications of storing the alternative situation (that could have been experienced) in the replay buffer?

Consider an environment where there are 2 outcomes (e.g. dead and alive) and a discrete set of actions. For example, a game where the agent has 2 guns $A$ and $B$ to shoot a monster (the monster dies ...
0
votes
0answers
43 views

Off-policy full-random training in easy-to-explore environment

Let say we are in an environment where a random agent can easily explore all the states of an environment (for example: tic-tac-toe). In those environments, using off-policy algorithm, is it a good ...
0
votes
0answers
27 views

Pytorch and keras ddqn seem identical, only keras learns

I followed a tutorial for ddqn to beat pong, it beats it with a perfect score in keras, but trying to translate it to pytorch it doesn't learn at all. What am I missing? I pasted all the code for each ...
0
votes
0answers
21 views

Reward and loss follow the same shape in DQN

If the accumulated reward increases, the loss increases and vice versa. This is a strange behaviour. See the figure below for an example. What is the possibility of having this behaviour in DQN? I ...
0
votes
0answers
34 views

Why do we use the Target Network for action evaluation in Double deep Q networks

Is there any specific reason as to why The target Network is used for evaluation and The online network Is used for selection, what would be the difference if both roles were switched, our online ...
0
votes
0answers
38 views

Having trouble understanding how Double deep Q networks work

I’ve looked at various articles and I’m still very confused, I understand the normal double Q learning about having two Action value estimates that use two different set of samples But coming to ...
0
votes
0answers
58 views

Why does my agent always takes a same action in RL?

I'm trying to reproduce the work in the paper Demand Response for Home Energy Management Using Reinforcement Learning and Artificial Neural Network. I want to optimize the power consumption for home ...
0
votes
0answers
35 views

How to define image as observation in tensorflow_agent?

I want to use the DQN algorithm from TensorFlow Agent API for a custom environment. In my environment, the agent's observations are image sequences. As far as I know, images should be encoded by a ...
0
votes
0answers
43 views

What does the notation “for t=T to 1,−1 do” in terms of time steps, in deep recurrent q network?

In looking at an algorithm in the paper Learning to Communicate with Deep Multi-Agent Reinforcement Learning. Here is the full algorithm: What does the notation ...
0
votes
0answers
35 views

DQN Tic-Tac-Toe does not quite become optimal

I am trying to implement a DQN agent for playing standard 3x3 Tic-Tac-Toe (it is a double DQN with experience replay, and using a target network). I got the hyperparameters to the point where the ...
0
votes
1answer
48 views

What reinforcement learning algorithm should I use in continuous states?

I want to use reinforcement learning in an environment I made. The exact environment doesn't really matter, but it comes down to this: The amount of different states in the environment is infinite e.g....
0
votes
0answers
35 views

Why doesn't my double deep Q network trained with the same training set give consistent performance?

I've written a Double DQN which can do either 1-step or multi-step learning. It also has a prioritised reply buffer. The internal network is an LSTM. My input data is a series of time series data and ...
0
votes
0answers
36 views

My Double DQN with Experience Replay produces a no-action decision most of the time. Why?

I've written a Double DQN-based stock trading bot using mainly time series stock data. The internal network of the Double DQN is a LSTM which handles the time series data. An Experience Replay buffer ...
0
votes
0answers
86 views

Should the network weights converge when training Deep Q networks?

I have two sets of data, one training and one test set. I use the train set to train the deep q network model variant. I also continuously evaluate the agent Q values obtained on the test set every ...
0
votes
0answers
135 views

How to choose hyperparameters in double DQN?

I'm looking for some indications about the tuning of hyper-parameters in building my double DQN. I have a time series problem (with about 2000 observations and no terminal state, I have to max the ...
0
votes
0answers
18 views

How is the parameterised server updated in distributed DQN?

In this paper about Massively Parallel Methods for Deep Reinforcement Learning, the parallelisation of DQN is done via separating the actors and learners. Multiple actors carry out the $\epsilon$ ...
0
votes
0answers
56 views

Reinforcement learning random agent always performing the same few actions

I have a DQN model which has 3 features as inputs (properly normalized) and should output q-values for each of the 100 possible actions. However, prior to any training, I would like to examine the ...
0
votes
0answers
26 views

DQN is unable to learn from image data

I am trying to write a DQN model that will be able to solve OpenAI gym CartPole environment. I successfully managed to do it using scalar observation data that ...
0
votes
0answers
37 views

How should I define the loss function when using DQN to estimate the probability density?

I'm doing a Deep Q-learning project. All of my rewards are positive and there are two terminal states. One of them has a zero reward and the other has a high positive reward. The rewards are ...
0
votes
0answers
54 views

DQN unlearns certain OpenAI-Gym environments

I solved the OpenAI-Gym MountainCar-v0 environment using dqn(using low-state-dimensional input). When I used the same code for solving CartPole-v0 environment, the network got trained in the reverse ...
0
votes
1answer
57 views

Encourage Deep Q to seek short-term reward

I understand that gamma is an important factor in determining the rewards for a deep Q agent, however during testing of my network I am noticing that the agent is outputting more actions to "do ...
0
votes
0answers
38 views

Next step after lane detection in vehicle automation

I have an RC car with a camera, I have implemented so that i can detect lanes on my track (think like a nascar track). I want to get this car to be able to go around the track autonomous. But I am ...
0
votes
0answers
318 views

DQN Q-mean values converge negatively

I'm trying to implement my own DQN. So far I think my code is good, but my Q-values (I'm getting the mean of all the values for every episode) tends to converge near-zero but negatively. It is normal? ...
0
votes
0answers
116 views

Having trouble solving Pong. My model is not improving

Im trying to solve pong by a DQN approach. These are my hyper parameters: ...
-1
votes
1answer
449 views

Problem over DQN Algorithm not converging on snake

I'm using a DQN Algorithm to play Snake. The input of the neural network is a stack of 4 images taken from the games 80x80. The output is an array of 4 values, one for every direction. The problem ...

1 2 3 4
5