Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [q-learning]

Use for questions that involve Q-learning, where Q is the value of a particular next action among a set of possible actions, based on a specified function of each action and its projected result.

2
votes
1answer
38 views

Reinforcement Learning n-step return

when reading the newest v2 of Sutton + Barto Reinforcement Learning, in Ch7 section 1 about N-step bootsrapping, they write about something they call the "n-step return error reduction property": But ...
0
votes
0answers
24 views

Reinforcement learning: How to combine action and magnitude within a single model?

Say for example you're training an AI-controlled bot (using a Markov Decision Process and a DQN) to clear a basic obstacle course; some obstacles you have to run over, some jump, sit, some squat etc, ...
1
vote
0answers
10 views

Exploration rate decay and training in Q learning

I'm trying to replicate the results of the DeepMind's paper with Breakout included in OpenAI Gym. I wonder how much frames should I keep until I reach the fixed exploration rate. Actually it reaches ...
-1
votes
1answer
53 views

DQN it's not working properly

I'm trying to build a DQN to replicate the DeepMind results. I'm doing with a simple DQN for the moment, but it isn't learning properly: after +5000 episodes, it couldn't get more than 9-10 points. ...
2
votes
1answer
82 views

How to build AI bots for board games like monopoly?

I am trying to build a Q learning-based bot for board games, specifically monopoly. I am fairly new to Q-learning and currently, I have only implemented some bots that can play simple games like Tic-...
0
votes
0answers
42 views

Q-Learning fails to converge even after 50K iterations for a simple board game - What could be the reason for this?

To get a feel of the model-free reinforcement learning, I tried to implement Q-learning for a simple 10 X 10 game board having 4 possible actions where the agent could move either North, East, South, ...
0
votes
1answer
40 views

Is it possible to use a feed-forward neural network to predict the actions in reinforcement learning?

I have done a lot of research on the internet about Reinforcement Learning and I found encountered methods of Reinforcement Learning: Q-Learning and Deep Q-Learning. And I have developed a vague idea ...
1
vote
1answer
45 views

Deep Q-Learning poor convergence on Stochastic Environment

I'm trying to implement a Deep Q-network in Keras/TF that learns to play Minesweeper (our stochastic environment). I have noticed that the agent learns to play the game pretty well with both small and ...
1
vote
0answers
46 views

How does Hindsight Experience Replay learn from unsuccessful trajectories

I am confused by how HER learns from unsuccessful trajectories. I understand that from failed trajectories it creates 'fake' goals that it can learn from. Ignoring HER for now, if in the case where ...
3
votes
1answer
26 views

Reason for issues with correlation in the dataset in DQN

From the paper Human level Control through DeepRL, the correlation in the data causes instability in the network and may causes the network to ...
0
votes
0answers
18 views

How to train a model by accounting for boundary constraints?

I've a robot traverse through a grid layout. Based on the wheel speed difference I classify actions as either straight, left or right. I computed the distances based on the time duration and the speed ...
1
vote
1answer
37 views

Some RL algorithms (especially policy gradients) initialize with random policies, which often manifests as random jitter on spot for a long time?

I am reviewing a statement on the website for ES regarding structured exploration. https://blog.openai.com/evolution-strategies/ Structured exploration. Some RL algorithms (especially policy ...
6
votes
1answer
58 views

Q-Learning the generic maze solution

After doing some exercices on Q-learning for maze solving, I wondered : my q-learning algorithms solve only ONE maze. The AI doesn't learn how to solve mazes, so how can I achieve it ? For instance ...
1
vote
3answers
223 views

difficulty in understanding identifiability in Dueling Network paper

I have difficulty understanding the following paragraph in bracketed in red parentheses in the below excerpts from page 4 to page 5 from the paper Dueling Network Architectures for Deep Reinforcement ...
1
vote
1answer
34 views

Q Llearning for Shortest distance

Is it possible to form a table that will have simply the shortest distance from each source to destination using q learning. If not suggest any other learning algorithm
5
votes
1answer
128 views

Is the discount not needed in a deterministic environment for Reinforcement Learning?

I'm now reading a book titled as "Deep Reinforcement Learning Hands-On" and the author said the following on the chapter about AlphaGo Zero: Self-play In AlphaGo Zero, the NN is used to ...
4
votes
1answer
86 views

Reinforcement Learning (Fitted Q): Qn on Concept & Implementation

I hope to get some clarifications on Fitted Q-Learning ('FQL'). My Research So Far I've read Sutton's book (specifically, chp 6 to 10), Ernst et al and this paper. I know that ...
2
votes
1answer
48 views

Convergence in multi-agent environment

I have a multi-agent environment where agents are trying to optimise the overall energy consumption of their group. Agents can exchange energy between themselves (actions for exchange of energy ...
9
votes
1answer
563 views

Why does DQN require two different networks?

I was going through this implementation of DQN and I see that on line 124 and 125 two different Q networks have been initialized. From my understanding, I think one network predicts the appropriate ...
3
votes
1answer
86 views

Number of Neuron in Q-Learning of Chess

So I just read about deep Q-Learning which is using a neural network for optimization instead of Q-table. I saw the example here: https://yanpanlau.github.io/2016/07/10/FlappyBird-Keras.html and he ...
2
votes
0answers
41 views

Q Learning with Multiple Agents Design

Can anyone recommend a reinforcement learning algorithm for a multi agent environment. In my simplified example, I'm implementing a Q-Learning system with different 10 agents. The agents compete for ...
1
vote
0answers
37 views

Deep Q-Network concepts and implementation

How does sequential DQN work? How would one construct the simple sequential DQN? OpenAI Baselines: DQN
1
vote
1answer
42 views

Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?

I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state. The central service ...
2
votes
1answer
96 views

Action Probability with Thompson Sampling in Deep Reinforcement Learning

In some implementations of off-policy Q learning we need to know the action probabilities given by the behavior policy mu(a) (e.g., if we want to use importance ...
1
vote
0answers
58 views

Training RL agent on timeseries trading data with Continous Deep Q or NAF

I am writing an MDP based agent that is supposed to learn to place bids and asks in a trading environment. The system requests 2 values (mWh energy and $, both being positive or negative). Every ...
1
vote
1answer
63 views

Q-learning, should the exploration rate be reset after each trial?

As the title says, should I reset the exploration rate between trials? I am currently doing the Open AI pendulum task and after a number of trials my model started playing but did not take any ...
3
votes
1answer
500 views

Training AI to play NES/SNES games on NN python

I am currently getting into Deep Learning and would like to set up an environment for training an Artificial Neural Network or NEAT to play simple video games on NES (Mario etc.) and SNES ( Donkey ...
9
votes
1answer
1k views

Q-learning vs Policy Gradients

As far as I understand Q-learning and policy gradients are the two major approaches used to solve RL problems. While Q-learning aims to predict the reward of a certain action taken in a certain state, ...
3
votes
1answer
154 views

Snake path finding variant : Algorithm choice

I am working on a project which maps to a variant of path finding problem. I am new to this area and I would be very grateful if you could give suggestions/ point to libraries for relevant algorithms. ...
3
votes
1answer
157 views

What are good learning strategies for Deep Q-Network with opponents?

I am trying to find out what are some good learning strategies for Deep Q-Network with opponents. Let's consider the well known game Tic-Tac-Toe as an example: How should an opponent be implemented ...
3
votes
1answer
248 views

Q Learning and State(Stochastic Environment)

I am currently building my first AI in a stochastic environment and the following question came to my mind. The Q function uses the states(current and future)to determine the action that gets the ...
2
votes
1answer
543 views

How to use DQN to handle an imperfect but complete information game?

I'm currently having troubles to win against a random bot playing the Schieber Jass game. It is a imperfect card information game. (famous in switzerland https://www.schieber.ch/) The environement I'...
2
votes
0answers
158 views

Q-learning in Python

I'm working on a q-learning project that involves a "robot" solving a maze, and there is a problem with how I update the Q values (every time the robot ends up switching between two squares instead of ...
2
votes
2answers
67 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
0
votes
0answers
205 views

Deep-Q-Network is unlearning after a few epochs

I've been trying to train my Deep-Q network to play Breakout. I'm trying to replicate the results published by Deepmind in the Nature Paper. But my DQN peeks at some point and then starts the whole ...
5
votes
2answers
1k views

How to implement exploration function and learning rate in Q Learning

I'm trying to implement Q-learning (state-based representation and no neural / deep stuff) but I'm having a hard time getting it to learn anything. I believe my issue is with the exploration function ...
3
votes
1answer
213 views

Which Reinforcement Learning algorithms are efficient for episodic problems?

I have some episodic datasets extracted from a turn-based RTS game in which the current actions leading to the next state doesn’t determine the final solution/outcome of the episode. The learning is ...
2
votes
2answers
328 views

Is Q-learning a type of model-based RL?

Model-based RL creates a model of the transition function. Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a ...
4
votes
2answers
805 views

Why is the target $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ in the loss function of the DQN architecture?

In the paper Human-level control through deep reinforcement learning, the DQN architecture is presented, where the loss function is as follows $$ L_i(\theta_i) = \mathbb{E}_{(s, a, r, s') \sim U(D)} \...
1
vote
1answer
56 views

Why is a dynamics model unrealistic in Q-Learning?

Pieter Abbeel says that having access to the dynamics model, that is P(s' | s,a), is unrealistic because it assumes we know the probability that we will reach all ...
1
vote
0answers
106 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
2
votes
2answers
444 views

Q Learning Algorithm not converging

I am trying to run Deep Q-learning algorithm on a game which i made in python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the ...
1
vote
1answer
115 views

Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Learner might be in training stage, where it update Q-table for bunch of epoch. In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by ...
2
votes
1answer
864 views

Q learning tic tac toe

I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it'...
4
votes
1answer
133 views

How q-learning solves the issue with value iteration in model-free settings

I can't understand what is the problem in applying value-iteration in reinforcement learning setting (where we don't the reward and transition probabilities). In one of the lectures, the guy said it ...
3
votes
1answer
221 views

State representation of position in 2D plane for Reinforcement Learning (Q Learning)

I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ...