Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [q-learning]

Use for questions that involve Q-learning, where Q is the value of a particular next action among a set of possible actions, based on a specified function of each action and its projected result.

0
votes
0answers
11 views

How to train a model by accounting for boundary constraints?

I've a robot traverse through a grid layout. Based on the wheel speed difference I classify actions as either straight, left or right. I computed the distances based on the time duration and the speed ...
1
vote
1answer
24 views

Some RL algorithms (especially policy gradients) initialize with random policies, which often manifests as random jitter on spot for a long time?

I am reviewing a statement on the website for ES regarding structured exploration. https://blog.openai.com/evolution-strategies/ Structured exploration. Some RL algorithms (especially policy ...
6
votes
1answer
44 views

Q-Learning the generic maze solution

After doing some exercices on Q-learning for maze solving, I wondered : my q-learning algorithms solve only ONE maze. The AI doesn't learn how to solve mazes, so how can I achieve it ? For instance ...
-1
votes
0answers
30 views

What problem does Double DQN solve in Deep Q learning?

What is meant by overestimation in q-learning? How does Double DQN solve this problem?
2
votes
3answers
192 views

difficulty in understanding identifiability in Dueling Network paper

I have difficulty understanding the following paragraph in bracketed in red parentheses in the below excerpts from page 4 to page 5 from the paper Dueling Network Architectures for Deep Reinforcement ...
1
vote
1answer
32 views

Q Llearning for Shortest distance

Is it possible to form a table that will have simply the shortest distance from each source to destination using q learning. If not suggest any other learning algorithm
4
votes
1answer
113 views

Is the discount not needed in a deterministic environment for Reinforcement Learning?

I'm now reading a book titled as "Deep Reinforcement Learning Hands-On" and the author said the following on the chapter about AlphaGo Zero: Self-play In AlphaGo Zero, the NN is used to ...
4
votes
1answer
83 views

Reinforcement Learning (Fitted Q): Qn on Concept & Implementation

I hope to get some clarifications on Fitted Q-Learning ('FQL'). My Research So Far I've read Sutton's book (specifically, chp 6 to 10), Ernst et al and this paper. I know that ...
2
votes
1answer
38 views

Convergence in multi-agent environment

I have a multi-agent environment where agents are trying to optimise the overall energy consumption of their group. Agents can exchange energy between themselves (actions for exchange of energy ...
6
votes
1answer
317 views

Why does DQN require two different networks?

I was going through this implementation of DQN and I see that on line 124 and 125 two different Q networks have been initialized. From my understanding, I think one network predicts the appropriate ...
3
votes
1answer
57 views

Number of Neuron in Q-Learning of Chess

So I just read about deep Q-Learning which is using a neural network for optimization instead of Q-table. I saw the example here: https://yanpanlau.github.io/2016/07/10/FlappyBird-Keras.html and he ...
2
votes
0answers
35 views

Q Learning with Multiple Agents Design

Can anyone recommend a reinforcement learning algorithm for a multi agent environment. In my simplified example, I'm implementing a Q-Learning system with different 10 agents. The agents compete for ...
1
vote
0answers
36 views

Deep Q-Network concepts and implementation

How does sequential DQN work? How would one construct the simple sequential DQN? OpenAI Baselines: DQN
1
vote
1answer
34 views

Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?

I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state. The central service ...
2
votes
1answer
56 views

Action Probability with Thompson Sampling in Deep Reinforcement Learning

In some implementations of off-policy Q learning we need to know the action probabilities given by the behavior policy mu(a) (e.g., if we want to use importance ...
1
vote
0answers
47 views

Training RL agent on timeseries trading data with Continous Deep Q or NAF

I am writing an MDP based agent that is supposed to learn to place bids and asks in a trading environment. The system requests 2 values (mWh energy and $, both being positive or negative). Every ...
1
vote
1answer
48 views

Q-learning, should the exploration rate be reset after each trial?

As the title says, should I reset the exploration rate between trials? I am currently doing the Open AI pendulum task and after a number of trials my model started playing but did not take any ...
3
votes
1answer
315 views

Training AI to play NES/SNES games on NN python

I am currently getting into Deep Learning and would like to set up an environment for training an Artificial Neural Network or NEAT to play simple video games on NES (Mario etc.) and SNES ( Donkey ...
4
votes
1answer
738 views

Q-learning vs Policy Gradients

As far as I understand Q-learning and policy gradients are the two major approaches used to solve RL problems. While Q-learning aims to predict the reward of a certain action taken in a certain state, ...
3
votes
1answer
130 views

Snake path finding variant : Algorithm choice

I am working on a project which maps to a variant of path finding problem. I am new to this area and I would be very grateful if you could give suggestions/ point to libraries for relevant algorithms. ...
3
votes
1answer
127 views

What are good learning strategies for Deep Q-Network with opponents?

I am trying to find out what are some good learning strategies for Deep Q-Network with opponents. Let's consider the well known game Tic-Tac-Toe as an example: How should an opponent be implemented ...
3
votes
1answer
169 views

Q Learning and State(Stochastic Environment)

I am currently building my first AI in a stochastic environment and the following question came to my mind. The Q function uses the states(current and future)to determine the action that gets the ...
2
votes
1answer
424 views

How to use DQN to handle an imperfect but complete information game?

I'm currently having troubles to win against a random bot playing the Schieber Jass game. It is a imperfect card information game. (famous in switzerland https://www.schieber.ch/) The environement I'...
2
votes
0answers
136 views

Q-learning in Python

I'm working on a q-learning project that involves a "robot" solving a maze, and there is a problem with how I update the Q values (every time the robot ends up switching between two squares instead of ...
2
votes
2answers
55 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
0
votes
0answers
177 views

Deep-Q-Network is unlearning after a few epochs

I've been trying to train my Deep-Q network to play Breakout. I'm trying to replicate the results published by Deepmind in the Nature Paper. But my DQN peeks at some point and then starts the whole ...
5
votes
2answers
649 views

How to implement exploration function and learning rate in Q Learning

I'm trying to implement Q-learning (state-based representation and no neural / deep stuff) but I'm having a hard time getting it to learn anything. I believe my issue is with the exploration function ...
3
votes
1answer
151 views

Which Reinforcement Learning algorithms are efficient for episodic problems?

I have some episodic datasets extracted from a turn-based RTS game in which the current actions leading to the next state doesn’t determine the final solution/outcome of the episode. The learning is ...
2
votes
2answers
210 views

Is Q-learning a type of model-based RL?

Model-based RL creates a model of the transition function. Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a ...
4
votes
2answers
600 views

I don't understand the “Target Network” in Deep Q-Learning

In deep Q learning, we define the loss function as follows: I don't understand how this sub-equation approximates the target at all. Literally none of it makes sense to me from a philosophical stand ...
1
vote
1answer
54 views

Why is a dynamics model unrealistic in Q-Learning?

Pieter Abbeel says that having access to the dynamics model, that is P(s' | s,a), is unrealistic because it assumes we know the probability that we will reach all ...
1
vote
0answers
100 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
2
votes
1answer
360 views

Q Learning Algorithm not converging

I am trying to run Deep Q-learning algorithm on a game which i made in python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the ...
1
vote
1answer
100 views

Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Learner might be in training stage, where it update Q-table for bunch of epoch. In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by ...
3
votes
1answer
739 views

Q learning tic tac toe

I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it'...
4
votes
1answer
117 views

How q-learning solves the issue with value iteration in model-free settings

I can't understand what is the problem in applying value-iteration in reinforcement learning setting (where we don't the reward and transition probabilities). In one of the lectures, the guy said it ...
3
votes
1answer
204 views

State representation of position in 2D plane for Reinforcement Learning (Q Learning)

I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ...