Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [q-learning]

For questions related to the Q-learning algorithm, which is a model-free and temporal-difference reinforcement learning algorithm that attempts to approximate the Q function, which is a function that, given a state s and an action a, returns a real number that represents the return (or value) of state s when action a is taken from s. Q-learning was introduced in the PhD thesis "Learning from Delayed Rewards" (1989) by Watkins.

Filter by
Sorted by
Tagged with
11
votes
1answer
1k views

Why does DQN require two different networks?

I was going through this implementation of DQN and I see that on line 124 and 125 two different Q networks have been initialized. From my understanding, I think one network predicts the appropriate ...
3
votes
1answer
407 views

Number of Neuron in Q-Learning of Chess

So I just read about deep Q-Learning which is using a neural network for optimization instead of Q-table. I saw the example here: https://yanpanlau.github.io/2016/07/10/FlappyBird-Keras.html and he ...
1
vote
0answers
52 views

Deep Q-Network concepts and implementation

How does sequential DQN work? How would one construct the simple sequential DQN? OpenAI Baselines: DQN
1
vote
1answer
56 views

Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?

I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state. The central service ...
2
votes
1answer
197 views

Action Probability with Thompson Sampling in Deep Reinforcement Learning

In some implementations of off-policy Q learning we need to know the action probabilities given by the behavior policy mu(a) (e.g., if we want to use importance ...
1
vote
0answers
133 views

Training RL agent on timeseries trading data with Continous Deep Q or NAF

I am writing an MDP based agent that is supposed to learn to place bids and asks in a trading environment. The system requests 2 values (mWh energy and $, both being positive or negative). Every ...
2
votes
1answer
123 views

Should the exploration rate be reset after each trial in Q-learning?

As the title says, should I reset the exploration rate between trials? I am currently doing the Open AI pendulum task and after a number of trials my model started playing but did not take any ...
3
votes
1answer
2k views

Training AI to play NES/SNES games on NN python

I am currently getting into Deep Learning and would like to set up an environment for training an Artificial Neural Network or NEAT to play simple video games on NES (Mario etc.) and SNES ( Donkey ...
19
votes
1answer
4k views

What is the relation between Q-learning and policy gradients methods?

As far as I understand, Q-learning and policy gradients (PG) are the two major approaches used to solve RL problems. While Q-learning aims to predict the reward of a certain action taken in a certain ...
3
votes
1answer
200 views

Snake path finding variant : Algorithm choice

I am working on a project which maps to a variant of path finding problem. I am new to this area and I would be very grateful if you could give suggestions/ point to libraries for relevant algorithms. ...
3
votes
1answer
274 views

What are good learning strategies for Deep Q-Network with opponents?

I am trying to find out what are some good learning strategies for Deep Q-Network with opponents. Let's consider the well known game Tic-Tac-Toe as an example: How should an opponent be implemented ...
4
votes
2answers
544 views

How does Q-learning work in stochastic environments?

The Q function uses the (current and future) states to determine the action that gets the highest reward. However, in a stochastic environment, the current action (at the current state) does not ...
2
votes
1answer
810 views

How to use DQN to handle an imperfect but complete information game?

I'm currently having troubles to win against a random bot playing the Schieber Jass game. It is a imperfect card information game. (famous in switzerland https://www.schieber.ch/) The environement I'...
2
votes
1answer
258 views

Q-learning in Python

I'm working on a q-learning project that involves a "robot" solving a maze, and there is a problem with how I update the Q values (every time the robot ends up switching between two squares instead of ...
2
votes
2answers
135 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
5
votes
2answers
2k views

How to implement exploration function and learning rate in Q Learning

I'm trying to implement Q-learning (state-based representation and no neural / deep stuff) but I'm having a hard time getting it to learn anything. I believe my issue is with the exploration function ...
3
votes
1answer
324 views

Which Reinforcement Learning algorithms are efficient for episodic problems?

I have some episodic datasets extracted from a turn-based RTS game in which the current actions leading to the next state doesn’t determine the final solution/outcome of the episode. The learning is ...
3
votes
2answers
588 views

Is Q-learning a type of model-based RL?

Model-based RL creates a model of the transition function. Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a ...
5
votes
2answers
995 views

Why is the target $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ in the loss function of the DQN architecture?

In the paper Human-level control through deep reinforcement learning, the DQN architecture is presented, where the loss function is as follows $$ L_i(\theta_i) = \mathbb{E}_{(s, a, r, s') \sim U(D)} \...
2
votes
2answers
85 views

Why is the access to the dynamics model unrealistic in Q-Learning?

Pieter Abbeel says that having access to the dynamics model, $P(s' \mid s,a)$, is unrealistic because it assumes we know the probability that we will reach all future states. I don't understand how ...
1
vote
0answers
146 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
1
vote
2answers
773 views

Q Learning Algorithm not converging

I am trying to run Deep Q-learning algorithm on a game which i made in python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the ...
1
vote
1answer
128 views

Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Learner might be in training stage, where it update Q-table for bunch of epoch. In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by ...
2
votes
1answer
1k views

Q learning tic tac toe

I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it'...
3
votes
1answer
163 views

How q-learning solves the issue with value iteration in model-free settings

I can't understand what is the problem in applying value-iteration in reinforcement learning setting (where we don't the reward and transition probabilities). In one of the lectures, the guy said it ...
3
votes
1answer
280 views

State representation of position in 2D plane for Reinforcement Learning (Q Learning)

I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ...