Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Use for questions that involve Q-Learning.

4
votes
1answer
92 views

Is the discount not needed in a deterministic environment for Reinforcement Learning?

I'm now reading a book titled as "Deep Reinforcement Learning Hands-On" and the author said the following on the chapter about AlphaGo Zero: Self-play In AlphaGo Zero, the NN is used to ...
3
votes
1answer
73 views

Reinforcement Learning (Fitted Q): Qn on Concept & Implementation

I hope to get some clarifications on Fitted Q-Learning ('FQL'). My Research So Far I've read Sutton's book (specifically, chp 6 to 10), Ernst et al and this paper. I know that ...
2
votes
1answer
34 views

Convergence in multi-agent environment

I have a multi-agent environment where agents are trying to optimise the overall energy consumption of their group. Agents can exchange energy between themselves (actions for exchange of energy ...
5
votes
1answer
210 views

Why does DQN require two different networks?

I was going through this implementation of DQN and I see that on line 124 and 125 two different Q networks have been initialized. From my understanding, I think one network predicts the appropriate ...
2
votes
1answer
41 views

Number of Neuron in Q-Learning of Chess

So I just read about deep Q-Learning which is using a neural network for optimization instead of Q-table. I saw the example here: https://yanpanlau.github.io/2016/07/10/FlappyBird-Keras.html and he ...
2
votes
0answers
31 views

Q Learning with Multiple Agents Design

Can anyone recommend a reinforcement learning algorithm for a multi agent environment. In my simplified example, I'm implementing a Q-Learning system with different 10 agents. The agents compete for ...
1
vote
0answers
33 views

Deep Q-Network concepts and implementation

How does sequential DQN work? How would one construct the simple sequential DQN? OpenAI Baselines: DQN
1
vote
1answer
28 views

Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?

I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state. The central service ...
2
votes
1answer
38 views

Action Probability with Thompson Sampling in Deep Reinforcement Learning

In some implementations of off-policy Q learning we need to know the action probabilities given by the behavior policy mu(a) (e.g., if we want to use importance ...
1
vote
0answers
36 views

Training RL agent on timeseries trading data with Continous Deep Q or NAF

I am writing an MDP based agent that is supposed to learn to place bids and asks in a trading environment. The system requests 2 values (mWh energy and $, both being positive or negative). Every ...
1
vote
1answer
42 views

Q-learning, should the exploration rate be reset after each trial?

As the title says, should I reset the exploration rate between trials? I am currently doing the Open AI pendulum task and after a number of trials my model started playing but did not take any ...
1
vote
0answers
173 views

Training AI to play NES/SNES games on NN python

I am currently getting into Deep Learning and would like to set up an environment for training an Artificial Neural Network or NEAT to play simple video games on NES (Mario etc.) and SNES ( Donkey ...
1
vote
1answer
327 views

Q-learning vs Policy Gradients

As far as I understand Q-learning and policy gradients are the two major approaches used to solve RL problems. While Q-learning aims to predict the reward of a certain action taken in a certain state, ...
3
votes
1answer
109 views

Snake path finding variant : Algorithm choice

I am working on a project which maps to a variant of path finding problem. I am new to this area and I would be very grateful if you could give suggestions/ point to libraries for relevant algorithms. ...
3
votes
1answer
92 views

What are good learning strategies for Deep Q-Network with opponents?

I am trying to find out what are some good learning strategies for Deep Q-Network with opponents. Let's consider the well known game Tic-Tac-Toe as an example: How should an opponent be implemented ...
3
votes
1answer
114 views

Q Learning and State(Stochastic Environment)

I am currently building my first AI in a stochastic environment and the following question came to my mind. The Q function uses the states(current and future)to determine the action that gets the ...
2
votes
1answer
304 views

How to use DQN to handle an imperfect but complete information game?

I'm currently having troubles to win against a random bot playing the Schieber Jass game. It is a imperfect card information game. (famous in switzerland https://www.schieber.ch/) The environement I'...
2
votes
0answers
122 views

Q-learning in Python

I'm working on a q-learning project that involves a "robot" solving a maze, and there is a problem with how I update the Q values (every time the robot ends up switching between two squares instead of ...
1
vote
1answer
39 views

Should the actor or actor-target model be used to make predictions after training is complete (DDPG)?

The situation I am referring to the paper T. P. Lillicrap et al, "Continuous control with deep reinforcement learning" where they discuss deep learning in the context of continuous action spaces ("...
0
votes
0answers
146 views

Deep-Q-Network is unlearning after a few epochs

I've been trying to train my Deep-Q network to play Breakout. I'm trying to replicate the results published by Deepmind in the Nature Paper. But my DQN peeks at some point and then starts the whole ...
5
votes
2answers
431 views

How to implement exploration function and learning rate in Q Learning

I'm trying to implement Q-learning (state-based representation and no neural / deep stuff) but I'm having a hard time getting it to learn anything. I believe my issue is with the exploration function ...
3
votes
1answer
115 views

Which Reinforcement Learning algorithms are efficient for episodic problems?

I have some episodic datasets extracted from a turn-based RTS game in which the current actions leading to the next state doesn’t determine the final solution/outcome of the episode. The learning is ...
2
votes
2answers
157 views

Is Q-learning a type of model-based RL?

Model-based RL creates a model of the transition function. Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a ...
4
votes
2answers
359 views

I don't understand the “Target Network” in Deep Q-Learning

In deep Q learning, we define the loss function as follows: I don't understand how this sub-equation approximates the target at all. Literally none of it makes sense to me from a philosophical stand ...
1
vote
1answer
54 views

Why is a dynamics model unrealistic in Q-Learning?

Pieter Abbeel says that having access to the dynamics model, that is P(s' | s,a), is unrealistic because it assumes we know the probability that we will reach all ...
1
vote
0answers
88 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
2
votes
1answer
313 views

Q Learning Algorithm not converging

I am trying to run Deep Q-learning algorithm on a game which i made in python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the ...
1
vote
1answer
89 views

Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Learner might be in training stage, where it update Q-table for bunch of epoch. In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by ...
3
votes
1answer
658 views

Q learning tic tac toe

I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it'...
3
votes
1answer
113 views

How q-learning solves the issue with value iteration in model-free settings

I can't understand what is the problem in applying value-iteration in reinforcement learning setting (where we don't the reward and transition probabilities). In one of the lectures, the guy said it ...
3
votes
1answer
187 views

State representation of position in 2D plane for Reinforcement Learning (Q Learning)

I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ...