A machine learning technique influenced by behavioral psychology which can be described as "learning by trial and error."
1
vote
1answer
12 views
Difference between state, model, and environment in reinforcement learning
In particular I would like to have a simple definition of environment and state.what are the differences between those two concepts? Also I would like to know how the concept of model relates to the ...
4
votes
2answers
28 views
Does Monte Carlo Search (specifically used by AlphaZero) Qualify as Machine Learning?
To the best of my understanding, Monte Carlo Search is an alternative method to Minimax for searching a tree of nodes. It works by choosing a move (generally the one with the highest chance of being ...
4
votes
1answer
92 views
Is the discount not needed in a deterministic environment for Reinforcement Learning?
I'm now reading a book titled as "Deep Reinforcement Learning Hands-On" and the author said the following on the chapter about AlphaGo Zero:
Self-play
In AlphaGo Zero, the NN is used to ...
3
votes
1answer
57 views
How to generalise over multiple simultaneous dependent actions in Reinforcement Learning
I am trying to build an RL agent to price paid-for-seating on commercial flights. I should reiterate here - I am not talking about the price of the ticket - rather, I am talking about the pricing you ...
1
vote
0answers
24 views
Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning
Why the optimization step of the algorithm a quadratic program? [See: Apprenticeship Learning via Inverse Reinforcement Learning; page 3]
Isn't the objective function linear? Why don't we treat ...
2
votes
2answers
39 views
Reinforcement Learning Batch Size
I am using a neural network as my function approximator for reinforcement learning. In order to get it to train well I need to choose a good learning rate. Hand picking one is difficult, so I read up ...
3
votes
1answer
73 views
Reinforcement Learning (Fitted Q): Qn on Concept & Implementation
I hope to get some clarifications on Fitted Q-Learning ('FQL').
My Research So Far
I've read Sutton's book (specifically, chp 6 to 10), Ernst et al and this paper.
I know that ...
0
votes
1answer
25 views
Which is more important, doubt or reinforcement?
Reinforcement?
We hear much about reinforcement, which is, in my opinion a poor choice of a term to describe a type of artificial network that continues to acquire or improve its behavioral ...
1
vote
1answer
22 views
RL: how can the cart-pole problem be a continuing task?
In Introduction to Reinforcement Learning (2ed), Sutton and Barto, there is an example of Pole-Balancing problem (Example 3.4).
In this example, it said, this problem can be treated with 'episodic ...
4
votes
1answer
26 views
Exploration Strategies for Reinforcement Learning w/ Continuous Action Space
I'm building a deep neural network to serve as the policy estimator in an Actor-Critic reinforcement learning algorithm for a continuing (not episodic) case. I'm trying to determine how to explore ...
2
votes
1answer
34 views
Does epsilon-greedy approach always choose the “best action” (100% of the time) when it does not take the random path?
I'm now reading the following blog post but on the epsilon-greedy approach, the author implied that the epsilon-greedy approach takes the action randomly with the probability epsilon, and take the ...
4
votes
1answer
44 views
What is the difference between Actor-Critic and Advantage Actor-Critic?
I'm struggling to understand the difference between Actor-Critic and Advantage Actor-Critic.
At least I know they are different from Asynchronous Advantage Actor-Critic (A3C), as A3C adds ...
1
vote
1answer
33 views
What is the difference between policy and action in Reinforcement Learning?
I'm confused with the two terminology - action and policy - in Reinforcement Learning. As far as I know, the action is:
It is what the agent makes in a given state.
However, the book I'm reading ...
1
vote
1answer
24 views
What is the “trajectory” in the Reinforcement Learning?
I'm now learning about the Reinforcement Learning but just found the word "trajectory" in this answer.
However, I'm not sure what it means. I read a few books on the Reinforcement Learning but none ...
0
votes
0answers
15 views
What is a copy of network to create a “target network” for in DQN? [duplicate]
I'm now reading a book titled "Hands-On Reinforcement Learning with Python" by O'Reilly, and the author said the following to implement the DQN algorithm.
To make training more stable, there is a ...
3
votes
2answers
50 views
Reinforcement Learning with asynchronous feedback
I want suggestions on literature on Reinforcement Learning algorithms that perform well with asynchronous feedback from the environment. What I mean by asynchronous ...
1
vote
1answer
21 views
A3C & TWEAN (NEAT)
As a amateur researcher and tinkerer, I've been reading up on neuro-evolution networks (e.g. NEAT) as well as the A3C RL approach presented by Mnih et al and got to wondering if anyone has ...
1
vote
0answers
19 views
Reinforcement Learning in Game Javascript
I really want to understand how reinforcement learning works. I built a simple game to test this. There are squares falling from the sky and you have the arrow keys to escape.
How could I code the ...
6
votes
1answer
158 views
Learning algorithms of Neural Networks
Could you please let me know which of the following classification of Neural Network's learning algorithm is correct?
The first one classifies it into:
supervised,
unsupervised and
reinforcement ...
3
votes
1answer
42 views
Can the optimal value of discount factor in Deep Reinforcement Learning be between 0.2 to 0.8?
I'm now reading a book titled as Hands-On Reinforcement Learning with Python, and the author explains the discount factor that is used in Reinforcement Learing to discount the future reward, with the ...
4
votes
1answer
56 views
What would be the best approach to teach an AI to learn how to “sing” along a beat?
I have heard and read about HyperGAN, LSTM and a few other techniques, but I have a hard time piecing the overall concept together.
End Goal
Being able to input an instrumental and get an output of ...
2
votes
1answer
32 views
Can the opponent's turn affect the reward for a DQN agent action?
I made an engine for a 2 players card game and now I am trying to make an environment similar to OpenAI Gym envs, to ease out the training.
I fail to understand this thing however:
If I use ...
3
votes
1answer
28 views
Maximizing or Minimizing in Trust Region Policy Optimization?
I happened to discover that the v1(19 Feb 2015) and the v5(20 Apr 2017) versions of TRPO papers have two different conclusions. The Equation (15) in v1 is minimize_θ...
2
votes
0answers
21 views
Gradient of boltzmann policy over reward function
I'm struggling with an inverse reinforcement learning problem which seems to appear quite often around the literature, yet I can't find any resources explaining it.
The problem is that of calculating ...
1
vote
0answers
54 views
Comprehensive list of MOOCs and books on Reinforcement Learning
I'm actually trying to learn more about reinforcement learning but I've some trouble to find good resources. Right now I'm in the condition where I'm not so good on the topic to fully understand the ...
2
votes
1answer
39 views
Can DQN announce it has things in its hand in a card game?
More informations on the card game I'm talking about are in my last question here: DQN input representation for a card game
So I was thinking about the output of the q neural network and, aside from ...
2
votes
1answer
29 views
Taking into consideration “Number of Steps” in RL
I am currently implementing this paper in Python. While reading about the reward scheme I came across the following:
Finally, the proposed reward scheme implicitly considers
the number of steps ...
1
vote
1answer
28 views
Gradient descent update
In the following, I put the link for the general algorithm of maximum entropy inverse reinforcement learning. http://178.79.149.207/assets/maxent/maxent_slide.jpg
This uses a gradient descent ...
3
votes
1answer
63 views
Representing inputs and outputs for a card game neural network
I'm attempting to create an AI for a card game using reinforcement learning. The basics of the game are that you can have (theoretically) up to 35 cards in your hand, you can also have to up to 35 ...
1
vote
1answer
44 views
DQN input representation for a card game
In order to learn about DP and RL, I chose to start a side project where I would train an AI to play a "simple" card game. I will be doing this using the DQN with replay memory.
The problem is, I can'...
1
vote
0answers
27 views
Why do we have to solve MDP in each iteration of Maximum Entropy Inverse Reinforcement Learning?
Gradient in Maximum Entropy IRL requires to find the probability of expert trajectories given the reward function weights. This is done in the paper by calculating state visitation probabilities but I ...
5
votes
4answers
112 views
What's a good resource for getting familiar with reinforcement learning?
I am familiar with supervised and unsupervised learning. I did the SaaS course done by Andrew Ng on Coursera.org.
I am looking for something similar for reinforcement learning.
Can you recommend ...
5
votes
1answer
210 views
Why does DQN require two different networks?
I was going through this implementation of DQN and I see that on line 124 and 125 two different Q networks have been initialized. From my understanding, I think one network predicts the appropriate ...
2
votes
1answer
41 views
Number of Neuron in Q-Learning of Chess
So I just read about deep Q-Learning which is using a neural network for optimization instead of Q-table.
I saw the example here: https://yanpanlau.github.io/2016/07/10/FlappyBird-Keras.html and he ...
2
votes
1answer
28 views
analogy between reinforcement learning and webservers?
I've recently come across the client-server model , from my understanding , the client requests the server for which the server responds with a response in this case both the request and responses are ...
3
votes
1answer
90 views
Finding goals in Hierarchical Reinforcement Learning
In a recent paper Data-Efficient Hierarchical Reinforcement Learning, O Nachum, S Gu, H Lee, S Levine, 2018, a promising agent controlling technique called Hierarchical Reinforcement Learning was ...
2
votes
0answers
31 views
Q Learning with Multiple Agents Design
Can anyone recommend a reinforcement learning algorithm for a multi agent environment.
In my simplified example, I'm implementing a Q-Learning system with different 10 agents. The agents compete for ...
1
vote
1answer
28 views
Can Q-learning working in a multi agent environment where every agent learns a behaviour independently?
I am currently exploring multi-agent reinforcement learning. I have multiple agents that communicate with each other and a central service that maintains the environment state.
The central service ...
2
votes
1answer
68 views
Reinforcement Learning over an MDP that is actually a POMDP
Look at Breakout:
We know that the underlying world behaves like an MDP, because for the evolution of the system it just need to know which is the current state, i.e. position, speed and speed ...
2
votes
1answer
38 views
Action Probability with Thompson Sampling in Deep Reinforcement Learning
In some implementations of off-policy Q learning we need to know the action probabilities given by the behavior policy mu(a) (e.g., if we want to use importance ...
2
votes
0answers
80 views
Reinforcement Learning in Commercial Strategy Games
I'm a professional game developer investigating the potential for using reinforcement learning to build strategy game AI opponents that have more creative behavior compared to traditional techniques ...
2
votes
1answer
36 views
How to implement a contextual reinforcement learning model?
In a reinforcement learning model, states depend on the previous actions chosen. In the case in which some of the states -but not all- are fully independent of the actions -but still obviously ...
3
votes
1answer
84 views
Issue with simple game AI
A few months ago I made a simple game that is similar to the dinosaur game in Google Chrome - you jump over obstacles, or don't jump over levitating obstacles, and jump to collect bitcoins, which can ...
1
vote
0answers
27 views
Implementing AI/ML in customer service
(I originally posted the question on stack overflow but someone directed me here. So this is my first post here)
I am working on a task where I am required to automate the customer service request ...
2
votes
0answers
55 views
When can we say an RL algorithm learns an Attari game?
If a Attari game's rewards can be between -100 and 100, when can we say an agent learned to play this game? Should it get the reward very close to 100 for each instance of the game? or it is fine if ...
1
vote
1answer
88 views
Why does self-playing TicTacToe not become perfect?
I trained a DQN that learns TicTacToe by playing against itself with a reward of -1/0/+1 for a loss/draw/win. Every 500 episodes I test the progress by letting it play some episodes (also 500) against ...
0
votes
0answers
40 views
reinforcement learning rmsprop does not improve, average reward through time oscillates
I have a reinforcement learning project using policy gradient method with rmsprop optimization. (used vanilla REINFORCE algorithm)(the game is a simple pong game in open-ai gym atari environment)
the ...
2
votes
1answer
40 views
Implementing experience replay in reinforcement learning
I've been reading Google's DeepMind Atari paper and I'm trying to understand how to implement experience replay. My question is that whether we update the ...
2
votes
1answer
54 views
Understanding experience replay in reinforcement learning
I've been reading Google's DeepMind Atari paper and I'm trying to understand the concept of experience replay. Experience replay comes up in a lot of other ...
2
votes
2answers
40 views
How to see terminal reward in self-play reinforcement learning?
there seems to be a major difference how the terminal reward is received/handled in self-play RL vs "normal" RL which confuses me.
I implemented TicTacToe the normal way, where a single agent plays ...
