Questions tagged [reinforcement-learning]

For questions related to learning controlled by external positive reinforcement or negative feedback signal or both, where learning and use of what has been thus far learned occur concurrently.

Filter by
Sorted by
Tagged with
1
vote
1answer
273 views

Where to publish reasonable article in Deep Reinforcement Learning?

Please, can someone give advice what journals are good for first publication in the field of Deep Reinforcement Learning? I am in process of writing about research results of DQN related algorithms. ...
3
votes
1answer
108 views

Programming an inference AI that computes the best outcomes like a quantum computer

I bought an Intel Movidius Neural Compute stick a few weeks ago. Even though I can use it with the examples, I want to actually use it for something! The documentation is messy, and hard to work ...
1
vote
0answers
63 views

Handling varied-size input with fixed-input network

I'm running A3C (Asynchronous Actor-Critic Agents) to learn a game where an agent needs to catch 3 rewards. The input of my network, among other things, is the relative position of the 3 rewards ...
1
vote
0answers
148 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
2
votes
1answer
206 views

Does eligibility traces and epsilon-greedy do the same task in different ways?

I understand that in Reinforcement Learning algorithms such as q-learning, to prevent selecting the actions with greatest q-values too fast and allow for exploration, we use eligibility traces. Here ...
1
vote
1answer
191 views

What is $I$ in the noise described in the paper “Parameter Space Noise for Exploration”?

In the paper Parameter Space Noise for Exploration, the authors describe the noise that they add to the parameter vector as: $$ \tilde{\theta} = \theta + \mathcal{N}(0, \sigma^2I) $$ is $I$ simply ...
1
vote
1answer
516 views

Reinforcement learning for robotic motion planning - Problem statement ideas

I am a first-semester grad student in Robotics and have taken a course on machine learning for robotics. I am completely new to machine learning. I am to select and execute a problem statement on my ...
2
votes
0answers
165 views

Is iLQG a good algorithm for model-based planning with simple environments?

In their work Continuous Deep Q-Learning with Model-based Acceleration, the author demonstrate great results of applying Imagination Rollouts for model-based acceleration of learning. They test their ...
4
votes
2answers
235 views

Inconsistency in TD-Leaf algorithm in KnightCap chess engine

Notice that, in the following formula, at the very right, the term multiplied with $\lambda$ is $d_i$ $$ w := w + \alpha \sum_{i=1}^{N-1} \nabla r(x_i^l, w) \Big \lfloor \sum_{j=i}^{N-1} \lambda^{j-i}...
9
votes
1answer
5k views

Policy gradients for multiple continuous actions

Question is regarding Deep Reinforcement Learning using Policy Gradients. Cutting edge policy gradients algorithms are TRPO (Trusted Region Policy Optimization) and PPO (Proximal Policy Optimization)....
3
votes
1answer
1k views

Is it necessary to clear the replay memory regularly in a DQN when an agent plays against itself?

I studied the article "Demystifying Deep Reinforcement Learning" extensively during the last days, while trying to implement the proposed algorithms myself. My goal is to have an agent learn by ...
3
votes
1answer
72 views

Hierarchical Agent Design

I read some light material earlier about the possibility of building AI agent trees, which leaf agents optimizing for primitive tasks, while higher level agents optimizing for orchestrating direct ...
4
votes
1answer
342 views

Is a decision tree less suitable for incremental learning than e.g. a neural net?

I can recall that a professor once said that decision trees are not good for incremental learning, as they have to be rebuilt from the ground up if new training examples arrive. Is this basically ...
4
votes
1answer
730 views

Traveling salesman problem variant: which algorithm to choose?

I have an industrial problem which I'm trying to cast as a Traveling Salesman problem (TSP) in 3D euclidian space. There are physical limitations which implies that some subpaths may or may not be ...
3
votes
1answer
221 views

What type of reinforcement learning can I do restricted to ~200MB on an average smartphone?

This concerns a set of finite, non-trivial, combinatorial games [M] in the form of an app. A sample game can be found here. Because this is a mass market product, we can't take up too much space, ...
5
votes
1answer
171 views

Did the Facebook robots both want everything but the balls?

According to this article, two Facebook ai's had the following "creepy" negotiation over a transaction: Bob: i can i i everything else . . . . . . . . . . . . . . Alice: balls have zero to me ...
6
votes
2answers
419 views

How should I handle action selection in the terminal state when implementing SARSA?

I recently started learning about reinforcement learning and currently I am trying to implement the SARSA algorithm, however I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state....
5
votes
1answer
1k views

Apply reinforcement learning algorithms to computer vision problems

Is there a way to apply reinforcement learning algorithms to computer vision problems?
4
votes
1answer
273 views

What's stopping Cepheus from generalizing to full poker games?

Cepheus is an artificial intelligence designed to play Texas Hold'em. By playing against itself and learning where it could have done better, it became very good at the game. Slate Star Codex comments:...
1
vote
1answer
183 views

Markov Model for a Traffic Intersection

I need some help in developing a Markov Model for a crossroads there is no one way road and i am assuming at this time that traffic is only allowed to go straight no turns are allowed. There are 4 ...
1
vote
2answers
782 views

Q Learning Algorithm not converging

I am trying to run Deep Q-learning algorithm on a game which i made in python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the ...
12
votes
3answers
7k views

Are there any applications of reinforcement learning other than games?

Is there a way to teach reinforcement learning in applications other than games? The only examples I can find on the Internet are of game agents. I understand that VNC's control the input to the ...
4
votes
1answer
61 views

What are state-of-the-art ways of using greedy heuristics to initially set the weights of a Deep Q-Network in Reinforcement Learning?

I am interested in the current state-of-the-art ways to use quick, greedy heuristics in order to speed up the learning in a Deep Q-Network in Reinforcement Learning. In classical RL, I initially set ...
6
votes
1answer
1k views

OpenAI Baselines DQN - handling of invalid actions

I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it. In my environment, the best possible outcome for the agent is 0 -...
1
vote
0answers
85 views

Agent exploration which leads to a negative state where actions are limited

I'm working on a project where I train a Q-learning agent to learn an optimal control policy for a water heater. I've set up a simulation which allows the agent to explore for one year. I then examine ...
2
votes
1answer
953 views

Reinforcement learning for 2048

I implemented Actor-Critic with N-step TD prediction to learn to play 2048 (link to the game : http://2048game.com/) For the enviroment I don't use this 2048 implementation. I use a simple one without ...
2
votes
1answer
369 views

Can we use MCTS/UCT without a generative model?

From what I have understood reading the UCT paper "Bandit based monte-carlo planning", MCTS/UCT requires a generative model. Does it mean, in case there is no generative model of the environment, we ...
1
vote
1answer
129 views

Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Learner might be in training stage, where it update Q-table for bunch of epoch. In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by ...
3
votes
0answers
371 views

RL to generate sentences

I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence. Eg: Input: capital, Paris, ...
2
votes
1answer
1k views

Q learning tic tac toe

I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it'...
20
votes
4answers
6k views

How to handle invalid moves in reinforcement learning?

I want to create an AI which can play five-in-a-row/gomoku. As I mentioned in the title, I want to use reinforcement learning for this. I use policy gradient method, namely REINFORCE, with baseline. ...
0
votes
1answer
141 views

Tensorboard problems

When trying to run tensorboard locally to show my logs with tensorboard --logdir logs/ it always shows nothing but the regular tensorboard menu options, such as ...
3
votes
1answer
439 views

'Propagation' in n-step Sarsa

I am trying to understand the algorithm for n-step Sarsa from Sutton/Barto (2nd Edition, p. 157, PDF) As I understand it, this algorithm should update n state action values, but I cannot see where it ...
6
votes
1answer
202 views

A solution for a famous problem in RL

I'm here to ask you for a solution on this problem which is: how to use Reinforcement Learning in Immersive Virtual Reality to make a person move to a specific location in a virtual environment. As ...
0
votes
2answers
650 views

What algorithm should I use to classify documents?

I'd like to build a program that would learn to automatically classify documents. The principle would be that, for each new document I add to the system, it would automatically infer in which category ...
2
votes
1answer
765 views

Understanding why the expectation is over the new policy $\pi'$ in the proof of the Policy Improvement Theorem

In reinforcement learning, policy improvement is a part of an algorithm called policy iteration, which attempts to find approximate solutions to the Bellman optimality equations. Pages 84 and 85 in ...
8
votes
1answer
212 views

Are there any other machine learning models apart from Reinforcement Learning and Q Learning to play video games?

OpenAI's Universe utilises RL algorithms and I have heard of some game-training projects using Q learning, but are there any others which are used to master/win games? Can genetic algorithms be used ...
1
vote
1answer
88 views

How do you generate the transition probabilities of a non-trivial MDP?

I understand an MDP (Markov Decision Process) model is a tuple of $\{S, A, P, R \}$ where: $S$ is a discrete set of states $A$ is a discrete set of actions $P$ is the transition matrix ie. $P(s' \mid ...
1
vote
2answers
73 views

Whats the name of the value that you add or subtract from a minimax tree node?

I am coding a tic-tac-toe program that demonstrates reinforcement learning. The program uses minimax trees to decide its moves. Whenever it wins, all the nodes on the tree that were involved in the ...
7
votes
3answers
10k views

What are different actions in action space of environment of 'Pong-v0' game from openai gym?

Printing actionspace for Pong-v0 gives 'Discrete(6)' as output, i.e.0,1,2,3,4,5 are actions defined in environment as per documentation, but game needs only two controls. Why this discrepency? Further ...
4
votes
2answers
4k views

Negative reward (penalty) in policy gradient reinforcement learning

I am using policy gradients in my reinforcement learning algorithm, and occasionally my environment provides a severe penalty when a wrong move is made. I'm using a neural network with stochastic ...
2
votes
1answer
134 views

Network representation for Q-Learning in carrom

I am trying to build an agent to play carrom. The problem statement is roughly to estimate three parameters (normalized) : force angle of striker position of strike Since the state and action ...
3
votes
1answer
166 views

How q-learning solves the issue with value iteration in model-free settings

I can't understand what is the problem in applying value-iteration in reinforcement learning setting (where we don't the reward and transition probabilities). In one of the lectures, the guy said it ...
3
votes
2answers
1k views

What is a time-step in a Markov Decision Process?

The “Discounted sum of future rewards” using discount factor $\gamma$ is $\gamma$ (reward in 1 time step) + $\gamma^2$ (reward in 2 time steps) + $\gamma^3$ (reward in 3 time steps) + ... I am ...
6
votes
3answers
791 views

Board/Card Game AI - Questions concerning state/action space - Deep Reinforcement Learning

Ok, I now know how a machine can learn to play to play Atari games (Breakout): Playing Atari with Reinforcement Learning With the same technique it is even possible to play FPS games (Doom): Playing ...
3
votes
1answer
283 views

State representation of position in 2D plane for Reinforcement Learning (Q Learning)

I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ...
8
votes
1answer
258 views

Getting to understand continuous state/action spaces MDPs and Reinforcement Learning

Most introductions to the field of MDPs and Reinforcement learning focus exclusively on domains where space and action variables are integers (and finite). This way we are introduced quickly to Value ...
5
votes
6answers
332 views

Is reinforcement learning needed to create Strong AI?

By reinforcement learning, I don't mean the class of machine learning algorithms such as DeepQ, etc. I have in mind the general concept of learning based on rewards and punishment. Is it possible to ...
4
votes
2answers
1k views

What is the current state-of-the-art in Reinforcement Learning regarding data efficiency?

In other words, which existing reinforcement method learns in fewest episodes? R-Max comes to mind, but its very old and I'd like to know if there is something better now.
6
votes
2answers
538 views

Is it possible to implement reinforcement learning using a neural network?

I've implemented the reinforcement learning algorithm for an agent to play snappy bird (a shameless cheap ripoff of flappy bird) utilizing a q-table for storing the history for future lookups. It ...