Questions tagged [reinforcement-learning]

For questions related to learning controlled by external positive reinforcement or negative feedback signal or both, where learning and use of what has been thus far learned occur concurrently.

Filter by
Sorted by
Tagged with
11
votes
1answer
6k views

How can policy gradients be applied in the case of multiple continuous actions?

Trusted Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are two cutting edge policy gradients algorithms. When using a single continuous action, normally, you would use some ...
3
votes
1answer
1k views

Is it necessary to clear the replay memory regularly in a DQN when an agent plays against itself?

I studied the article "Demystifying Deep Reinforcement Learning" extensively during the last days, while trying to implement the proposed algorithms myself. My goal is to have an agent learn by ...
3
votes
1answer
104 views

How can I design a hierarchy of agents each of which with different goals?

I read some light material earlier about the possibility of building a hierarchy of agents, where the agents at the leaves solve primitive tasks while higher-level agents are optimized for ...
4
votes
1answer
656 views

Is a decision tree less suitable for incremental learning than e.g. a neural net?

I can recall that a professor once said that decision trees are not good for incremental learning, as they have to be rebuilt from the ground up if new training examples arrive. Is this basically ...
4
votes
1answer
751 views

Traveling salesman problem variant: which algorithm to choose?

I have an industrial problem which I'm trying to cast as a Traveling Salesman problem (TSP) in 3D euclidian space. There are physical limitations which implies that some subpaths may or may not be ...
3
votes
1answer
235 views

What type of reinforcement learning can I do restricted to ~200MB on an average smartphone?

This concerns a set of finite, non-trivial, combinatorial games [M] in the form of an app. A sample game can be found here. Because this is a mass market product, we can't take up too much space, ...
5
votes
1answer
175 views

Did the Facebook robots both want everything but the balls?

According to this article, two Facebook ai's had the following "creepy" negotiation over a transaction: Bob: i can i i everything else . . . . . . . . . . . . . . Alice: balls have zero to ...
6
votes
2answers
582 views

How should I handle action selection in the terminal state when implementing SARSA?

I recently started learning about reinforcement learning and currently I am trying to implement the SARSA algorithm, however I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state....
5
votes
1answer
1k views

Can reinforcement learning algorithms be applied to computer vision problems?

Can reinforcement learning algorithms be applied to computer vision problems? If yes, what are some examples of these applications?
4
votes
1answer
299 views

What's stopping Cepheus from generalizing to full poker games?

Cepheus is an artificial intelligence designed to play Texas Hold'em. By playing against itself and learning where it could have done better, it became very good at the game. Slate Star Codex comments:...
1
vote
1answer
185 views

Markov Model for a Traffic Intersection

I need some help in developing a Markov Model for a crossroads there is no one way road and i am assuming at this time that traffic is only allowed to go straight no turns are allowed. There are 4 ...
1
vote
2answers
1k views

Q Learning Algorithm not converging

I am trying to run Deep Q-learning algorithm on a game which i made in python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the ...
13
votes
3answers
7k views

Are there any applications of reinforcement learning other than games?

Is there a way to teach reinforcement learning in applications other than games? The only examples I can find on the Internet are of game agents. I understand that VNC's control the input to the ...
4
votes
1answer
66 views

What are state-of-the-art ways of using greedy heuristics to initially set the weights of a Deep Q-Network in Reinforcement Learning?

I am interested in the current state-of-the-art ways to use quick, greedy heuristics in order to speed up the learning in a Deep Q-Network in Reinforcement Learning. In classical RL, I initially set ...
6
votes
1answer
1k views

OpenAI Baselines DQN - handling of invalid actions

I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it. In my environment, the best possible outcome for the agent is 0 -...
1
vote
0answers
86 views

Agent exploration which leads to a negative state where actions are limited

I'm working on a project where I train a Q-learning agent to learn an optimal control policy for a water heater. I've set up a simulation which allows the agent to explore for one year. I then examine ...
3
votes
1answer
1k views

Reinforcement learning for 2048

I implemented Actor-Critic with N-step TD prediction to learn to play 2048 (link to the game : http://2048game.com/) For the enviroment I don't use this 2048 implementation. I use a simple one without ...
2
votes
1answer
402 views

Can we use MCTS without a generative model?

From what I have understood reading the UCT paper Bandit based monte-carlo planning, by Levente Kocsis and Csaba Szepesvári, MCTS/UCT requires a generative model. Does it mean that, in case there is ...
1
vote
1answer
131 views

Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Learner might be in training stage, where it update Q-table for bunch of epoch. In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by ...
3
votes
0answers
389 views

RL to generate sentences

I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence. Eg: Input: capital, Paris, ...
2
votes
1answer
1k views

Q learning tic tac toe

I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it'...
25
votes
4answers
8k views

How to handle invalid moves in reinforcement learning?

I want to create an AI which can play five-in-a-row/gomoku. As I mentioned in the title, I want to use reinforcement learning for this. I use policy gradient method, namely REINFORCE, with baseline. ...
0
votes
1answer
157 views

Tensorboard problems

When trying to run tensorboard locally to show my logs with tensorboard --logdir logs/ it always shows nothing but the regular tensorboard menu options, such as ...
3
votes
1answer
502 views

When does backward propagation occur in n-step SARSA?

I am trying to understand the algorithm for n-step SARSA from Sutton and Barto (2nd Edition). As I understand it, this algorithm should update n state-action values, but I cannot see where it is ...
6
votes
1answer
213 views

A solution for a famous problem in RL

I'm here to ask you for a solution on this problem which is: how to use Reinforcement Learning in Immersive Virtual Reality to make a person move to a specific location in a virtual environment. As ...
1
vote
1answer
672 views

What algorithm should I use to classify documents?

I'd like to build a program that would learn to automatically classify documents. The principle would be that, for each new document I add to the system, it would automatically infer in which category ...
2
votes
1answer
1k views

Understanding why the expectation is over the new policy $\pi'$ in the proof of the Policy Improvement Theorem

In reinforcement learning, policy improvement is a part of an algorithm called policy iteration, which attempts to find approximate solutions to the Bellman optimality equations. Pages 84 and 85 in ...
8
votes
1answer
239 views

Are there any other machine learning models apart from Reinforcement Learning and Q Learning to play video games?

OpenAI's Universe utilises RL algorithms and I have heard of some game-training projects using Q learning, but are there any others which are used to master/win games? Can genetic algorithms be used ...
1
vote
1answer
100 views

How do you generate the transition probabilities of a non-trivial MDP?

I understand an MDP (Markov Decision Process) model is a tuple of $\{S, A, P, R \}$ where: $S$ is a discrete set of states $A$ is a discrete set of actions $P$ is the transition matrix ie. $P(s' \mid ...
1
vote
2answers
73 views

Whats the name of the value that you add or subtract from a minimax tree node?

I am coding a tic-tac-toe program that demonstrates reinforcement learning. The program uses minimax trees to decide its moves. Whenever it wins, all the nodes on the tree that were involved in the ...
8
votes
3answers
12k views

What are different actions in action space of environment of 'Pong-v0' game from openai gym?

Printing actionspace for Pong-v0 gives 'Discrete(6)' as output, i.e.0,1,2,3,4,5 are actions defined in environment as per documentation, but game needs only two controls. Why this discrepency? Further ...
6
votes
2answers
6k views

Negative reward (penalty) in policy gradient reinforcement learning

I am using policy gradients in my reinforcement learning algorithm, and occasionally my environment provides a severe penalty when a wrong move is made. I'm using a neural network with stochastic ...
2
votes
1answer
150 views

Network representation for Q-Learning in carrom

I am trying to build an agent to play carrom. The problem statement is roughly to estimate three parameters (normalized) : force angle of striker position of strike Since the state and action ...
3
votes
1answer
175 views

How q-learning solves the issue with value iteration in model-free settings

I can't understand what is the problem in applying value-iteration in reinforcement learning setting (where we don't the reward and transition probabilities). In one of the lectures, the guy said it ...
3
votes
2answers
1k views

What is a time-step in a Markov Decision Process?

The “Discounted sum of future rewards” using discount factor $\gamma$ is $\gamma$ (reward in 1 time step) + $\gamma^2$ (reward in 2 time steps) + $\gamma^3$ (reward in 3 time steps) + ... I am ...
6
votes
3answers
1k views

Board/Card Game AI - Questions concerning state/action space - Deep Reinforcement Learning

Ok, I now know how a machine can learn to play to play Atari games (Breakout): Playing Atari with Reinforcement Learning With the same technique it is even possible to play FPS games (Doom): Playing ...
4
votes
1answer
338 views

State representation of position in 2D plane for Reinforcement Learning (Q Learning)

I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ...
8
votes
1answer
296 views

What are some resources on continuous state and action spaces MDPs for reinforcement learning?

Most introductions to the field of MDPs and Reinforcement learning focus exclusively on domains where space and action variables are integers (and finite). This way we are introduced quickly to Value ...
5
votes
6answers
385 views

Is reinforcement learning needed to create Strong AI?

By reinforcement learning, I don't mean the class of machine learning algorithms such as DeepQ, etc. I have in mind the general concept of learning based on rewards and punishment. Is it possible to ...
5
votes
2answers
1k views

What is the current state-of-the-art in Reinforcement Learning regarding data efficiency?

In other words, which existing reinforcement method learns in fewest episodes? R-Max comes to mind, but its very old and I'd like to know if there is something better now.
6
votes
2answers
566 views

Is it possible to implement reinforcement learning using a neural network?

I've implemented the reinforcement learning algorithm for an agent to play snappy bird (a shameless cheap ripoff of flappy bird) utilizing a q-table for storing the history for future lookups. It ...

1
21 22 23 24
25