Questions tagged [reinforcement-learning]

For questions related to learning controlled by external positive reinforcement or negative feedback signal or both, where learning and use of what has been thus far learned occur concurrently.

Filter by
Sorted by
Tagged with
1
vote
1answer
432 views

What is the relation between back-propagation and reinforcement learning?

What is the relation between back-propagation and reinforcement learning?
3
votes
1answer
323 views

Which Reinforcement Learning algorithms are efficient for episodic problems?

I have some episodic datasets extracted from a turn-based RTS game in which the current actions leading to the next state doesn’t determine the final solution/outcome of the episode. The learning is ...
5
votes
2answers
347 views

Neural network for data visualization

At my work we're currently doing some research into data visualisation for highly inter connected data, basically graphs. We've been implementing all sorts of different layouts and trying to see ...
2
votes
0answers
84 views

Which features and algorithm could optimize this air-conditioner problem?

Imagine we have 2 air conditioner systems (AA) and 2 "free cooling" systems which mix external and internal air (FC) in a closed box which always tends to warm up. For each system, we have to find ...
3
votes
1answer
201 views

Can we teach machine to tie shoe lace?

Is it possible with any of machine learning methods to train machine to tie shoe lace? If possible how data should be interpreted for the training? If we are using reinforcement learning, how will it ...
1
vote
1answer
41 views

What is uncontrolled logging policy?

I am reading Learning from Logged Implicit Exploration Data It says Formally, given a dataset of the form S = (x, a, r_a)* generated by the interaction of an uncontrolled logging policy What is ...
1
vote
0answers
43 views

Regarding Tensorflow: How to Avoid Duplicate Use of Scope/Variable_names

I am trying to train Chess data through CNN. To proceed reinforcement learning, I had divided into two - "current network" and "reinforcement network". For each checkpoint file stored in different ...
7
votes
1answer
1k views

Do off-policy policy gradient methods exist?

Do off-policy policy gradient methods exist? I know that policy gradient methods themselves using the policy function for sampling rollouts. But can't we easily have a model for sampling from the ...
3
votes
2answers
581 views

Is Q-learning a type of model-based RL?

Model-based RL creates a model of the transition function. Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a ...
6
votes
1answer
161 views

Why does Bellman Equation solve an indirect policy?

I was watching a lecture on policy gradients vs Bellman equations. And they say that the Bellman equation indirectly creates a policy. While the policy gradient directly learns a policy? Why is this?
-1
votes
1answer
61 views

Skill primitives are not robust enough

I've modelled a micromanipulation domain with 22 subtasks: grasp: absinit, pregrasp, close graspvertical: pregraspinit, pregrasp, close, remove rotate: touchleft, touchright, closeleft, closeright ...
5
votes
2answers
994 views

Why is the target $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ in the loss function of the DQN architecture?

In the paper Human-level control through deep reinforcement learning, the DQN architecture is presented, where the loss function is as follows $$ L_i(\theta_i) = \mathbb{E}_{(s, a, r, s') \sim U(D)} \...
2
votes
2answers
85 views

Why is the access to the dynamics model unrealistic in Q-Learning?

Pieter Abbeel says that having access to the dynamics model, $P(s' \mid s,a)$, is unrealistic because it assumes we know the probability that we will reach all future states. I don't understand how ...
2
votes
1answer
190 views

Is it expected that during self-play reinforcement learning that player 1 or player 2 wins the majority of games?

I'm testing various learning rates and neural network configurations. I'm testing over 10000 games, with the first 2000 having random starting moves and general randomness throughout of about 20%, i.e....
2
votes
1answer
114 views

Why are neural networks always trained “by themselves”?

In the current rush of artificial intelligence research, fueled by NN, independent of the paper I choose, the NN are always trained by themselves. Sure, there are architectures that combine CNN and ...
2
votes
2answers
3k views

How to combine backpropagation in neural nets and reinforcement learning?

As I am trying to make an AI with reinforcement learning, I have found out and implemented a lot of things such as both these topics (NNs and RL) separately. But when trying to combine them, I have ...
9
votes
1answer
1k views

A few doubts regarding the application of reinforcement learning to games like chess

I invented a chess-like board game. I built an engine so that it can play autonomously. The engine is basically a decision tree. It's composed by: A search function that at each node finds all ...
27
votes
5answers
11k views

What's the difference between model-free and model-based reinforcement learning?

What's the difference between model-free and model-based reinforcement learning? It seems to me that any model-free learner, learning through trial and error, could be reframed as model-based. In ...
1
vote
1answer
249 views

Where to publish reasonable article in Deep Reinforcement Learning?

Please, can someone give advice what journals are good for first publication in the field of Deep Reinforcement Learning? I am in process of writing about research results of DQN related algorithms. ...
3
votes
1answer
107 views

Programming an inference AI that computes the best outcomes like a quantum computer

I bought an Intel Movidius Neural Compute stick a few weeks ago. Even though I can use it with the examples, I want to actually use it for something! The documentation is messy, and hard to work ...
1
vote
0answers
63 views

Handling varied-size input with fixed-input network

I'm running A3C (Asynchronous Actor-Critic Agents) to learn a game where an agent needs to catch 3 rewards. The input of my network, among other things, is the relative position of the 3 rewards ...
1
vote
0answers
146 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
2
votes
1answer
202 views

Does eligibility traces and epsilon-greedy do the same task in different ways?

I understand that in Reinforcement Learning algorithms such as q-learning, to prevent selecting the actions with greatest q-values too fast and allow for exploration, we use eligibility traces. Here ...
1
vote
1answer
191 views

What is $I$ in the noise described in the paper “Parameter Space Noise for Exploration”?

In the paper Parameter Space Noise for Exploration, the authors describe the noise that they add to the parameter vector as: $$ \tilde{\theta} = \theta + \mathcal{N}(0, \sigma^2I) $$ is $I$ simply ...
1
vote
1answer
515 views

Reinforcement learning for robotic motion planning - Problem statement ideas

I am a first-semester grad student in Robotics and have taken a course on machine learning for robotics. I am completely new to machine learning. I am to select and execute a problem statement on my ...
2
votes
0answers
162 views

Is iLQG a good algorithm for model-based planning with simple environments?

In their work Continuous Deep Q-Learning with Model-based Acceleration, the author demonstrate great results of applying Imagination Rollouts for model-based acceleration of learning. They test their ...
4
votes
2answers
232 views

Inconsistency in TD-Leaf algorithm in KnightCap chess engine

Notice that, in the following formula, at the very right, the term multiplied with $\lambda$ is $d_i$ $$ w := w + \alpha \sum_{i=1}^{N-1} \nabla r(x_i^l, w) \Big \lfloor \sum_{j=i}^{N-1} \lambda^{j-i}...
9
votes
1answer
4k views

Policy gradients for multiple continuous actions

Question is regarding Deep Reinforcement Learning using Policy Gradients. Cutting edge policy gradients algorithms are TRPO (Trusted Region Policy Optimization) and PPO (Proximal Policy Optimization)....
3
votes
1answer
1k views

Is it necessary to clear the replay memory regularly in a DQN when an agent plays against itself?

I studied the article "Demystifying Deep Reinforcement Learning" extensively during the last days, while trying to implement the proposed algorithms myself. My goal is to have an agent learn by ...
3
votes
1answer
71 views

Hierarchical Agent Design

I read some light material earlier about the possibility of building AI agent trees, which leaf agents optimizing for primitive tasks, while higher level agents optimizing for orchestrating direct ...
4
votes
1answer
287 views

Is a decision tree less suitable for incremental learning than e.g. a neural net?

I can recall that a professor once said that decision trees are not good for incremental learning, as they have to be rebuilt from the ground up if new training examples arrive. Is this basically ...
4
votes
1answer
727 views

Traveling salesman problem variant: which algorithm to choose?

I have an industrial problem which I'm trying to cast as a Traveling Salesman problem (TSP) in 3D euclidian space. There are physical limitations which implies that some subpaths may or may not be ...
3
votes
1answer
217 views

What type of reinforcement learning can I do restricted to ~200MB on an average smartphone?

This concerns a set of finite, non-trivial, combinatorial games [M] in the form of an app. A sample game can be found here. Because this is a mass market product, we can't take up too much space, ...
5
votes
1answer
171 views

Did the Facebook robots both want everything but the balls?

According to this article, two Facebook ai's had the following "creepy" negotiation over a transaction: Bob: i can i i everything else . . . . . . . . . . . . . . Alice: balls have zero to me ...
6
votes
2answers
403 views

How should I handle action selection in the terminal state when implementing SARSA?

I recently started learning about reinforcement learning and currently I am trying to implement the SARSA algorithm, however I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state....
5
votes
1answer
999 views

Apply reinforcement learning algorithms to computer vision problems

Is there a way to apply reinforcement learning algorithms to computer vision problems?
4
votes
1answer
271 views

What's stopping Cepheus from generalizing to full poker games?

Cepheus is an artificial intelligence designed to play Texas Hold'em. By playing against itself and learning where it could have done better, it became very good at the game. Slate Star Codex comments:...
1
vote
1answer
180 views

Markov Model for a Traffic Intersection

I need some help in developing a Markov Model for a crossroads there is no one way road and i am assuming at this time that traffic is only allowed to go straight no turns are allowed. There are 4 ...
1
vote
2answers
772 views

Q Learning Algorithm not converging

I am trying to run Deep Q-learning algorithm on a game which i made in python using pygame library. The algorithm accepts the game screen (4 frames) as input to neural network which used as the ...
12
votes
3answers
6k views

Are there any applications of reinforcement learning other than games?

Is there a way to teach reinforcement learning in applications other than games? The only examples I can find on the Internet are of game agents. I understand that VNC's control the input to the ...
4
votes
1answer
61 views

What are state-of-the-art ways of using greedy heuristics to initially set the weights of a Deep Q-Network in Reinforcement Learning?

I am interested in the current state-of-the-art ways to use quick, greedy heuristics in order to speed up the learning in a Deep Q-Network in Reinforcement Learning. In classical RL, I initially set ...
6
votes
1answer
1k views

OpenAI Baselines DQN - handling of invalid actions

I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it. In my environment, the best possible outcome for the agent is 0 -...
1
vote
0answers
85 views

Agent exploration which leads to a negative state where actions are limited

I'm working on a project where I train a Q-learning agent to learn an optimal control policy for a water heater. I've set up a simulation which allows the agent to explore for one year. I then examine ...
2
votes
1answer
943 views

Reinforcement learning for 2048

I implemented Actor-Critic with N-step TD prediction to learn to play 2048 (link to the game : http://2048game.com/) For the enviroment I don't use this 2048 implementation. I use a simple one without ...
2
votes
1answer
369 views

Can we use MCTS/UCT without a generative model?

From what I have understood reading the UCT paper "Bandit based monte-carlo planning", MCTS/UCT requires a generative model. Does it mean, in case there is no generative model of the environment, we ...
1
vote
1answer
128 views

Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Learner might be in training stage, where it update Q-table for bunch of epoch. In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by ...
3
votes
0answers
370 views

RL to generate sentences

I want to develop a system to generate grammatically correct sentences. The input would be some words. The output would be a grammatically correct human-like sentence. Eg: Input: capital, Paris, ...
2
votes
1answer
1k views

Q learning tic tac toe

I have a tic-tac-toe with a Q-learning algorithm, and the AI plays against the same algorithm (but they don't share the same Q matrix). But after 200,000 games, I still beat the AI very easily and it'...
19
votes
4answers
5k views

How to handle invalid moves in reinforcement learning?

I want to create an AI which can play five-in-a-row/gomoku. As I mentioned in the title, I want to use reinforcement learning for this. I use policy gradient method, namely REINFORCE, with baseline. ...
0
votes
1answer
140 views

Tensorboard problems

When trying to run tensorboard locally to show my logs with tensorboard --logdir logs/ it always shows nothing but the regular tensorboard menu options, such as ...