Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [reinforcement-learning]

For questions related to learning controlled by external positive reinforcement or negative feedback signal or both, where learning and use of what has been thus far learned occur concurrently.

211 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
41 views

How to design the reward for an action which is the only legal action at some state

I am working on a RL project,but got stuck at one point: The task is continuous (Non-episodic). Following some suggestion from Sutton's RL book, I am using a value function approximation method with ...
1
vote
0answers
150 views

How can a fully automated vacuum cleaner use and update room information?

Assistive Subsystems Consider an automated vacuum cleaner with the following subsystems under the command of an AI system to be designed. These subsystems limit the AI complexity to just intelligent ...
1
vote
0answers
56 views

Reinforcement learning for segmenting the robot path to reflect the true distances

I've a grid of rectangles acting as blocks. The robot traverses through the inter-spaces between these consecutive blocks. Now I have sensor data streaming in representing Right and left wheel speeds. ...
1
vote
0answers
15 views

Does inflation should occur in output layer when I do Artificial Neural Network to increase smartness of the model?

The idea that come to my mind is called Value Based Model for ANN. We use simple DCF formula to calculate kind of Q value: Rewards/Discount rate. Discount rate is a risk of getting the reward on the ...
1
vote
0answers
41 views

Optimization step in Apprenticeship Learning via Inverse Reinforcement Learning

Why the optimization step of the algorithm a quadratic program? [See: Apprenticeship Learning via Inverse Reinforcement Learning; page 3] Isn't the objective function linear? Why don't we treat ...
1
vote
0answers
66 views

Comprehensive list of MOOCs and books on Reinforcement Learning

I'm actually trying to learn more about reinforcement learning but I've some trouble to find good resources. Right now I'm in the condition where I'm not so good on the topic to fully understand the ...
1
vote
0answers
55 views

Why do we have to solve MDP in each iteration of Maximum Entropy Inverse Reinforcement Learning?

Gradient in Maximum Entropy IRL requires to find the probability of expert trajectories given the reward function weights. This is done in the paper by calculating state visitation probabilities but I ...
1
vote
0answers
13 views

Can you make the first layer of a net have discernible shapes?

Coming from the YT videos of 3blue1brown which showed that the individual layers do not have discernible shapes in the case of hand written letter recognition, I wondered if you could penalize ...
1
vote
0answers
56 views

Learning from events. Supervised, Unsupervised or MDP?

I have a large set of simulation logs for a market simulation of which I want to learn from. The market includes: customers products (subscriptions) The customers choose products and then stick with ...
1
vote
0answers
27 views

Name of a multiarmed bandit with only some levers available

In order to model a card game as an exercise I was thinking an elementary setting as a multiarmed bandit, each lever being the distribution of expected rewards of an specific card. But of course the ...
1
vote
0answers
36 views

Dealing with input to recurrent net with changing dimensions

I have a problem in which the dimensions of the input are increasing in row and column at each timestep. What method for preprocessing could be done or are there any architectures used for solving ...
1
vote
0answers
43 views

Regarding Tensorflow: How to Avoid Duplicate Use of Scope/Variable_names

I am trying to train Chess data through CNN. To proceed reinforcement learning, I had divided into two - "current network" and "reinforcement network". For each checkpoint file stored in different ...
1
vote
0answers
64 views

Handling varied-size input with fixed-input network

I'm running A3C (Asynchronous Actor-Critic Agents) to learn a game where an agent needs to catch 3 rewards. The input of my network, among other things, is the relative position of the 3 rewards ...
1
vote
0answers
148 views

Help with implementing Q-learning for a feedfoward network playing a video game

I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives. In ...
1
vote
0answers
85 views

Agent exploration which leads to a negative state where actions are limited

I'm working on a project where I train a Q-learning agent to learn an optimal control policy for a water heater. I've set up a simulation which allows the agent to explore for one year. I then examine ...
1
vote
1answer
119 views

Is Reinforcement Learning the future of Natural Language Processing?

I was reading about the grounding problem after seeing it mentioned in another answer today. The article states that, in order to avoid the "infinite regress" of defining all words with other words, ...
1
vote
2answers
281 views

Is Monte Carlo Tree Search appropriate for problems with large state and action spaces?

I'm doing a research on a finite-horizon Markov decision process with $t=1, \dots, 40$ periods. In every time step $t$, the (only) agent has to chose an action $a(t) \in A(t)$, while the agent is in ...
1
vote
1answer
330 views

Continuous Advantage Actor Critic Implementation

I'm having trouble implementing AC for continuous action space. As far as I can tell, my code doesn't seem to have any bugs! The agent is learning "something" as its behaviour seems to vary ...
0
votes
0answers
10 views

Same implementation, but agent is not learning in Retro Pong Environment

I tried to implement the exact same python coding by Andrej Karpathy to train RL agent to play Pong, except that I migrated the environment from Gym to Retro. Everything is the same except the action ...
0
votes
0answers
14 views

How is the general return-based off-policy equation derived?

I'm wondering how is the general return-based off-policy equation in Safe and efficient off-policy reinforcement learning derived $$\mathcal{R} Q(x, a):=Q(x, a)+\mathbb{E}_{\mu}\left[\sum_{t \geq 0} \...
0
votes
0answers
8 views

Training methods for bipedal robot

I am looking to train a bipedal robot using unity as a scape with a genetic algorithm. I will import the CAD into unity so the hardware is exact. My questions: Is Unity physics accurate enough to ...
0
votes
0answers
21 views

Can someone explain how and why actor-critic networks are created this way?

Deep Deterministic Policy Gradients (DDPG) and stable Baseline Code is presented here. The actor-critic networks are created as follows: ...
0
votes
0answers
19 views

Ideas on how to train an AI to play Mario Kart with the DeSmuME Emulator

Gday guys, i have this idea in my mind for quite a while. I want to teach an AI to play Mario Kart on the NDS, which can hopefully beat me and my friends one day. Iam familiar with the theoretical ...
0
votes
0answers
14 views

Choosing best combinations from all possible combination expressions based few variables, unary operators, binary operators

I have a few financial variables of a stock universe like OHLC prices, volume, and other fundamentals with varying time-frequency. Using this set I'm creating an expression that gives the weights to ...
0
votes
0answers
20 views

DQN unlearns certain OpenAI-Gym environments

I solved the OpenAI-Gym MountainCar-v0 environment using dqn(using low-state-dimensional input). When I used the same code for solving CartPole-v0 environment, the network got trained in the reverse ...
0
votes
0answers
21 views

A2C for the game of Hanabi underfits

I am trying to solve the game of Hanabi (paper describing game) with actor-critic algorithm. I took code for the environment from the Deepmind's repository and implemented a2c algorithm myself. From ...
0
votes
0answers
33 views

Finding optimal Value function and Policy for an MDP

I am solving an RL MDP problem which is model based. I have an MDP which has four possible states S1-S4 and four different actions A1-A4, with S4 being terminal state and S1 is the beginning state. ...
0
votes
0answers
18 views

Loss reduction, but constant performance with CNN

I made a CNN with a reasonable loss curve, but the performance of the model does not improve. I have tried making the model larger, I am using three convolutional layers with batch norms. Thanks for ...
0
votes
0answers
19 views

Suggestion on How to Push SARS to Memory Buffer of Losing RL Agent in Adversarial Learning

I am trying to implement my first RL program where there are multiple agents, rather than just one. The environment I am using is the connect four game, which is turn-based. In DQ-Learning, an agent ...
0
votes
0answers
13 views

DDPG: how to implement continous action space bounded in the interval [-2, 2]

I am newbie in reinforcement learning and trying to understand how to implement continuous actions bounded by [-2, 2]. My research shows that doing nothing is a possible solution (i.e. action of 4.5 ...
0
votes
2answers
42 views

How should I interpret the weights file of the Leela Zero neural network?

I am trying to understand the NN architecture given at https://github.com/leela-zero/leela-zero/blob/next/training/caffe/zero.prototxt. So, I downloaded the NN weights from http://zero.sjeng.org/. ...
0
votes
0answers
13 views

Derivation for Value Iteration of CVaR

I am reading a paper named Risk-sensitive and Robust Decision-making: a CVaR Optimization Approach. In appendix A.3 they provide a proof for their Theorem $4$. The $n=1$ case for equation (11) is ...
0
votes
0answers
28 views

Why epsilon-greedy hyperparameter is annealed smoothly?

Regarding of DQN, or DQRNN, (reinforcement learning) To me, RL is a process that can be divided into 2 stages: Exploring wide range of paths (acting randomly) Refining the current optimal paths (...
0
votes
0answers
31 views

Questions performance SimPLe pong for AI demo

For a demo I need to develop an AI solution to learn how to play pong. I have the following requirements: Computer needs to play against a human player. Learn while playing the game. Poor AI result ...
0
votes
0answers
24 views

Weak gradient around one hot values of softmax

If you do the math for the softmax gradient, the gradient is very weak around the simplexical vertices. Aka for a 5-class softmax,[1, 0, 0, 0, 0] has a hard time moving to [0, 1, 0, 0, 0]. I have this ...
0
votes
0answers
51 views

Online normalization of database for DQN

I have an issue with the normalization of the database (a large time series) for my DQN. I obtained optimal results and saved the NN (5 LSTM layers) weights training on a database normalized as such: ...
0
votes
0answers
37 views

How to learn to sample?

Imagine you have access to a dataset of pairs $(s, \hat{\pi}(s))$ where s is a state in a high dimension continuous space $S$, $\pi(s)$ is a probabilistic distribution on a large discrete space $D$ (...
0
votes
0answers
53 views

DQN Algorithm not Converging for HVAC Control Task

I am trying to implement the DQN algorithm for the task of HVAC control. I have the algorithm implemented using Pytorch. I know that the HVAC simulator is working as it works for other control methods....
0
votes
0answers
16 views

Problems while training a DQN Agent on DSTC dataset

I am trying to create a dialogue policy model on DSTC data. This model takes in a state of the conversation and outputs an act the machine must take. I am creating this model using reinforcement ...
0
votes
0answers
29 views

Are there real-world problems where case-based reasoning is not suitable?

I know case-based reasoning has four stages: retrieve, retain, re-use and revise. Used for solving new problems by adapting solutions that were used to solve old problems, like car issues. The ...
0
votes
0answers
93 views

Details of implementing an LSTM in Reinforcement Learning

I'm currently looking into the context of adding an LSTM to my PPO pytorch implementation. My plan is to add one LSTM layer right after the last convolutional layer. I'm wondering now whether it is ...
0
votes
0answers
33 views

How do I define the reward function in the case of self-driving raspberry pi car?

I am working on a self driving car powered by a raspberry pi. My first step is to use PPO to teach it to not run into walls. But I am having trouble getting it to work. I want to allow the car to ...
0
votes
1answer
174 views

Reinforcement learning with PPO: rewards decreasing

I'm trying to train a PPO agent and my average rewards graph looks like this. Could this indicate that it's stuck at a local maximum? Do I need to promote exploring by increasing the entropy or does ...
0
votes
0answers
24 views

Shared actor-critic using only local rules

I was wondering if the following ‘shared actor-critic’ principal using local rules has been established ?.. Take an actor network, which can form abstract (ie hidden layer) representations using a ...
0
votes
0answers
39 views

$\epsilon$-greedy policies for huge state space

I'm currently building an agent that learns to play Kalah through reinforcement learning. I've gotten quite far along. With an $\epsilon$ of 0, meaning no exploration and only exploitation, it is able ...
0
votes
0answers
20 views

Turn a NES ROM into object/tile representation

So i have a rom of a hacked super mario game (it has 2 players: Mario and Luigi). Feeding in the raw pixel data of this results in very poor rewards. I was wondering if there was a way to transform ...
0
votes
0answers
136 views

Policy gradient in keras predicts only one action

I have trouble with the REINFORCE algorithm in keras with Atari games. After round about 30 episodes the network converges to one action. But the same algorithm is working with CartPole-v1 and ...
0
votes
0answers
123 views

DQN Q-mean values converge negatively

I'm trying to implement my own DQN. So far I think my code is good, but my Q-values (I'm getting the mean of all the values for every episode) tends to converge near-zero but negatively. It is normal? ...
0
votes
0answers
53 views

Having trouble solving Pong. My model is not improving

Im trying to solve pong by a DQN approach. These are my hyper parameters: ...
0
votes
0answers
106 views

How to back-propagate illegal actions for policy gradient learning

When training a AI RL agent to play a game there'll be situations where the AI cannot perform certain actions lest they violate the game rules. That's easy to handle, and I can set illegal actions to ...