Questions tagged [reinforcement-learning]

For questions related to reinforcement learning, i.e. a machine learning technique where we imagine an agent that interacts with an environment (composed of states) in time steps by taking actions and receiving rewards (or reinforcements), then, based on these interactions, the agent tries to find a policy (i.e. a behavioural strategy) that maximizes the cumulative reward (in the long run), so the goal of the agent is to maximize the reward.

566 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
22 views

Are there any board game appropriate to examine the performance of multiple agents that cooperate both inter-group and intra-group?

I want to find out scenarios that useful to examine the performance of intra-group and inter-group cooperation in MARL. Specifically, I prefer a board game (like sudoku) that is suitable for the ...
1
vote
0answers
37 views

Is Q-Learning suitable for time-dependent spaces?

Many Q-learning techniques have been developed to capture discrete state(observation), actions like a robot in a grid world, and even continuous (state or action) spaces. But I am wondering how we can ...
1
vote
0answers
17 views

Using AI to find the correct set of object/numbers based on previous data

There are 11 objects of which 4 are "Bad" objects. So there are 7 "Good" objects. You have to choose as many Good objects before proceeding to another set of objects of a different sequence. How ...
1
vote
0answers
35 views

Subtracting the entropy from our policy gradient will prevent our agent from being stuck in the local minimum?

In the information theory, the entropy is a measure of uncertainty in some system. Being applied to agent policy, entropy shows how much the agent is uncertain about which action to make. In math ...
1
vote
0answers
23 views

Do I need to maintain a separate population in each distributed environment when implementing PBT in a MARL context?

I have questions regarding on how to implement PBT as described in Algorithm 1 (on page 5) in the paper, Population Based Training of Neural Networks to train agents in a MARL (multi-agent ...
1
vote
0answers
28 views

How can I design a DQN or policy gradient model to explore and collect all optimal solutions?

I am working to use DQN and Policy Gradient reinforcement learning models to solve classic maze escaping problems. So far, I have been able to train a model, which, after around 100 episodes, ...
1
vote
0answers
26 views

Why does this tutorial on reinforced learning not check whether the environment is 'game over' during training?

I am following the tutorial Train a Deep Q Network with TF-Agents. It uses the hello world environment of reinforced learning: cart pole. At the end, the agent is ...
1
vote
0answers
17 views

Has there been any work done on AI-driven operant conditioning?

For example, a RL algorithm that gains points when a rat presses a lever and loses points when it dispenses a pellet, water, treat, and/or sugar water. After a few days of controlling the rewards ...
1
vote
0answers
25 views

Why are Dueling Q Networks not used more often to approximate Q-values in reinforcement learning algorithms?

I've just learned about Dueling Network Architectures to estimate $Q$-values and am wondering why this architecture is not used more often in deep RL algorithms? DDPG and TD3 estimate the $Q$-function ...
1
vote
0answers
25 views

Do the variance and bias belong to the policy or value functions?

Recently, I read many papers on variance and bias. But I am still confused by the two notions, the variance or bias belongs to who? Policy or value? If the variance or bias is large or low, what ...
1
vote
1answer
64 views

Reinforcement learning with industrial continuous process

I am new to RL and wish to realize a RL control for an industrial process. The goal is to control the temperature and humidity in a vegetal food production chamber. States: External temperature and ...
1
vote
0answers
156 views

Replace epsilon greedy action selection and the standard DQN by an Independent Gaussian Noise Network Model

Here is my code Recently, I solved the game of Atari Breakout using a classic DQN model. The convergence of the mean reward slowly improved during three days. I was interested in learning a method ...
1
vote
0answers
36 views

How are n-dimensional vectors state vectors represented in Q-learning?

Using this code: ...
1
vote
0answers
42 views

NaNs after a while in training of PPO

My problem is that every time I am trying to train my PPO agent I get NaN values after a while. The diagnostic that I get is the following: ...
1
vote
0answers
75 views

Why isn't my DQN agent improving when trained on Atari Breakout?

Lately, I have implemented DQN for Atari Breakout. Here is the code: https://github.com/JeremieGauthier/AI_Exercices/blob/master/Atari_Breakout/DQN_Breakout.py I have trained the agent for over ...
1
vote
0answers
48 views

How to integrate dict space of OpenAI gym into a reinforcement learning framework?

I am implementing a gym environment and I have several input arrays as my input (different sizes). The most simple method to integrate my environment into the gym is to use the dict space as my ...
1
vote
0answers
19 views

Running a simple graph network example in gym

This a fix example to run in gym open ai ...
1
vote
0answers
23 views

What kind of artificial intelligence is this? A decentralized swarm intelligence where the input and output is split among the agents

I have an AI design for deciding the length of green and red lamps of the traffic. In my design, every crossroads has its own agent. This agent has input the amount of vehicle in each road in a single ...
1
vote
0answers
44 views

Why can the core reinforcement learning algorithms be applied to POMDPs?

Why can an AI, like AlphaStar, work in StarCraft, although the environment is only partially observable? As far as I know, there are no theoretical results on RL in the POMDP environment, but it ...
1
vote
0answers
33 views

How can I find the appropriate reward value for my reinforcement learning problem?

I am wondering how can I find the appropriate reward value for each specific problem. I know this is a highly empirical process, but I am sure that the value is not set totally at random. I want to ...
1
vote
0answers
27 views

Is the TD-residual defined for timesteps $t$ past the length of the episode?

Let $\mathcal{S}$ be the state-space in a reinforcement learning problem where rewards are in $\mathbb{R}$, and let $V:\mathcal{S} \to \mathbb{R}$ be an approximate value function. Following the GAE ...
1
vote
0answers
45 views

Efficient algorithm to obtain near optimal policies for an MDP

Given a discrete, finite Markov Decision Process (MDP) with its usual parameters $(S, A, T, R, \gamma)$, it is possible to obtain the optimal policy $\pi^{*}$ and the optimal value function $V^{*}$ ...
1
vote
0answers
21 views

What are the most common non-Markov RL paradigms?

I am interested in doing model-free RL but not using the Markov assumptions typical for MDPs or POMDPs. What are alternative paradigms that don't rely on the Markov assumptions? Are there any common ...
1
vote
0answers
46 views

How to define observation and action space for an array-like input?

I am working on a problem, and I want to implement it as a reinforcement learning problem and integrate it into the OpenAI's gym. My states are in the form of lists of length $n$, where each element ...
1
vote
0answers
81 views

How can I add logic for invalid moves when using stable-baselines in OpenAI's gym?

I want to integrate my environment into the OpenAI's gym and then use the stable baselines library for training it. The learning method in the stable baseline is with one-line learning and you don't ...
1
vote
0answers
29 views

How should I avoid illegal states in OpenAI's gym?

I'm trying to make a gym environment for a simulation problem. In my gym environment, I have a set of illegal states which I don't want my agent to go into them. What is the easiest way to add such ...
1
vote
0answers
36 views

Atari Breakout Infrastructure

This is how they describe their infrastructure in https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf. I want to implement the game of Atari Breakout. ...
1
vote
0answers
76 views

What are the pros and cons of deep learning and machine learning to develop a trading system?

As I want to start coding a new Trading AI in this year (first based on Python and later maybe in C++) I stumbled over the following question: Today, I would like to make a pro/contra list with you ...
1
vote
0answers
46 views

Why haven't we solved the problem of bipedal walking?

This has been a mystery to me. All the walking robots look like idiots now. But we do have a lot of simulation-based results (Flexible Muscle-Based Locomotion for Bipedal Creatures ), so why can't we ...
1
vote
0answers
48 views

Which model should I choose to maximise reward of having chosen two numbers from a list?

I am looking for a technique to train a machine learning model to choose two items from a list. So, given a list $x=[x_1, x_2, x_3, x_4, \dots, x_n]$, the model needs to choose two elements $(x_i, ...
1
vote
1answer
57 views

How is the probability transition matrix populated in the Markov process (chain) for a board game?

Following on from my other (answered) question: With regards to the Markov process (chain), if an environment is a board game and its states are the various position the game pieces may be in, how ...
1
vote
0answers
56 views

How should I define the state space for this life science problem?

I would like to ask for a piece of advice with regard to Q-learning. I am studying RL and would like to do a basic project applied to life science and calculate the reward. I have been trying to get ...
1
vote
0answers
49 views

Does SARSA(0) converge to the optimal policy in expectation if the Robbins-Monro conditions are removed?

The conditions of convergence of SARSA(0) to the optimal policy are : The Robbins-Monro conditions above hold for $α_t$. Every state-action pair is visited infinitely often The policy is greedy with ...
1
vote
0answers
16 views

Drone Deployment Platform for Neural Networks

Good day everyone, I would just like to ask if anyone part of a lab or company doing research on aerial robotics has any suggestions of a good platform for deploying computer vision algorithms for ...
1
vote
1answer
143 views

OpenAI spinning up convolutional networks with PPO

I am using pytorch version of PPO and I have image input that I need to process with convolutional neural networks, are there any examples on how to set up the network? I know that stable baselines ...
1
vote
0answers
25 views

Does apprenticeship learning require prospective data?

I am thinking of applying apprenticeship learning on retrospective data. From looking at this paper by Ng https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf which talks about apprenticeship ...
1
vote
0answers
73 views

Implementing Actor-Critic with Experience Replay for Continuous Action Spaces

I have been trying to implement the ACER algorithm for continuous action spaces in reinforcement learning. The paper for the algorithm can be found here: Sample Efficient Actor-Critic with Experience ...
1
vote
0answers
23 views

Can I apply experience on naive actor critic directly? Should it work?

Can I apply experience replay on naive actor-critic directly? Should it work? I have tried that but unfortunately it didn't work.
1
vote
0answers
32 views

How to encode board before input into the neural net?

Currently I'm working on an educational project (implementation of AlphaZero approach to different types of board games). My biggest concern at the moment is how to encode board before input into the ...
1
vote
0answers
39 views

What kind of enemy to train a good RL-agents

So I want to create an RL-agent for two players-board game. I want to use a simple DQN for the first player (my RL-agent). Then, what kind of algorithm that should I use on the second player (my RL-...
1
vote
0answers
65 views

How to represent a state in a card game environment? (Wizard)

We are attempting to build an AI that manages to play the cardgame Wizard. So far er have a working network (based on the YOLO object-detection) that is abled to detect which cards are played. When ...
1
vote
0answers
28 views

Can I solve this assignment problem with RL or AI planning, and if yes how?

I have a list of positive nonzero integers $T=[v_1,\dots,v_𝑛|v_𝑖\in Z^{\neq}]$ which sum up to $V=\sum_i v_i$. Typically, the length of T (number of integers) goes from 100 to 1000. The list is not ...
1
vote
2answers
868 views

Understanding the loss function in deep Q-learning

I am trying to understand how deep Q learning (DQN) works. To my current understanding, each $Q(s, a)$ functions is estimated to be a function of a feature vector of its state $\phi$(s) and the weight ...
1
vote
0answers
153 views

Expected SARSA, SARSA and Q-learning

I would much appreciate if you could point me in the right direction regarding this question about targets for approximate ...
1
vote
0answers
41 views

Reinforcement learning CNN input weakness

I'm trying to train a network to navigate a 48x48 2D grid, and switch pixels from on to off or off to on. The agent receives a small reward if correct, and small punishment if incorrect pixel plotted. ...
1
vote
0answers
21 views

Why we multiply probabilities with support to obtain Q-values in Distributional C51 algorithm?

In 'Deep Reinforcement Learning Hands-On' book and chapter about Distributional C51 algorithm I'm reading, that to obtain Q-values from the distribution I need to calculate the weighted sum of the ...
1
vote
0answers
33 views

Hyperparameter optimisation over entire range or shorter range of training episodes in Deep Reinforcement Learning

I am optimising hyperparameters for my deep reinforcement learning project (using PPO2, DQN and A2C) and was wondering: Should I find the optimum hyperparameters to get maximum reward from training ...
1
vote
0answers
15 views

How to perform Interpretability analysis toward a simple reinforcement learning network

We are currently using a RL network with the following simple structure to train a model which helps to solve a transformation task: Environment (a binary file) + reward ---> LSTM (embedding) --> FC ...
1
vote
0answers
23 views

Deciding std. deviation for policy network output?

When I try to fit a Normal Distribution to the output of a policy network, for a continuous action space problem, what should be its standard deviation? mean for the distribution will directly be the ...
1
vote
0answers
31 views

How would you differentiate between different on-policy reinforcement learning algorithms?

How would you differentiate between different on-policy reinforcement learning algorithms?

1
6 7
8
9 10
12