New answers tagged reinforcement-learning
1
What you're looking for is called Imitation Learning (IL), in which we are interested in learning an expert policy $\pi_*$ which we assume to be optimal.
However, there are many different ways we can approach such learning setting.
Just to give some examples, we might be interested in Behavioural Cloning, where our parametrised policy $\pi_\theta$ (the RL ...
0
Most OpenAI gym environments are thin wrappers around existing games and libraries. You could do the same with your game. See e.g. https://towardsdatascience.com/beginners-guide-to-custom-environments-in-openai-s-gym-989371673952 for a tutorial. There are many others, you can search "open ai gym custom environment" for more
3
A relatively recent but interesting paper that discusses this topic in more detail is Reward is enough (Artificial Intelligence, 2021) by David Silver, Satinder Singh, Doina Precup, and Richard S. Sutton (so by some of the godfathers of RL, who are all at DeepMind).
Their reward-is-enough hypothesis (RIEH) (page 4) is
Hypothesis (Reward-is-Enough). ...
1
If your goal is to create a controller for the mountain car problem, and you have access to the model, then RL probably offers no advantage over your code. I am saying probably, because I am taking you at your word that the code performs well over multiple tests, and it doesn't matter too much if it does not because there are many equivalent solutions based ...
1
My question is if I should select state_action pairs by theirs immediate reward or should I select them by the episode reward?
By the return (sum of all rewards) from the whole episode. A lot of decisions made in "good" episodes do not lead to immediate rewards, but instead transition towards states where better rewards are possible.
In retrospect,...
0
the answer is adding lambda inputs: inputs["your_key_for_observation"] to the network in case someone encounters this issue in the future
1
I am not sure how in-depth the information you want need to be, but maybe I can share some thoughts that may help you!
Environment
I would highly recommend using OpenAI Gym with your environment, since most of the already implemented RL-algorithms are designed to use gym environments.
You can design your environment any way you want to and then use gym to ...
1
This is simply overfitting. The model is performing well on train data but bad on test (unseen) data; you can measure it also by noticing a huge difference between the training accuracy and the validation accuracy. Of course this is not a natural behavior, to solve this you need to apply some data or network modifications in order to avoid overfitting. Some ...
0
I found the solution, it was changing reward function and using reward scaling. A little bit change in architecture and learning rate fixed the problem.
1
This is the original Q-Learning paper by Watkins, though you may need to pay for access to this.
This is the Nature paper that introduced the DQN.
2
An experimental paper exist in arxiv about the effect of whether to mask or to give negative rewards to invalid actions. There are some references in this paper which also discuss the effects and the mechanism to handle invalid actions. However, those main references are still only pre-prints in the arxiv (not published and presumably not peer-reviewed yet).
...
1
The value of the objective depends on policy (probabilities of taking an action). Intuitively speaking, better actions lead to better returns and by "pushing up" the probabilities (log or not same thing since log is monotonically increasing function) of those actions you're making sure you're getting better returns and increasing the value of your ...
3
Scale your neural network inputs.
The raw observations are in range $[0,89]$, and neural networks will cope badly with that used as inputs.
The ideal case for NN for each input feature is a gaussian distribution with mean 0, standard deviation 1. You don't need that to be perfect, though. A simple scale - divide each element by $30$ and subtract $1.5$ - will ...
1
First of all, the support of a normal distribution is the entire real line (or, in general, $\mathbb{R}^n$ for an $n$-dimensional multivariate normal distribution) so your action can be any number in $\mathbb{R}$. What you may be getting confused with is that with probability 0.68 you will obtain an action that is within +/- 1 standard deviation from the ...
1
First of all, we assume that we have a finite MDP, i.e. the set of states $\mathcal{S}$, the set of actions $\mathcal{A}$ and the set of rewards $\mathcal{R}$ all have a finite number of elements (I didn't think about how the explanations below would extend to other cases, but I suspect you will need differential equations).
For simplicity, let's only ...
3
Provided you have a finite number of states and actions, then there will not be an infinite number of terms. Therefore the state and action spaces need to be discrete and finite before the quote from the book applies.
I am having a hard time understanding how one could solve this system of equations.
There are a few techniques for solving simulteneous ...
1
In case the question is if NNs can be trained without data, as pointed by others, the answer is negative - any training by definition involves the use of data in some way - supervised, semi-supervised, reward, etc.
However, if the question is whether one can obtain something useful I would think about the following use cases:
One can use randomly ...
2
Neural networks are trained by using pairs of example input/output vectors that they learn to associate and can generalise from. In that sense, they always need training data.
For supervised learning, a neural network (NN) is trained on a dataset of example inputs and outputs (aka "a labelled dataset") that the user must provide somehow.
There are ...
5
You cannot train a neural network without training data. It would be like training a football player without making him/her play/watch football or anything that resembles football: it's simply not possible. The definition of training/learning in machine learning strictly requires data.
You can train a neural network in different ways (e.g. supervised or ...
0
This is considered in the original paper, but is rejected due to training instability. To quote:
[The average-advantage formulation] increases the stability of the optimization...the advantages only need to change as fast as the mean, instead of having to compensate any change to the optimal action's advantage in [the formulation where $Q(s, a) is ...
3
I'll assume Q-player is being trained with Q learning (note, Q tables can be useful in other algorithms too, like SARSA).
Q learning is an off policy algorithm, meaning that the Q values can be learned regardless of the policy used to collect data. So the Q player can be following a random policy, or even a fixed pre defined policy if you want. Usually, ...
0
The policy is used in determining the next sequence of state-action pairs in the next episode. This means that the policy is determining indirectly the next Gt
Top 50 recent answers are included
