New answers tagged reinforcement-learning
0
You could try using a Reinforcement Learning approach in an appropriate "Supervised" Kaggle competition, to beat other people that are using a purely Supervised ML approach. If you win, you'll get exposure in the winners blog and you can talk about your use of RL. And if you don't win, you could share your notebook publicly and inspire others, start ...
0
There is not only reinforcement learning competitions but some are : https://www.aicrowd.com/challenges
0
OpenAI has leaderboards for their gym-environments, if you want to compete with other people on runtime and efficiency.
https://github.com/openai/gym/wiki/Leaderboard
Is this what you are looking for?
1
logp seen in code is actually logit p which has this story behind:
Given a probability p, the corresponding odds are calculated as p / (1 – p). For example if p=0.75, the odds are 3 to 1: 0.75/0.25 = 3.
The logit function is simply the logarithm of the odds: logit(x) = log(x / (1 – x)).
Sigmoid near logp is like follows:
The inverse of the logit ...
0
We can manipulate a model's test data set if the machine learning model takes user input and uses it to resample test data set. The actual training dataset of the ML model does not get manipulated, but if we figure out the ML model through an exploratory attack (sending a lot of inquiries to the ML model to find out its nature), we can generate a training ...
1
In deep reinforcement learning for portfolio optimization, many researchers (Xiong at al. for example) use historical market data for model training. The resulting MDP dynamics is of course completely deterministic (if historical prices are used as states) and there's no real sequentiality involved.
Whilst I cannot comment on the specific financial model, I ...
0
This is what I got from the manual :
The value returned by run() has the same shape as the fetches argument,
where the leaves are replaced by the corresponding values returned by
TensorFlow.
sess.run(fetches, feed_dict=None, options=None, run_metadata=None)
Example:
python
a = tf.constant([10, 20])
b = tf.constant([1.0, 2.0])
# 'fetches' can be ...
0
In adverserial machine learning, someone (program or human) attempts to fool an existing model with a malicious input.
The best human example would be an optical illusion. The human brain's model for image processing starts outputting wrong information when looking at an optical illusion. So in the end we see wrong colour, shape, etc. In this case, the ...
1
They don't have acces to the original training or test dataset. Machine learning environments are build on the premise of a benign environment. The models are trained on real data (real inputs). When someone sends a made up input (fake input) it is very easy to fool the model.
This is used for example in image recognition. Imagine a fotograph of a panda. ...
2
RL can be used in the context of Neural Architecture Search (NAS), with is a form of automated ML. A model searches for an architecture that performs a given task. How well this task is performed guides how the architecture will be modified (improved) on the next pass. It works but is very computation-intensive (think hundreds of GPUs).
See for instance:
B....
1
In step 2 I need to decide for an initial estimate $\tilde{Q}_n$. Is it a decent option to use $\tilde{Q}_n=Q_{n-1}$?
Yes, this is a common choice. It's actually common to update the table for $\tilde{Q}$ in place, without any separate initialisation per step. The separate phases of estimation and policy improvement are easier to analyse for theoretical ...
3
Automated machine learning (AutoML) is an umbrella term that encompasses a collection of techniques (such as hyper-parameter optimization or automated feature engineering) to automate the design and application of machine learning algorithms and models.
Reinforcement learning (RL) is a sub-field of machine learning concerned with the task of making ...
1
You cannot do this:
$\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\bigtriangledown log \pi (\tau )]$
That is because $r(\tau )$ and $\bigtriangledown log \pi (\tau )$ are correlated by their dependence on $\tau$. In a simpler concrete example, if your expectation ...
2
So, what is the purpose of the new index for $V$ in Chapter 7, and why is it more important at this particular chapter?
My guess would be that your intuition is correct, and that it's mostly introduced just to clarify exactly which "version" of our value function approximator is going to be used in any particular equation. In previous chapters, which ...
0
I'll share my understanding so far.
This kind of behavior is actually normal when using on-policy algorithms with a sparse final reward.
Issue stems from the fact that once you get stuck in a behavior policy which does nothing (uses a "do nothing" action, for instance, until timeout), it's quite hard to get out of it, because you keep getting experiences ...
0
Some of the popular deep learning algorithms used in IoT retail space include LSTM for time-series prediction and CNN for image analysis.
Reinforcement Learning (RL) in Artificial Intelligence includes algorithms that work in an environment to take decisions to maximize the cumulative reward and improve learning efficiency. RL could show to slot machine (or ...
1
From your description, it seems that you are implementing a version of an algorithm called REINFORCE. This algorithm belongs to a family called Policy Gradient methods, which directly optimizes the policy network $\pi(a_t|s_t)$ from rewards without ever worrying about estimating a value function. This type of algorithm is usually pretty slow and presents ...
2
Dealing with a Non-Markovian process is unusual in Reinforcement Learning. Although some explicit attempts have been made, the most common approach when confronted with a non-Markovian environment is to try and make the agent's representation of it Markovian.
After reducing Agent's model of the dynamics to a Markovian process, rewards are assigned from the ...
1
Page 6 of the paper describes the exact reward functions, and why they were used:
Goals: Goals describe the desired position of the object (a box or a
puck depending on the task) with some fixed tolerance of $\epsilon$ i.e. $G = \mathcal{R}^3$
and $f_g(s) = [|g − s_{object}| ≤ \epsilon]$, where $s_{object}$ is the position of
the object in the ...
2
Although you can frame your problem as a bandit problem or RL, it has other workable interpretations. Critical information from your comments is that:
Total reward is not a simple sum of all the results from 66 different machines. There are interactions between machines.
Total reward is deterministic.
This looks like a problem in combinatorial optimisation....
2
Hi Seewoo Lee and welcome to our community!
The essence of your observation is that Sutton's version of REINFORCE is taking into consideration all of the trajectory to compute the returns while in the pytorch version only the future is taken into consideration, hence going in reverse to sum the future rewards and ignore the previous rewards. The consequence ...
1
https://github.com/openai/retro
Current list of machines is
Atari
Atari2600 (via Stella)
TurboGrafx-16/PC Engine (via Mednafen/Beetle PCE Fast)
Game Boy/Game Boy Color (via gambatte)
Game Boy Advance (via mGBA)
Nintendo Entertainment System (via FCEUmm)
Super Nintendo Entertainment System (via Snes9x)
GameGear (via Genesis Plus GX)
Genesis/Mega Drive (via ...
0
Just use one class inheriting from nn.Module called e.g. ActorCriticModel.
Then, have two members called self.actor and self.critic and define them to have the desired architecture.Then, in the forward() method return two values, one for the actor output (which is a vector) and one for the critic value (which is a scalar).
This way you can use only one ...
2
The double sampling problem is referenced in Chaper 11.5 Gradient Descent in the Bellman Error in Reinforcement Learning: An Introduction (2nd edition).
From the book, this is the full gradient descent (as opposed to semi-gradient descent) update rule for weights of an estimator that should converge to a minimal distance from the Bellman error:
$$w_{t+1} ...
1
Unfortunately no, the way to go is track the total reward and see if it's increasing and converging eventually. Value loss isn't a useful metric as the loss can be 0 when the value network always predicts 0 and the agent doesn't collect any reward, meaning very poor behavior.
0
I posted the question to the project's issue page, where the recommendation was to manually download and run run_demo.py. That worked for me.
3
I believe this is a pedagogical decision. Because the sentence occurs in the first chapter of the book, I think the authors are trying to avoid the objection that a neophyte might make: learning from random movements seems like it will cause you to learn strange behaviors.
Certainly, the statement is inaccurate. We need only reach page 26 to see a ...
1
Is it possible for value-based methods to learn stochastic policies?
Yes, but only in a limited sense, due to the ways it is possible to generate stochastic policies from a value function. For instance, the simplest exploratory policy used by SARSA and Monte Carlo Control, $\epsilon$-greedy, is stochastic.
SARSA natually learns the optimal $\epsilon$-...
0
One thing you could try to simplify the output logic would be to use a softmax output and then with your outputs set a var to = (max_output - min_output)/2 then treat that number as your long/short "threshold" and this ensures that your ouput always sums to 1 while still allowing the net to learn to output short signals. I would also check to make sure you ...
2
So does that mean, that the input of the first hidden layer was simply the state and the input of the second hidden layer the output of the first hidden layer concatenated with the actions?
Yes.
Why would you do that? To have the first layer focus on learning the state value independent of the selected action? How would that help?
Neural networks hidden ...
3
My first question is whether the following "implementation" of the 𝑇𝐷(0) algorithm for the first two of the above observed trajectories correct?
$V(a)\leftarrow0 + 0.1(1+0-0)= 0.1; \quad V(b)\leftarrow0+0.1(1+0-0)=0.1$
$V(b)\leftarrow0.1+(0.1)(1+0-0.1)= 0.19$
Your calculations for the first trajectory $(A,1,B,0)$ is incorrect for either TD ...
Top 50 recent answers are included
