pecey
  • Member for 1 year, 9 months
  • Last seen more than a month ago
A neural network with 2 or more hidden layers is a DNN?
Accepted answer
1 votes

I don't think there is a fixed threshold that differentiates between Shallow and Deep Learning, but I would say that a 2 layer NN should not be considered deep. But now-a-days, almost all NN ...

View answer
How is trajectory sampling different than normal (importance) sampling in reinforcement learning?
1 votes

Here is my understanding: In trajectory sampling as the book describes it, we use the current policy on the simulator to get (next-state, action) pairs. The advantage is that if some states occur more ...

View answer
Should I build an environment from scratch myself or it is not always needed?
1 votes

I guess it would always be better if you can reuse existing environments to make it work for yourself. Since most of the environment codes is anyway opensourced, you can always edit it to your liking. ...

View answer
How does Monte Carlo Exploring Starts work?
0 votes

$Q(s,a)$ denotes the $Q-value$ for the state-action pair. It means the expected returns if we start from state $s$, take action $a$, and act according to whatever policy we are currently following. ...

View answer
How do I derive the gradient with respect to the parameters of the softmax policy?
Accepted answer
3 votes

Softmax policy $\pi_\theta(s,a)$ is defined as $\frac{\exp{(\phi(s,a)^T \theta})}{\Sigma \exp{(\phi(s,a) ^T \theta) }}$, where the summation is over the action space. Taking log, this becomes $$ \log \...

View answer
Q table not converging for an arbitrary experiment
0 votes

If your intention is to learn make the agent learn which has the min arbitrary value, then you would need to modify your rewards a bit. The current reward structure provides the incentive to just ...

View answer
How to calculate the advantage in policy gradient functions?
1 votes

The advantage is basically a function of the actual return received and a baseline. The function of the baseline is to make sure that only the actions that are better than average receive a positive ...

View answer
Why aren’t heuristics for Connect Four Monte Carlo tree search improving the agent?
1 votes

It might be the case that if you perform a large number of random rollouts, the "best action" as chosen by the agent without the domain knowledge, is same as the agent with the domain knowledge. I ...

View answer
Calculating accuracy for cross validation
2 votes

I guess you could train your model with 10 different folds and in each fold calculate the average accuracy. So you would have 10 values - one corresponding to each fold. And then take the mean of all ...

View answer
Is my understanding of the value function, Q function, policy, reward and return correct?
1 votes

I think most of it is correct. Q function(also called state-action value, or just action value): How good it is to be in a state S and perform action A while following policy π. It uses reward to ...

View answer
Is the Q value the same as the state-action pair value?
1 votes

I don't understand your question very clearly. Q-value of a particular state-action pair (s,a) under policy $\pi$ is the total reward you would expect to collect if you start from the state s, take ...

View answer