3
votes
Accepted
Why are only neural networks (and not SVMs, for example) used for reinforcement learning?
The biggest problem with SVMs, random forests, gradient boosting and others for reinforcement learning (RL) is that they are not able to learn online, adjusting for new data as it arrives, and equally ...
- 26.6k
2
votes
Accepted
Why does the average-reward estimator for continuing tasks use the TD error?
Mystery solved thanks to Exercise 10.8 in the book. The reason is that we want the running mean to converge to the actual value of the average reward.
With $\bar{R}_{t + 1} = \bar{R}_t + \beta \delta$...
- 86
2
votes
Why are policy gradient methods more effective in high-dimensional action spaces?
Above softmax in action preferences is used for policy gradient methods with (large) spaces with discrete actions, while for continuous spaces with infinite number of actions Gaussian distribution is ...
- 561
1
vote
Accepted
The reason behind using MCTS over Alpha Beta Pruning in Alphazero
Let's begin with what position evaluation means, as it is the core of everything.
AlphaZero evaluates positions using non-linear function approximation based on a deep neural network, rather than the ...
- 215
1
vote
Can I minimize a mysterious function by running a gradient descent on her neural net approximations?
Yes, this is a standard approach. An improvement is to do gradient descent on $F$ (not $F_1$), but use the gradient of $F_1$ as your estimate for the gradient of $F$. In other words, when you ...
- 266
1
vote
Can I minimize a mysterious function by running a gradient descent on her neural net approximations?
I do not know any specific name for this method, but it is a common approach for approximating and optimizing complex functions. You can find an industrial use-case of this approach in this paper (...
- 1,723
1
vote
PRNG Function Prediction
Yes, this is possible but only within certain cases and depending on the amount of work one is willing to invest. All PRNGs make use of an underlying (deterministic) generative process that can be ...
- 373
1
vote
Accepted
Could Softmax Action Selection be useful to solve an episodic task with more than 100000 possible states and 2000 actions?
Your question contains the answer. Use value function approximation. Tabular methods must compute a value for each state. That becomes unfeasible with large state spaces. Function approximators can ...
- 2,005
1
vote
Accepted
Unclear points for polynomial basis for function approximation
Q1- Why is each $x_i$ an "order-n" polynomial? I think this is wrong: in my opinion, order of $x_i$ can be in the range [1, n*k]
The text does not claim that $x_i$ is an "order-n" ...
- 26.6k
Only top scored, non community-wiki answers of a minimum length are eligible