New answers tagged math
0
Both your notation and terminology are quite confusing. For example, I'm not sure what is an "optimal" Bellman operator is. Here's a good clarification on definition of a Bellman operator. Likewise, your description of the DQN algorithm completely ignores the averaging over states/actions/rewards sampled from the replay memory.
Trying to savage ...
1
There is no sign error and we should not change to $\arg\max$. With Policy Gradients I find that it is not useful to think about things such as a 'loss'.
In short, we want to first find the derivative of the RL objective $J(\theta) = v_\pi(s_0)$, where $\pi$ is our policy that depends on some parameters $\theta$. The policy gradient theorem tells us that
$$\...
0
After working on it for a while this is what I got.
Concerning proposition 1 in the paper, a rigorous statement could be the following version of the Gradient Theorem for line integrals:
Proposition 1. (Gradient Theorem for Lipschitz Continuous Functions).
Let $U$ be an open subset of $\mathbb{R}^n$. If
$F : U \to \mathbb{R}$ is Lipschitz continuous, and
$\...
Top 50 recent answers are included
Related Tags
math × 177machine-learning × 51
neural-networks × 36
deep-learning × 29
reinforcement-learning × 28
backpropagation × 12
proofs × 10
notation × 10
gradient-descent × 9
activation-function × 9
convolutional-neural-networks × 8
loss-functions × 8
terminology × 7
definitions × 7
algorithm × 7
function-approximation × 7
papers × 6
getting-started × 6
ai-design × 5
datasets × 5
agi × 5
optimization × 5
applications × 5
policy-gradients × 5
natural-language-processing × 4