3
If $t^i - o^i$ is negative, doesn't the power of 2 eliminate any negative result?
In the loss function, yes that is correct, and is what you want - a measurement that gets higher due to any difference between the predicted and correct results. Minimising the value of that measurement is a goal for the optimiser.
How can then exist any negative gradient at ...
3
By convention, the $\mathrm{ReLU}$ activation is treated as if it is differentiable at zero (e.g. in [1]). Therefore it makes sense for TensorFlow to adopt this convention for tf.nn.relu. As you've found, of course, it's not true in general that we treat the gradient of the absolute value function as zero in the same situation; it makes sense for it to be an ...
2
Yes, if the activation function of the network is not zero centered, $y = f(x^{T}w)$ is always positive or always negative. Thus, the output of a layer is always being moved to either the positive values or the negative values. As a result, the weight vector needs more update to be trained properly and the number of epochs needed for the network to get ...
1
Creating custom gradient for tf.abs may solve the problem:
@tf.custom_gradient
def abs_with_grad(x):
y = tf.abs(x);
def grad(div): # Derivation intermediate value
g = 1; # Use 1 to make the chain rule just skip abs
return div*g;
return y,grad;
1
In general, if you have a composite function $h(x) = g(f(x))$, then $\frac{dh}{dx} = \frac{d g}{df} \frac{d f}{dx}$. In your case, the function to differentiate is
$$L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim U(D)} \left[ \left(r+\gamma \max_{a '} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}) \right)^2 \right]$$
So, we want to calculate $\nabla_{\theta_i}...
Only top voted, non community-wiki answers of a minimum length are eligible
