New answers tagged math
0
votes
What is the derivative of the Leaky ReLU activation function?
In terms of the Heaviside step function $$H(x)=\frac{1+sign(x)}{2}$$
the leaky ReLU can be written as
$$ (\alpha + (1-\alpha)H(x))x $$
so its derivative is
$$ \alpha + (1-\alpha)H(x) $$
In your ...
Top 50 recent answers are included