2
votes
Accepted
Why use ReLU over Leaky ReLU?
Your understanding or Leaky ReLU is correct, and, yes, it has been proposed to mitigate the dying neurons issue in ReLU: when these are negative, they got zeroed.
Regarding the answer of @Regresslt:
...
- 1,243
1
vote
Why use ReLU over Leaky ReLU?
Leaky ReLU is indeed an improvement over the standard ReLU activation function, but comes with some of the following limitations:
It may suffer from the "dying ReLU" problem, where a large ...
- 11
Only top scored, non community-wiki answers of a minimum length are eligible