1
$\begingroup$

I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?

$\endgroup$
2
$\begingroup$

ELU does not suffer from from dying neurons issue, unlike ReLU. While ELU can help you to acheive a better accuracy, it is a slower than ReLU because of its non linearity in its negative range.

Choosing a right activation function totally depends on situations but you need to also consider other similar types of activation function such as leaky ReLU.

Check this link out. It could be useful.

| improve this answer | |
$\endgroup$
0
$\begingroup$

The answer above makes some great comparisons/trade-offs. To help address the non-linearity issue with eLU units that the previous answer brings up, you can also use Leaky-ReLU units, which are linear in both the positive and negative range, and piecewise-linear across the whole real domain.

Please see the link here for more details: https://qr.ae/pNsatH.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Note that the answer you mention already mentions leaky ReLU. $\endgroup$ – nbro Jul 29 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.