Are there relatively new research papers that describe how to make back-propagation more efficient?
So, I am looking for papers similar to Efficient Backprop by LeCun but newer. The papers could describe why ReLU now "dominates" tanh or even sigmoid (but tanh was Yann's favorite, as explained in the paper). ReLU is just one thing I am interested in, but the paper could also analyze e.g. the inputs from a statistical standpoint.