I'm new to NN, and trying to understand some of its foundations. One question that I have is, why the **derivative** of an activation function is important (not the function itselt), and why it's the derivative which is tied to how the network performs learning? 

For instance, when we say a constant derivative isn't good for learning, what is the intuition behind that? Is the activation function somehow like a **hash function** that needs to well differentiate small variance in inputs?