I have read what the loss function is but I am not sure if I have understood it. For each neuron in the output layer the loss function is equal most usually to the square of the difference value of the neuron and the result we want. Is that correct? Most sites don't help me understanding so answer would be appreciated a lot.
A loss function is what helps you "train" your neural network to do what you want it to do. A better way to word it to begin with would be an "objective" function. This function describes what objective you'd like your neural network to fit to (or to be good at).
The loss function that you've described is "squared error", which, as the name suggests, is the squared difference between the expected output and the output from the neural network. This trains the network to match the expected output value.
Other loss (or "objective") functions could train your network to look for different things. For example, training on cross entropy loss helps your network learn certain probabilities. That's why it's usually used for classification, like when you want to determine which digit from 0-9 was fed into your MNIST classifier.