Then how do each filter differ by? Is it in hovering over the input matrix? Or is it in the values contained by filter itself? Or differs in both hovering and content?
The filters (aka kernels) are the learnable parameters of the CNN, in the same way that the weights of the connections between the neurons (or nodes) are the learnable parameters of a multi-layer perceptron (or feed-forward neural network).
So, the value of these filters is not fixed or pre-determined, but will depend on how you train the CNN, i.e. the learning algorithm, the objective function and the data. If you use gradient descent as the learning algorithm, you will be minimizing a loss (aka cost or error) function (e.g. the cross-entropy, in the case of classification problems). To do that, you need to find the gradient of the loss function with respect to the filters. You then apply a step of gradient descent (i.e. you add a scaled version of the gradient of the loss function with respect to the parameters to the parameters), so that this loss decreases.
To answer your question more directly, the only thing that usually changes is just the value of the filters. The convolution (or cross-correlation) operation is the same for all filters.
Why do you use more than one filter? The usual explanation is that each filter, when convolved with the input, will extract different features from it, and the specific features that they will extract will depend on the specific values of the filters, which, in turn, depend on the data, so we can say that CNNs are data-driven feature extractors. If you are familiar with image processing techniques, then you know that different filters, when convolved with the same image, can have different effects (e.g. blurring or de-noising).
