Why Fully-Connected Neural Network is not always better than Convolutional Neural Network?
The main reason why in many cases, a CNN will outperform a fully-connected (FC) neural network, i.e. MLP, is grounded in symmetry. For example, a CNN responds very naturally to image translations. This behavior is called translational equivariance which arises from shifting the same weights (i.e. the convolutional kernel) over the space of an image. The result is a model that - when trained on images where an object is in the center - will generalize to images where an object is off-center. This happens even though such images are not in your training set. An MLP cannot do this, it would have be trained on a dataset containing the objects in all possible positions in order to generalize nicely.
As a side note: Some architectures like the transformer use FC layers in the style of a convolution as well, because the exact same weights are applied to all input tokens individually. So a model doesn't have to be your typical 2D-ConvNet to be a type of CNN.
FCNN is easily overfitting due to many params, then why didn't it reduce the params to reduce overfitting.
The model does not 'know' that it is overfitting because all it ever sees in training is the training dataset and an overfitted model performs great on this data. Another way to look at this is to view backpropagation as finding the path of least resistance to the optimal model. In many cases (especially with large models for small datasets) it is easier for the model to memorize the training samples instead of finding a very general solution.
We know that fully connected means better.
All layers have benefits and drawbacks, you can't really call any layer generally better, a fully-connected layer can do some things better than a CNN and other things (e.g. images) not so much.
And I think kernel size 1×1 and stride 1 in CNN (which basically equal same as FCNN) is performing better even though overfitting.
An overfitted model might look nice in training, but as soon as you show the model novel samples, these are much more likely going to be misclassified. Especially messy real-world data will likely result in bad model performance when you actually want to use such a model.