I started to learn machine learning early, and I studied the convolutional neural network and its ability to understand images and how it helps to reduce the number of parameters that need to be tuned.
So, if I used CNN rather than a feed-forward network for the image classification problem, how many neurons and layers would I save or discard?
I don't have an exact answer to your question because I think your question is a bit ambiguous. It's like asking "How many defenders do you need to replace your forwards?": they do slightly different things, but a defender occasionally may be able to play as a forward. Anyway, given that I think this question arises because you don't understand how we can view a CNN as an FFNN or vice-versa, I will try to give you an idea of the relationship between the two and the number of parameters in a CNN and an FFNN.
Before proceeding, read my answer here (and maybe the other answer there too), which will give you an idea of what we could consider a neuron in a CNN, and how we can compare CNNs to FFNNs. After that, you could read this answer to understand why we need CNNs and why we use them in the context of computer vision. From now on, I will assume that you have an idea of how CNNs work, what filters are, etc., but I will remind you of a few details.
CNNs are specifically designed to deal with images, so CNNs have a more appropriate inductive bias than FFNNs for image processing.
A specific filter/kernel in a CNN can be thought of as extracting the same type of feature from different parts of the image. At a high level, this makes sense, because the same object/feature (e.g. an eye) can appear in different parts of an image. If you are familiar with image processing techniques, this interpretation of the role of kernels in CNNs shouldn't be too strange.
One of the main properties of CNNs, as explained in the linked answers, is that of weight sharing. This means that you apply the same filter (weights) to different parts of the image (this is basically what the convolution operation is doing). So, you try to extract the same "features" with the same kernel from different parts of the image.
Now, imagine if you didn't do that, and you had to learn different filters for different parts of the image: we would need weights for all these different parts, so we would need a lot more weights.
In the case of an FFNN, we can view the operation of a fully connected layer as a convolution with a kernel as big as the input image. More precisely, if you have $n$ neurons in layer $l$ and $m$ neurons in layer $l-1$, then you can view this FC layer as performing $n$ convolutions, one for each neuron in $l$, with a kernel with $m$ weights.
At this point, you should start to see the relationship between fully connected layers and convolutional layers.
So, what's the answer to your question? I don't know. It depends on what you mean by "equivalent". If by "equivalent" you mean they are the same hypothesis class, then that's a question to which I don't have an answer. However, both FC and CNNs are universal function approximators, so, in a way, they are equivalent, exactly how (which is your question), I don't know and this is probably an open research question. One thing we can say is that CNNs have a more appropriate inductive bias than FFNNs to deal with images.
