Now, the main idea behind ResNet is to enable the network to learn the residual value needed to modify the input for achieving the best result. In other words, given an input x, the output y is computed as y = F(x) + x, and the goal is to learn the function F (the filter). If the ideal output is the identity function, it implies that the model has discovered the best filter to be all zeros. This situation could arise due to bad initialization, resulting in the best filter being all zeros. Alternatively, it could be the optimal solution if there are no more residual blocks after it. If this is not the optimal solution, we can reinitialize the filters and retrain them.
I would greatly appreciate any guidance or corrections.