I am reading this paper about Group Equivariant Convolutional Networks. Basically, it is a CNN whose construction makes the network naturally equivariant to Group transformations (e.g. rotations) of the input.
This is, a GE-CNN trained with the Rotations Group in its architecture, will for instance predict correctly the label of a rotated MNIST digit, even though it was never trained on rotated MNIST digits.
My question here is, are there any situations in which these GE-CNN will not perform better than a regular CNN? In other words: is the property of being equivariant to any transformation of the input always desirable?