Rotated MNIST is a popular dataset for benchmarking models equivariant to rotations on $\mathbb{R}^2$, described by $SO(2)$ group or its discrete subgroups like $\mathbb{Z}^{n}$:
It consists of all digits from 0 to 9 rotated on an arbitrary angle from $[0, 2 \pi)$. However, what makes me a bit puzzled is that digits $6$ and $9$ seem to be confused by any learning algorithms, since from the view of human perception $6$ rotated by 180 degrees is equivalent to $9$ and vice versa.
The original paper in the description of Rotated MNIST doesn't comment on this point at all, which is strange, since it is a very natural question to ask.
In the paper Oriented Response Networks - authors plot embeddings of rotated digits projected via t-SNE on a 2d plane. There is a clear separation between all rotated versions of 6 and the rotated version of 9 for ORN.
I do not understand how it can be achieved? Probably, the networks understand much more in writing the digit, there are some subtle features, inaccessible to humans, but recognizable by powerful classifier?