Detection of similar objects is a categorization problem, but not of the simplistic kind that can be done in machine learning through training.
A probable effort reducing option is to exploit the idea that, once a similar pair can be found and a certainty can be attached to their similarity, the grouping algorithm is pure statistically derived algebra and arithmetic. A likely difficulty in completing this project with current technology is that translating an essentially conceptual idea into software is not a purely machine learning problem and not a purely logical one. It will require prowess across a number of AI disciplines.
Similarity in this project's context is not like numerical similarity, textual similarity, shape similarity, color similarity, or even visual similarity. The quantity of image pairs that would be needed to train a network deep enough to work from camera input directly into physical similarity is beyond the current convergence technology.
The visual systems of mammals probably don't make that direct transition in one functional step either. It is likely that 2D motion is translated into physical (3D) motion and that the ability for humans to be able to recognize object similarity in photos is an extension of that motion oriented translation of 2D into 3D. The features of edges, texture, shading, and the obscurity of far away objects by closer ones is an advantage in object recognition, not yet exploited by many AI software products.
Once there is a recognition of three dimensional form, optical characteristics, and texture, independent of proximity, orientation, partial obscurity, and lighting, then similarity detection can proceed in a way consistent with the project objective.
Another challenge in this project is that similarity is subjective. A person unfamiliar with traditional taxonomy would consider a coyote similar to a jaguar. Those with taxonomy education might consider a coyote similar to a dog and a jaguar similar to a cat. A bacteriologist might consider all four mammals similar.
If you solve both the mathematical imprecision of similarity and the 2D to 3D problems together and finish the project with the procurement of a system with reasonable accuracy and reliability, working out the expression of such a system mathematically and publishing would be indicated. Such would be worthy of notice.