# How can I "measure" an object using Computer Vision techniques and neural networks?

I would like to develop a neural network to measure the distance between two opposite sides of an object in an image (in a similar way that the fractional caliper tool measures an object).

So, given an image of an object, the neural network should produce the depth or height of the object.

Which computer vision techniques and neural networks could I use to solve this problem?

Seriously -- if you have stereo images it should be possible, since that's what we use for depth perception. If you know how far away points x1 and x2 are, then you can measure distance using trigonometry. No neural networks needed, I guess. https://en.wikipedia.org/wiki/Triangulation_(computer_vision)

If the measurements you want from the object aren't too complicated (ie. length of a clearly defined feature), and if you are able to acquire a training dataset of images of the objects similar to what your model will see in your use case (same scale/distance), their bounding boxes and their measurements, a model you could try to implement is a Multi-Task Convolutional Neural Network (MTCNN).

MTCNNs are typically used for face detection and alignment, but I would imagine it is possible to adapt them to your use case given proper training and tuning. If there are more complicated measurements that you want to obtain, you could pass on the detected objects to another model to make more specific measurements.

You will have a problem however, with measuring depth. Depth is hard to estimate from an image because of the information that we lose when moving from a 3D to a 2D space. A longer explanation on this is available in MachineEpsilon's answer to the Cross Validated question "how to detect the exact size of an object in an image using machine learning?" but quoting his main statements:

This task of depth estimation is part of a hard and fundamental problem in computer vision called 3D reconstruction. Recovering metric information from images is sometimes called photogrammetry. It's hard because when you move from the real world to an image you lose information.

Specifically, the projective transformation 𝑇 that takes your 3D point 𝑝 to your 2D point 𝑥 via 𝑥=𝑇𝑝 does not preserve distance. Since 𝑇 is a 2×3 matrix, calculating 𝑇−1 to solve 𝑇−1𝑥=𝑝 is an underdetermined inverse problem. A consequence of this is that pixel lengths are not generally going to be meaningful in terms of real world distances

However, that's not to say you could add additional sensors to resolve the depth estimation problem (ie. stereoscopic cameras or infrared distance sensors) if additional cost is not an issue.