How to use AI to depth map video? [closed]

Question

To be honest, I had no idea where to put this question, but it's sure that it's related to AI. I want to build an application which uses camera, and by the movement it can calculate the -camera's position compared to the objects -the objects creator and edge points by the movement.

What it means that if the camera is in a static position, it's just a picture. A set of coloured pixels. If we move the camera, we calculate the time, the gyroscope's values, but most importantly, we can have a comparison of two images taken by the same objects. This way: -we can detect the edges -from the edges, we can detect which is closer than the others

Today's phone camera's are accurate enough to create ~60 crystal clear images per second, and it should be enough resource to accurately create high res models from just moving the camera according to some instructions (that's why I'm surprised why it isn't existing in just a phone app). Here comes the problem. I think the idea is worth for the try, but I'm just a JavaScript developer. The browser can have access to the camera, with TensorFlow I can use machine learning to detect edges, but if I want to be honest, I have no idea where to start, and how to continue step by step. Can you please provide me some guidelines how it would be ideal to create the idea?

Answer 1

I think that the tasks you are referring to are the Simultaneous localization and mapping (SLAM) and, in particular, the Structure from Motion (SfM).

These methods are usually based on geometrical constraints and do not employ neural networks, but there exist some recent methods that make use of CNNs (such as this one).

Structure from Motion algorithms are a fundamental component of the Google ARCore (former Project Tango) and the Apple ARKit. Unfortunately, these kits usually provide an interface to put elements in a specific position, but they don't provide the access to the 3d reconstruction of the scene.

If you'd like to play with SfM algorithms, I suggest you to start from this repository, but you can find a lot of other valuable tutorials online.