3
$\begingroup$

To be honest, I had no idea where to put this question, but it's sure that it's related to AI. I want to build an application which uses camera, and by the movement it can calculate the -camera's position compared to the objects -the objects creator and edge points by the movement.

What it means that if the camera is in a static position, it's just a picture. A set of coloured pixels. If we move the camera, we calculate the time, the gyroscope's values, but most importantly, we can have a comparison of two images taken by the same objects. This way: -we can detect the edges -from the edges, we can detect which is closer than the others

Today's phone camera's are accurate enough to create ~60 crystal clear images per second, and it should be enough resource to accurately create high res models from just moving the camera according to some instructions (that's why I'm surprised why it isn't existing in just a phone app). Here comes the problem. I think the idea is worth for the try, but I'm just a JavaScript developer. The browser can have access to the camera, with TensorFlow I can use machine learning to detect edges, but if I want to be honest, I have no idea where to start, and how to continue step by step. Can you please provide me some guidelines how it would be ideal to create the idea?

|improve this question
$\endgroup$
0
$\begingroup$

Image recognition is based on two layers. The physical detection of objects is done with background subtraction for motion detection, edge detection of extracting the contours and shape similarity search for identifying circles and boxes. On top of the physical layer, a semantic engine takes the input and creates a probabilistic description. It has to track objects in the scene and matches contours against a database. Machine learning is used to support the process. It can be used for increasing the accuracy in the physical layer, but it can also be utilized for improving the semantic layer. For example, it make sense to learn which object size is equal to which distance to the camera.

A depth map is a 3d scene recognition. It can be generated by physical inputs from two cameras or it can be created for a 2D scene as a virtual depth map. In this case, the semantic layer has to determine how far the pixels are ahead from the camera.

Before the software itself can be created, a prototype allows the designer to test the functionality. Common languages like Javascript, Visual Basic, AutoIt, Python or Matlab are a good choice in doing so. All of them are providing access to a webcam and have build-in libraries for manipulating images. In most cases, the created sourcecode won't life very long. That means, the Javascript macro is created only for the reason to rewrite it in the next iteration.

semantic scene parsing

  • Joo, Seong-Wook, and Rama Chellappa. "Recognition of multi-object events using attribute grammars." Image Processing, 2006 IEEE International Conference on. IEEE, 2006.

  • Lin, Liang, et al. "A stochastic graph grammar for compositional object representation and recognition." Pattern Recognition 42.7 (2009): 1297-1307.

3d layout detection

  • Huang, Siyuan, et al. "Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image." European Conference on Computer Vision. Springer, Cham, 2018.

  • Yu, Chengcheng, Xiaobai Liu, and Song-Chun Zhu. "Single-image 3D scene parsing using geometric commonsense." Proceedings of the 26th International Joint Conference on Artificial Intelligence. AAAI Press, 2017.

|improve this answer
$\endgroup$
  • $\begingroup$ With what technologies and languages, frameworks (Javascript preferable) can this be done? $\endgroup$ – Gergő Horváth Dec 10 '18 at 14:27
  • $\begingroup$ What do you mean that the source code won't live long? The code written will always become "old" by short time? So newer technologies can be used? $\endgroup$ – Gergő Horváth Dec 10 '18 at 21:44
  • $\begingroup$ @GergőHorváth Yes, Javascript and Python code will become old after short time. The code fragment is only created to post it at Stackoverflow, that means the lifespan is around 2 days. This is called sourceless programming, because the audience is not the harddrive but other developers who see the update in their timeline, read it and forget it. $\endgroup$ – Manuel Rodriguez Dec 10 '18 at 22:19
  • $\begingroup$ How can I get to know, and understand more about the semantic engine which keeps track of, and processes the data? Are there any examples, learning resources, anything? $\endgroup$ – Gergő Horváth Dec 11 '18 at 15:26
2
$\begingroup$

I think that the tasks you are referring to are the Simultaneous localization and mapping (SLAM) and, in particular, the Structure from Motion (SfM).

These methods are usually based on geometrical constraints and do not employ neural networks, but there exist some recent methods that make use of CNNs (such as this one).

Structure from Motion algorithms are a fundamental component of the Google ARCore (former Project Tango) and the Apple ARKit. Unfortunately, these kits usually provide an interface to put elements in a specific position, but they don't provide the access to the 3d reconstruction of the scene.

If you'd like to play with SfM algorithms, I suggest you to start from this repository, but you can find a lot of other valuable tutorials online.

|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.