3
$\begingroup$

Is it possible to train a neural network to learn something via video footage (which is essentially a sequence of images)?

In other words, if I have a video teaching me how to draw an animal from scratch, can I then use this video to teach the computer to draw the animal in the same way?

There are many machine learning and image processing techniques (such as RNNs) that can be applied to sequences of images or videos. So, I guess the difficult part becomes mapping the activity to an action like moving a pen or something.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
1
$\begingroup$

Short answer: No.

Long answer: A neural network is a function that maps input data (e.g. a picture) to output data (e.g. probability that the picture contains a dog). What you propose does not seem like the sort of task this tool is suited for, though I'm not willing to claim that it can't be done.

If you are interested in computer-generated artwork, I encourage you to learn about Generative Adversarial Networks and style transfer.

Improve this answer
$\endgroup$
5
  • $\begingroup$ You have only described a neural network as a classification tool.. what about reinforcement learning and regression-based nets? $\endgroup$
    – Mark Markrowave Charlton
    Jun 13, 2017 at 5:19
  • $\begingroup$ The use case doesn't change the fundamental concept of the net - it's still a function mapping input variables to output variables. $\endgroup$
    – Jessica Alan
    Jun 13, 2017 at 15:52
  • $\begingroup$ Ok, am i able to map 3d coordinates from a 2d input? lets say i make a 3d object and screenshot it, can i map one to the other? $\endgroup$
    – Mark Markrowave Charlton
    Jun 13, 2017 at 22:27
  • $\begingroup$ So a network that maps a 2D representation of an object to its 3D vertices? That sounds feasible. $\endgroup$
    – Jessica Alan
    Jun 14, 2017 at 15:07
  • $\begingroup$ ok cool im on to it! $\endgroup$
    – Mark Markrowave Charlton
    Jun 14, 2017 at 23:06
1
$\begingroup$

Just like the other answer - it can't really be done like that. However, you might want to take a look at Google Deepdream, which actually enhances images to look more like what they detected. E.g., if a cat is detected, it will make the picture more cat-like.

Improve this answer
$\endgroup$
0
$\begingroup$

About the screenshot you mentioned in comments: don't forget that NN build the mappings from learning. With only one screenshot, you'll only be able to construct a NN that outputs the correct 3D for THAT screenshot. With many screenshots, you may come closer to what you want (AFAI understand your question).
But remember that without 2 eyes, you have problems with 3D objects. Doesn't mean it's impossible.
May I suggest you to have a look at convolutional neural networks also https://en.wikipedia.org/wiki/Convolutional_neural_network

Improve this answer
$\endgroup$
7
  • $\begingroup$ So am i correct in saying that if i can get enough visual information of an object (ie enough screenshots of different angles) and encode it into a higher representation (like identifying that theres a face) i can map that encoded information to a mesh of 3d coordinates? $\endgroup$
    – Mark Markrowave Charlton
    Jun 13, 2017 at 23:06
  • $\begingroup$ If you know the angles (more precisely the position of the...say camera), you don't even have to use a NN. ;) $\endgroup$
    – jcm69
    Jun 13, 2017 at 23:18
  • $\begingroup$ if i have a 3d mesh with geometry (lets say 4 square faces, each face containing 4 vertices) is there a way to represent the polygons edges? becuase if i have 4 vertices i still dont know where the edges are $\endgroup$
    – Mark Markrowave Charlton
    Jun 13, 2017 at 23:21
  • $\begingroup$ Ok, say you have 2 views from 2 cameras (with known position) of a single point (say a light), then you can reconstruct it's position in a 3D space. Easy. Based on that, you can do many things. That's been the case for years. But it's complicated and specialized. While NN have incredible results that classical approaches can't achieve in a quite simple way. Here, quite means that you've got to be familiar with NN. Often takes time, really ;) $\endgroup$
    – jcm69
    Jun 13, 2017 at 23:40
  • $\begingroup$ ok but how can i represent a 3d mesh in terms of its faces instead of its vertices? $\endgroup$
    – Mark Markrowave Charlton
    Jun 13, 2017 at 23:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .