3
$\begingroup$

I'd like to design a network that gets two images (an image under construction, and an ideal image), and has to come up with an action vector for a simple motor command which would augment the image under construction to resemble the ideal image more. So basically, it translates image differences into motor commands to make them more similar?

I'm dealing with a 3D virtual environment, so the images are snapshots of objects and motor commands are simple alterations to the 3D shape.

Probably the network needs two pre-trained CNNs with the same weights that extract image features, then output and concatenate those into a dense layer (or two), which converges into action-space. Training should probably happen via reinforcement learning

Additionally, in the end it needs recurrence, since there are multiple motor actions it needs to do in a row to get closer to the intended result.

Would there be any serious difficulties with this? Or are there any approaches to achieve the intended result? or any similar examples?

Thanks in advance

Improve this question
$\endgroup$
3
  • $\begingroup$ Is there an obvious/detectable endpoint for a sequence of actions (e.g getting within some limit of ideal), or an enforced one (such as max number of steps) - or both, or is this issue open in your design? $\endgroup$
    – Neil Slater
    Commented Nov 11, 2019 at 10:33
  • $\begingroup$ This is still an open issue in my design. Probably starting more basic, with an enforced one of a few steps to see how it performs $\endgroup$
    – SumakuTension
    Commented Nov 11, 2019 at 10:38
  • $\begingroup$ Gotta say, Great idea. $\endgroup$
    – user9947
    Commented Nov 11, 2019 at 11:46

0

Reset to default

You must log in to answer this question.