2
$\begingroup$

Is there a type of neural network that can be fed patterns to train itself on to complete new patterns that it has not seen before?

What I'm trying to do is train a neural network to transform an image into another image. The image may be slightly different each time (denoted with different lines in the shapes) but a human would get the idea of how the new images should look. I'd like to make a network that can learn how to learn what comes next and then predict the rest of the sequence from the first part of a new sequence.

Taking the picture below as an example. The neural network would be fed the patterns in grey and learn how to predict the next ones in the sequence. Then the user would put the blue shapes into the network and hope to get the green ones out.

Is there a neural network that could perform this type of function of completing a pattern based on only a small number of examples to start the pattern based on the other patterns it has seen?

enter image description here

EDIT: Corrected image and added more context

Improve this question
$\endgroup$
4
  • $\begingroup$ If the sequence of symbols or geometrical objects really has a pattern (e.g. after circles we always have squares, or something like that), then the neural network may be able to capture it, but if the sequence of objects is random, how can you predict anything without an error less than 50%? Do you know if the sequence really has patterns that can be recognized? $\endgroup$
    – nbro
    Jul 25, 2020 at 13:51
  • $\begingroup$ Moreover, neural networks typically require a lot of data to learn well, although there are already some approaches that attempt to overcome this issue. Have a look at terms like "one-shot learning", "few-shot learning" or "zero-shot learning". $\endgroup$
    – nbro
    Jul 25, 2020 at 13:53
  • $\begingroup$ Thanks for your response. The shape would always be similar (if there is a round shape there will always be a rectangular shape). I've updated the image as I realised I hadn't drawn it completely correctly. $\endgroup$
    – Eind997
    Jul 25, 2020 at 15:50
  • $\begingroup$ What I'm trying to do is train a neural network to transform an image to another image. The image may be slightly different each time (denoted with different lines in the shapes) but a human would get the idea of how the new images should look. I'd like to make a network that can learn how to learn what comes next and then predict the rest of the sequence from the first part of a new sequence. $\endgroup$
    – Eind997
    Jul 25, 2020 at 16:13

1 Answer 1

Reset to default
1
$\begingroup$

The idea is simple, but it requires some time to develop.

Assumption: I am assuming in your problem the final model will have seen all possible shapes.

What your algorithm needs is a convolutional NN to understand each shape by extracting features, but you just need to be very careful with pooling.

Then what you need is a recurrent NN. In the example you showed (the image of shapes) we have bigrams (sequence of 2) which means we have the first shape as an input and the second shape is a target. In this case normal RNN should work.

But if you have sequence of many shapes, for example 10 shapes and let's say the final shape, 10th, is the target; And also if the sequence is in the way that 10th shape could be more depend on initial shapes (e.g. 1st or 2nd shape) then what you also need to consider beside RNN is long-short-term-memory (LSTM).

I cannot think of a simpler solution than this.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .