0
$\begingroup$

I am trying to make a NN(probably with dense layers) to map a specific input to a specific output (or basically sequence2sequence). I want the model to learn the relation between the sequences and predict the output of any other input I give it.

I have 2 files - one with the inputs and another with all the corresponding outputs and would probably use a bunch of Dense Layers with word embeddings to vectorize it into higher dimensions. However, I cannot find any good resources out there for that.

Does anyone know how to accomplish such an NN? Which architectures are best for pattern matching? examples, links, and other resources would be very welcome. I was considering using RNN's but found them not very good in the pattern matching tasks so had ditched them. I would still consider them if someone can provide a plausible explanation...

Improve this question
$\endgroup$
1
  • $\begingroup$ Can you provide a sample of data? The only clue you provided is word embeddings so I assume you have text. Why do you speak of patterns? Are you looking for specific phrases or sequences of words? If so are you interested in semantics and not just syntax? $\endgroup$
    – Brian O'Donnell
    Jan 8, 2021 at 21:18

1 Answer 1

Reset to default
0
$\begingroup$

I think you meant pattern recognition instead of pattern matching in you question. Because pattern matching has nothing to do with NNs as far as I know. So RNNs is the easiest architecture you could use for this task. Have a look at this post. It's long, but it's very well written and explains why RNNs work well with sequential data.

In brief, RNNs accumulate information about what was fed to them previously and store it in the hidden state vector. That information is a summary of a sequence which helps it to predict the output sequence. RNNs can be applied to variable length input and catch dependencies among different parts of the input (as opposed to dense layers that can be applied to a sequence only point-wise). Here's how exactly you can apply reccurent layers (2, 3). There are more advanced architectures like LSTMs. Also you can try transformers (4) which use attention mechanism (5).

Improve this answer
$\endgroup$
2
  • $\begingroup$ solotsky Can you give an explanation on why you find RNN's to be good for finding abstract patterns? Also, My input sequence is just a word of about 40 characters so I doubt it would be very useful... $\endgroup$
    – neel g
    Sep 12, 2020 at 15:02
  • $\begingroup$ Michael: NNs are used for pattern matching. $\endgroup$
    – Brian O'Donnell
    Jan 8, 2021 at 21:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .