1
$\begingroup$

I am trying to make a model that uses a Transformer to see the relationship between several data vectors, but the order of the data is not relevant in this case, so I am not using the Positional Encoding.

Since the performance of models using Transformers is quite improved with the use of this part, do you think that if I remove that part I am breaking the potential of Transformers or is it correct to do so?

Improve this question
$\endgroup$
1
  • $\begingroup$ Maybe you could make a little example of the inputs and how they should be given to the transformer, to make it clear why the order of the data should not be relevant. $\endgroup$
    – dexteritas
    Nov 15, 2021 at 15:55

1 Answer 1

Reset to default
2
$\begingroup$

Positional Encodings in Transformers exist to give the model some information about the position of the embedding. This makes sense in fields like NLP or Time Series Data, since the position(order) matters in this case.

However, since you say that order of the data is not relevant in your use case, positional encoding would not be necessary.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .