I have recently come across transformers, I am new to Deep Learning. I have seen a paper using CNN and BiLSTM on top of a transformer, the paper uses a transformer(XLM-R) for sentiment analysis in code-mixed domain. But many of the blogs only use a normal feed formal network on top of the transformer.
I am trying to use transformers for sentiment analysis, short text classification.
Is it overkill to use models like CNN and BiLSTM on top of the transformer considering the size of the data it is trained on and its complexity?