I have created a data set with 30.000 text documents (each text file is rather small with respect to its length), which are labelled with 0 and 1. Using this data set, I want to train a deep learning model in order to be able to classify new text files.

For this purpose, I want to use a state of the art NLP model like BERT, XLNET, GPT2, GPT3, some model derived from BERT (e.g., RoBERTa, StructBERT, T5, ELECTRA, DeBERTa), or any other more recent model that outperforms benchmarks.

In this context, which state of the art NLP model for classification would you currently suggest?

Thereby, three points are especially important for me: i) Is for the model a straightforward implementation in some library available? ii) How long does it take to fine tune the model with my data set (assuming the availability of a gaming cpu and gpu)? iii) How well does the model perform in terms of accuracy?

Many thanks.

  • $\begingroup$ Could you please put your specific question in the title "Choice of appropriate state of the art architecture for NLP" is not a question and it's also not specific. Thanks. $\endgroup$
    – nbro
    Feb 23 at 21:56


You must log in to answer this question.

Browse other questions tagged .