I am trying to use PyTorch's transformers as a part of a research project to do sentiment analysis of several types of review data (laptop and restaurant).

To do this, my team is taking a token-based approach and we are using models that can perform token analysis.

One problem we have encountered is that many of the models in PyTorch's transformers do not support token classification, but do support sequence classification. One such model we wanted to test is GPT-2.

In order to overcome this, we proposed using sequence classifiers on single tokens which should work in theory, but possibly at reduced accuracy.

This raises the following questions:

  • Is it possible to do token classification using a model such as GPT-2 using PyTorch's transformers?

  • How do sequence classifiers perform on single token sequences?



You must log in to answer this question.

Browse other questions tagged .