1
$\begingroup$

As a first step in many NLP courses, we learn about text preprocessing. The steps include lemmatization, removal of rare words, correcting typos etc. But I am not so sure about the actual effectiveness of doing such a step; in particular, if we are learning a neural network for a downstream task, it seems like modern state of the art (BERT, GPT-2) just take essentially raw input.

For instance, this ACL paper seems to show that the result of text preprocessing is mixed, to say the least.

So is text preprocessing really all that necessary for NLP? In particular, I want to contrast/compare against vision and tabular data, where I have empirically found that standardization usually actually does help. Feel free to share your personal experiences/what use cases where text preprocessing helps!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

It all depends on the quality of data. Due to old rule "Garbage in, garbage out" link , if you have bad quality data(data redundancy, unstructured data, too much memory, etc) your results won't be spectacular.

In other cases, everybody could be a Data Scientist, because its only task was "put raw text into classifier". Also, you should remember that BERT or GPT-2 it's deep learning algorithms so they not need too much processing. Using preprocessing in machine learning is more that needed(prediction of sentiment for example).

Shortly, preprocessing is optional, but highly advisable.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Depends, I agree in some tasks it can help. But I don't think in general we should think about raw data as "garbage' necessarily. $\endgroup$
    – information_interchange
    Apr 12, 2020 at 1:00
0
$\begingroup$

It depends on the dataset we have and algorithm we use, usually text preprocessing can help your model perform better. But, some preprocessing method can have no significant impact on accuracy. We need to choose which preprocessing method that can help us make better quality dataset to give to the model.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .