How many training example text classifier needs to be trained?

Question

I want to train text classifier (using https://www.uclassify.com) with 12 classes/categories. I will be training it to classify news/articles (I know that there are existing classifier but I want to train my own).

uclassify uses following algorithm (directly copied from their site):

The core is a multinominal Naive Bayesian classifier with a couple of steps that improves the classification further (hybrid complementary NB, class normalization and special smoothing). The result of classifications are probabilities [0-1] of a document belonging to each class. This is very useful if you want to set a threshold for classifications. E.g. all classifications over 90% is considered spam. Using this model also makes it very scalable in terms of CPU time for classification/training.

I was wondering how many examples I will need to train such classifier? It is possible to estimate the number? Let's assume that one article will "fit" 2 categories by average.

Answer 1

As a general rule of thumb I typically use 10*(# of features) for shallow machine learning models such as Naive Bayes with only 2 classes.

So it all depends on the number of features you will be using. However, the more output classes the more data you will need for proper discrimination. The addition of more classes is not linear but I think you can get away with: 10*(# of features)*(# of output classes)