To create a classifier for a fixed corpus of texts is straightforward. Take all the documents, form the tfidf matrix and from that matrix take a subset that is tagged accordingly. The classifier built against that subset can then be used to tag the remainder of the corpus represented in the tfidf matrix.
However, if the corpus is constantly increasing in size and perhaps vocabulary, eg streaming news articles, this approach does not seem valid. There is no longer a fixed tfidf matrix, since it will change as new articles arrive. Even if the change was small enough when a single new document arrived the cost of recompiling the matrix would render the method impractical and over time inaccurate as the changes piled up.
So what is the best way of building a text classifier that can adapt to newly arriving articles? A semi supervised approach would seem possible but I haven’t found suitable references.