I am doing a project on multi-class text classification and could do with some advice.

I have a dataset of reviews that are classified into 7 product categories.

Firstly, I create a term-document matrix using TF-IDF (tfidfvectorizer from sklearn). This generates a matrix of n x m where n is the number of reviews in my dataset and m is the number of features.

Then after splitting the term-document matrix into 80:20 train: test, I pass it through the K-Nearest Neighbours (KNN) algorithm and achieve an accuracy of 53%.

In another experiment, I used the Google News Word2Vec pre-trained embedding (300 dimensional) and averaged all the word vectors for each review. So, each review consists of x words and each of the words has a 300-dimensional vector. Each of the vectors is averaged to produce one 300-dimensional vector per review.

Then I pass this matrix through KNN. I get an accuracy of 72%.

As for other classifiers that I tested on the same dataset, all of them performed better on the TF-IDF method of vectorization. However, KNN performed better on word2vec.

Can anyone help me understand why there is a jump in accuracy for KNN in using the word2vec method as compared to when using the tfidf method?

  • 1
    $\begingroup$ Maybe KNN performs worse on sparse data that the TF-IDF representation provides. Why are you using a KNN instead of lets say a Linear SVM when using TFIDF vectors? $\endgroup$
    – Isbister
    May 7 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.