I am doing a project on multi-class text classification and could do with some advice.
I have a dataset of reviews that are classified into 7 product categories.
Firstly, I create a term-document matrix using TF-IDF (tfidfvectorizer from sklearn). This generates a matrix of n x m where n is the number of reviews in my dataset and m is the number of features.
Then after splitting the term-document matrix into 80:20 train: test, I pass it through the K-Nearest Neighbours (KNN) algorithm and achieve an accuracy of 53%.
In another experiment, I used the Google News Word2Vec pre-trained embedding (300 dimensional) and averaged all the word vectors for each review. So, each review consists of x words and each of the words has a 300-dimensional vector. Each of the vectors is averaged to produce one 300-dimensional vector per review.
Then I pass this matrix through KNN. I get an accuracy of 72%.
As for other classifiers that I tested on the same dataset, all of them performed better on the TF-IDF method of vectorization. However, KNN performed better on word2vec.
Can anyone help me understand why there is a jump in accuracy for KNN in using the word2vec method as compared to when using the tfidf method?