Questions tagged [tf-idf]

For questions related to TF-IDF(Term Frequency — Inverse Document Frequency) a technique to quantify a word in documents

Filter by
Sorted by
Tagged with
7
votes
2answers
156 views

Why are documents kept separated when training a text classifier?

Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF ...
3
votes
2answers
79 views

Why do we commonly use the $\log$ to squash frequencies?

Term frequency and inverse document frequency are well-known terms in information retrieval. I am presenting the definitions for both from p:12,13 of Vector Semantics and Embeddings On term frequency ...
0
votes
0answers
68 views

Why does KNN algorithm perform better on Word2Vec than on TF-IDF vector representation?

I am doing a project on multi-class text classification and could do with some advice. I have a dataset of reviews that are classified into 7 product categories. Firstly, I create a term-document ...