Questions tagged [tf-idf]

For questions related to TF-IDF(Term Frequency — Inverse Document Frequency) a technique to quantify a word in documents

Filter by
Sorted by
Tagged with
3
votes
2answers
76 views

Why do we commonly use the $\log$ to squash frequencies?

Term frequency and inverse document frequency are well-known terms in information retrieval. I am presenting the definitions for both from p:12,13 of Vector Semantics and Embeddings On term frequency ...
0
votes
0answers
32 views

Why does KNN algorithm perform better on Word2Vec than on TF-IDF vector representation?

I am doing a project on multi-class text classification and could do with some advice. I have a dataset of reviews that are classified into 7 product categories. Firstly, I create a term-document ...
0
votes
0answers
13 views

What is the best NLP model for handeling a rapid extension of the production data for a question answeing task

I'm lately writing a chat-bot to answer questions about children's fantasy books I save to my database . When a user opens a book it loads the chat-bot for the given book . I don't need it to work on ...
7
votes
2answers
151 views

Why are documents kept separated when training a text classifier?

Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF ...