Questions tagged [tf-idf]

For questions related to TF-IDF(Term Frequency — Inverse Document Frequency) a technique to quantify a word in documents

Filter by
Sorted by
Tagged with
0 votes
0 answers

How to get match probability of the input text to the historical data?

I have 4 bln rows of data and unique 7500 products. Product ID has the commercial name of the product, which means that Product X can have a different set of descriptions. For example, ...
0 votes
1 answer

Distinguishing text with opposite meanings in SVM (False Information Detection)

I am currently working on a Binary Text Classification Model (False Information Detection) using Support Vector Machine and used TF-IDF as text vectorizer in Python. I have already tried training the ...
0 votes
1 answer

Which data representation of text as input for NLP Deep Learning models?

I have been given a data set with 30.000 text documents (each text file is rather small with respect to its length and consists in most cases of around 20 sentences), which are labelled with 0 or 1. ...
3 votes
2 answers

Why do we commonly use the $\log$ to squash frequencies?

Term frequency and inverse document frequency are well-known terms in information retrieval. I am presenting the definitions for both from p:12,13 of Vector Semantics and Embeddings On term frequency ...
  • 3,251
8 votes
1 answer

Why are documents kept separated when training a text classifier?

Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF ...
  • 246