Skip to main content

Questions tagged [tf-idf]

For questions related to TF-IDF(Term Frequency — Inverse Document Frequency) a technique to quantify a word in documents

Filter by
Sorted by
Tagged with
1 vote
0 answers
31 views

Is there a metric to compare BOW vs TFIDF results?

I am working on a document search task and have used Bag of Words (BOW) and TFIDF vectorization techniques. My observation after going through some sample searches are - Both of them seem to provide ...
Amit Pathak's user avatar
0 votes
1 answer
57 views

Distinguishing text with opposite meanings in SVM (False Information Detection)

I am currently working on a Binary Text Classification Model (False Information Detection) using Support Vector Machine and used TF-IDF as text vectorizer in Python. I have already tried training the ...
alexand88r's user avatar
0 votes
1 answer
617 views

Which data representation of text as input for NLP Deep Learning models?

I have been given a data set with 30.000 text documents (each text file is rather small with respect to its length and consists in most cases of around 20 sentences), which are labelled with 0 or 1. ...
MiFischer22's user avatar
3 votes
2 answers
295 views

Why do we commonly use the $\log$ to squash frequencies?

Term frequency and inverse document frequency are well-known terms in information retrieval. I am presenting the definitions for both from p:12,13 of Vector Semantics and Embeddings On term frequency ...
hanugm's user avatar
  • 3,890
8 votes
1 answer
194 views

Why are documents kept separated when training a text classifier?

Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF ...
freesoul's user avatar
  • 246