Questions tagged [n-gram]
For questions about the concept of an n-gram in the context of natural language processing and other AI sub-fields.
4 questions
1
vote
1
answer
95
views
Bag of Tricks: n-grams as additional features?
I've been playing with PyTorch's nn.EmbeddingBag for sentence classification for about a month. I've been doing some feature engineering, playing with different ...
- 143
1
vote
1
answer
102
views
What is the lowest possible loss for a language model?
Example: Suppose a character-level language model (three input letters to predict the next one), trained on a dataset which contains three instances of the sequence ...
- 179
0
votes
1
answer
50
views
If the unigram precision is (N-1)/N, then the bigram precision is :
Consider the following machine translation scenario. The reference translation has N words (do not consider sentence beginner ‘hat’ and sentence finisher ‘dot’). The machine output also has N words. ...
- 101
0
votes
1
answer
161
views
Why would adding all the possible embeddings be "worse" than using 1D-convolutions?
Suppose we are using word2vec and have embeddings of individual words $w_1, \dots, w_{10}$. Let's say we wanted to analyze $2$ grams or $3$ grams.
Why would adding all the possible embeddings, $\binom{...
- 51