I've been using Tensorflow and just started learning PyTorch. I was following the tutorial: https://pytorch.org/tutorials/beginner/nlp/word_embeddings_tutorial.html#sphx-glr-beginner-nlp-word-embeddings-tutorial-py
Where we try to create an n-gram language model. However, there's something I don't understand.
class NGramLanguageModeler(nn.Module): def __init__(self, vocab_size, embedding_dim, context_size): super(NGramLanguageModeler, self).__init__() self.embeddings = nn.Embedding(vocab_size, embedding_dim) self.linear1 = nn.Linear(context_size * embedding_dim, 128) self.linear2 = nn.Linear(128, vocab_size)
self.linear1 = nn.Linear(context_size * embedding_dim, 128) why did we multiply embedding_dim with context_size? Isn't the embedding_dim input size? So why do we multiply it by the context size?