In the paper Efficient Estimation of Word Representations in Vector Space, the authors say that "avoiding normalized models completely by using models that are not normalized during training" is a practical solution for reducing the complexity that comes from "H × V" for an NNLM with a projection layer, a hidden layer and an output layer where H is the hidden layer size and V is the vocabulary size.
I can't quite understand why it is the case and I'd love some hints because for me normalization won't change the dimensions (and I think it helps with convergence) so it shouldn't reduce complexity.
Thank you.
