I've been working on learning about NLP via a beginners competition on Kaggle.
I first trained a model with an embedding layer and then a simple linear layer. I actually got way better than a flip of the coin with this model, so I decided to try to step it up with an LSTM.
What happened was that training loss decreased and then palteaued while validation loss never decreased at all.
In the case of overfitting, I would expect validation loss to decrease for a while but then either remain steady or perhaps even increase as the model starts to overfit.
I can't find any reason for the strange loss curves I'm seeing:
What could cause such a phenomenon?
I would be happy to share my network architecture and training code if there isn't a straightforward answer (I know there usually isn't).