So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample.
Next I tried to get it to generate limericks using these 100,000 limericks. It generated garbage output.
When I limited it to 10,000 characters, it then started giving reasonable limerick output.
How could this happen? I thought more data was always better.
The AI was a neural network with some LSTM layers, implemented in keras.