I am currently using Huggingface transformers to generate text using GPT-2. The problem is, that it only generates 1024 tokens, what seems to be a hard limit in the code of the script, which enforces a maximum length of the model generation length model.config.max_position_embeddings.
As far as I understood it, one should be able to generate infinite text, with the drawback that only the last 1024 tokens are considered as context. I tried feeding a paragraph from the output into the text run, but the input prompt seems to be limited to much fewer tokens than 1024 (I am not sure how many).
So it may only be the script that's limited, but I am not sure how one would use the model to continue generation with the state the models has after generating the last token.
