I’m thinking of fine-tuning a pre-trained language model for a Q&A task. More specifically, I’d like to fine-tune the model on a single chapter in a classic college textbook. Afterward, the reader of the chapter should be able to engage in a Q&A session with the model about the content of the chapter.
But how do I make sure that the model stays on topic and doesn’t go out of a tangent? I know it is possible when looking at what https://play.aidungeon.io/ has achieved, but I don’t know if it will require me to build a model from the ground for each chapter. Can anyone tell me if I’m out of my mind or if it’s feasible?
I've just learned that AI Dungeon's models has also struggled with a tendency to create graphic and sexual content despite not being prompted by players.