I'm trying to generate rhymes, so it would be very helpful to have a language model where I could input a final word, and have it output a sequence of words that ends with that word.

I could train my own model and reverse the direction of the mask, but I was wondering if there was any way I could use a pretrained model but apply a different mask to achieve this goal.

If this isn't possible, what's the best way to sample a forward-predicting model to achieve the highest probability sentence ending with a particular word?


  • $\begingroup$ What the particular final word depend on? Does that depend on previous sequences? Or previous words? $\endgroup$ – pedrum Jul 30 at 0:25
  • $\begingroup$ @pedrum It depends on the previous sentence. It's exactly like the usual transformer case except predicting from right to left instead of left to right $\endgroup$ – lightning Jul 31 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.