Currently I am finetuning transformers T5 model for translation task. As part of the dataset, I am given sentences in Japanese, their translation to English, and for every English sentence I am also given a few (English) words which need to be in the translated sentence.

Is there a way to modify the Seq2SeqTrainer and the loss being used in order to penalize the model (while training) for not including the given words in the translated sentence?


1 Answer 1


Actually I encountered this issue myself.

My solution was to wrap the model in my own class. I then overrode the forward method. ( I was using pytorch lightning)

class WrapModel(pl.LightningModule):
    self.model = T5ForConditionalGeneration...

def forward(self, **inputs):
    outputs = self.model(**inputs)
    loss = new_loss_logic(outputs.specific_hidden_state)
    outputs.loss = loss

I am relatively new to the field, but I think this should work. If you wish to stay within HF, you can always make the wrapping class inherit the T5Model, then the call to the underlying forward is self(**inputs) or can be down with super(), but the rest remains the same


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .