I want to finetune GPT2 to extract relevant data from a given text. So for (a trivial) example, given the text "the car was manufactured in X, can reach Y km/h, and has Z horse powers", my desired output would be manufacturer: X
, max speed:Y
, horsepowers:Z
.
I don't have a lot of labeled data, so I thought it would be reasonable to take every training sample and add to it a prefix than contains an actual example. That is - Instead of providing the model with the text
INPUT: a well-known brand, the new
X
, can reachY
km/h, and hasZ
horsepowerOUTPUT:
and expect the model to understand how to fill in the details, I would provide a longer prompt that contains an actual example like
INPUT: the new
BMW
can reach up to200
kmh. Even though the previous model disappointed the users, the brand-new one rocks athousand
engine horsepowerOUTPUT: Manufacturer:
BMW
, Max Speed:200
, Horse Powers:thousand
INPUT: though is mostly considered an outdated version of the cls200, the new
Mercedes
has the capabilities of reaching up to 100kmh in turns and300
kmh overallOUTPUT:
Is this considered a common way of engineering the prompt? Do notice that the first provided example is the same for every training sample.