Skip to main content
edited title
Link

can I add to a language model a prompt with two output examplesexample?

Source Link

can I add to a language model a prompt with two output examples?

I want to finetune GPT2 to extract relevant data from a given text. So for (a trivial) example, given the text "the car was manufactured in X, can reach Y km/h, and has Z horse powers", my desired output would be manufacturer: X, max speed:Y, horsepowers:Z.

I don't have a lot of labeled data, so I thought it would be reasonable to take every training sample and add to it a prefix than contains an actual example. That is - Instead of providing the model with the text

INPUT: a well-known brand, the new X, can reach Y km/h, and has Z horsepower

OUTPUT:

and expect the model to understand how to fill in the details, I would provide a longer prompt that contains an actual example like

INPUT: the new BMW can reach up to 200 kmh. Even though the previous model disappointed the users, the brand-new one rocks a thousand engine horsepower

OUTPUT: Manufacturer: BMW, Max Speed: 200, Horse Powers: thousand

INPUT: though is mostly considered an outdated version of the cls200, the new Mercedes has the capabilities of reaching up to 100kmh in turns and 300kmh overall

OUTPUT:

Is this considered a common way of engineering the prompt? Do notice that the first provided example is the same for every training sample.