I am reading the paper "Training language models to follow instructions with human feedback"

It says:

"Our labelers provide demonstrations of the desired behavior on the input prompt distribution (see Section 3.2 for details on this distribution). We then fine-tune a pretrained GPT-3 model on this data using supervised learning."

The paper also says:

"On our test set, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having over 100x fewer parameters."

I am not able to understand how the aligned model is a fine tune of GPT-3 using supervised learning (and other steps associated with reinforcement learning) and at the same time the aligned model has fewer parameters than the original model.

Can someone give me a hint on the subject?

I am reading the paper "Training language models to follow instructions with human feedback"

It says:

Our labelers provide demonstrations of the desired behavior on the input prompt distribution (see Section 3.2 for details on this distribution). We then fine-tune a pretrained GPT-3 model on this data using supervised learning.

The paper also says:

On our test set, outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having over 100x fewer parameters.

I am not able to understand how the aligned model is a fine tune of GPT-3 using supervised learning (and other steps associated with reinforcement learning) and at the same time the aligned model has fewer parameters than the original model.

Can someone give me a hint on the subject?