0
$\begingroup$

I have tried training T5-small, T5-base and T5-Large on around 15K rows of data where input data was something like but I did not get desired results

Nutrition Facts,
100g per,
Energy 646.95Kcal Carbohydrates 19.31g,
 Protein 21.94g 53.55g Total Fat 6.64g Saturated Fat 14.97g Dietary Fiber,
<1.Omg Cholesterol Sodium 0.29g Sugars 3.39g,
Lightly Salted and to Perfection,
Ingredients: Peanuts, Almonds,,
Cashews, Pistachios, Vegetable Oil, Salt,
aa, ,
74G,
Pistachio, 61129110611336177
WE ARE NUTS ABOUT QUALITY,
Baked,
Nuts Salted,
Mixed
WE ARE NUTS ABOUT,
Community 364, 13 Street Plot No. 36,
Al Area 1, 24149,UAE 4971 4 3355777,
License Number: 224614 VAT No: 100058529700003,
CERTIFIED COMPANY,
ALLERGEN WARNING: in a facility that also processes nuts, sesame and mustard,
Store in a cool dry place away from heat moisture,
Instruction Once store in airtight container and consume before expiry date,
Pro: 14/12/23,
Exp:13/12/24,
Net 40gms

Output data will be in JSON format of the above details. What language models can be trained for this purpose and minimum how many Parameters it should have?

$\endgroup$

1 Answer 1

1
$\begingroup$

I've tried Mistral-7B-Instruct-v0.2, and it worked out of the box for your example, as did Qwen1.5-1.5b. However, Qwen1.5-0.5b had issues, but I think with enough training, you can achieve decent results.

I believe the size/quality trade-off of starting from 1.5b and lowering it could deteriorate in corner cases, but the only way to find out is through experimentation with validation.

One note is that you can generate almost an infinite amount of data by removing the JSON structure. 15k seems sufficient if you have just a product JSONs, but for more diversity, you might want more data. The SFT stage in an LLM usually takes from 200k up to a million examples.

You can also constrain the output using a scheme by restricting the output tokens. See one example of such an approach at https://github.com/1rgs/jsonformer.

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .