1
$\begingroup$

I want to know what things I should be learning before trying to fine-tune or for that matter working with a large language model.

In my case, I am trying to fine-tune bloom (https://huggingface.co/blog/bloom).

But considering my inadequate knowledge of this, I want to know what things I should cover first before taking on something so big.

Any suggestions are appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

There are considerable free and excellent resources out there (in the wild).

You can check The Stanford Natural Language Processing Group teaching page; you can easily follow their YouTube courses on NLP (natural language processing) and NLU (natural language understanding).

Moreover, you can read scientific papers on Transformers, the bread and butter of HuggingFace's models; a quick Google search should do the work.

Finally, you must know Pytorch since (if I recall correctly) HuggingFace's implementations use it.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .