Tips and tricks when training a very large language model?

Question

Have never trained a (very) large language model, so I am wondering if the process is the same as training a (regular) language model, i.e. you prepare the data, set up the architecture, hyperparameters, loss function to minimize perplexity and predicting the next word, and then do gradient descent over the giant dataset. Or if there are any special gotchas or tricks you must do when training it. I know there's at least one involving the training dynamics:

1. training dynamics: most LLMs stop seeing performance improvement even before a single epoch is finished.

I am wondering if there are any others

Answer 1

So one thing that might occur:

Note that we typically only train PMs for a single epoch, so the learning curves themselves (Figure 7 left) indicate how performance scales with dataset size (we used a fixed learning rate).

From: https://arxiv.org/pdf/2204.05862.pdf

Additionally:

Supervised fine-tuning (SFT). We fine-tune GPT-3 on our labeler demonstrations using supervised learning. We trained for 16 epochs, using a cosine learning rate decay, and residual dropout of 0.2. We do our final SFT model selection based on the RM score on the validation set. Similarly to Wu et al. (2021), we find that our SFT models overfit on validation loss after 1 epoch; however, we find that training for more epochs helps both the RM score and human preference ratings, despite this overfitting. https://arxiv.org/pdf/2203.02155.pdf

Here PM refers to the preference model used to evaluate generated outputs

In particular, the final line

we find that our SFT models overfit on validation loss after 1 epoch; however, we find that training for more epochs helps both the RM score and human preference ratings, despite this overfitting.

indicates some sort of overtraining/overparameterized phenomenon

Answer 2

Yes, when we work with LLM, first what we want to do is to calculate how much computational power we may need.

1. Find energy needed that is 100M between 1,000 to 10,000, then just multiply it by 10 all numbers to get an idea for 1 billion 10 etc.
2. Find how much memory we need that is one 16-bit float per parameter. In other word, 1 billion × 2 bytes = how much GPU memory we need.
3. (After we did all of trivial calculations) Explore dataset and identify bias as we will have to explain it.
4. Identify what we are looking for. If we want to make our LLM less biased we use GELU.
5. Find approximate learning rate for LLM in a unique way: We can use central limit theorem to figure out minimum set size for the dataset in order to insure that our results have 95% confidence interval and 5% error rate.
   In other words, whatever results we produce, we have 95% confidence that we reproduce identical results. For dataset we select random data, but in non trivial way. We need to divide our parameters in percentage as the model has different categories. Then we choose some material from those topics (randomly), and it is probably non trivial task.
6. Run grid search on the dataset, get parameters and rerun on bigger set. (Shoot out to all researchers, who forgot about.)