1
$\begingroup$

I have sentence data in a language that is not widely in use and as such popular LLMs do not support the language. I want to train some language model such that given some question, it is able to respond back in the same language, just as in ChatGPT just with a different language.

In such a case, what language model is publically available and is sufficiently powerful? Or would it be possible to use popular LLMs (such as ChatGPT) to achieve such a goal?

$\endgroup$

1 Answer 1

0
$\begingroup$

One can use the CulturaX dataset (paper) to train an LLM in another language. It contains 27 TB of texts divided into 167 languages.

Otherwise, one can use Okapi (paper):

Okapi Resources: We provide resources to perform instruction tuning with RLHF for 26 languages, including ChatGPT prompts, multilingual instruction datasets and multilingual response ranking data.

Okapi Models: We provide RLHF-based instruction-tuned LLMs for 26 languages on Okapi dataset. Our models include both BLOOM-based and LLaMa-based versions. We also provide scripts to interact with our models and fine-tune LLMs with our resources.

Multilingual Evaluation Benchmark Datasets: We provide three benchmark datasets for evaluating Multilingual Large Language Models (LLMs) for 26 languages. You can access the full datasets and evaluation scripts: here.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .