It seems the keywords you're looking for are [knowledge distillation](https://en.wikipedia.org/wiki/Knowledge_distillation), [transfer learning](https://en.wikipedia.org/wiki/Transfer_learning), [domain adaptation](https://en.wikipedia.org/wiki/Domain_adaptation), and [fine tuning of LLMs](https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)) which are all based on the fundamental semantic vector embedding and similarity learning as discussed in today's another related [post](https://ai.stackexchange.com/questions/46458/what-is-the-existing-literature-on-why-vector-embeddings-work) which you may refer. Knowledge base acquisition and learning are inherently *semantic* in nature, thus similarity learning of the knowledge manifold in the embedding space is foundational in this respect in addition to the traditional syntactic concept learning and proof-tree natural deduction.

>Knowledge distillation transfers knowledge from a large model to a smaller model without loss of validity. As smaller models are less expensive to evaluate, they can be deployed on less powerful hardware (such as a mobile device)... The idea of using the output of one neural network to train another neural network was studied as the teacher-student network configuration. Hope below quoted sections are relevant and help your research.

>Domain adaptation is a field associated with machine learning and transfer learning. This scenario arises when we aim at learning a model from a source data distribution and applying that model on a different (but related) target data distribution. For instance, one of the tasks of the common spam filtering problem consists in adapting a model from one user (the source distribution) to a new user who receives significantly different emails (the target distribution).

>In deep learning, fine-tuning is an approach to transfer learning in which the parameters of a pre-trained model are trained on new data.[1] Fine-tuning can be done on the entire neural network, or on only a subset of its layers, in which case the layers that are not being fine-tuned are "frozen" (or, not changed during the backpropagation step)... Fine-tuning can be combined with a reinforcement learning from human feedback-based objective to produce language models like ChatGPT (a fine-tuned version of GPT-3) and Sparrow.