I am reading the paper "Large Language Models Can Self-Improve" https://arxiv.org/abs/2210.11610 in which the authors consider that LLM can generate Chain-of-Thoughts sequences and even novel questions and their respective CoT responses and then selects the best sequences and fine-tunes themselves. Such self-reflection improves LLM. Such self-reflection also mimics how the human beings learns material by rethinking it. The usual human learning process is guided by the preset questions - courses and textbooks have them. But more mature learners usually can discover knowledge and understanding gaps in their minds themselves and form the most relevant questions and open problems, whose solution can improve the understanding and hence the assimilation of material in the actionable form.

Such process is known in psychology as apperception https://en.wikipedia.org/wiki/Apperception.

My personal experience shows that apperception can be greatly enhanced (sometimes it even required) the reading of additional, carefully selected material. Sometimes it gives additional information. Sometimes it just reformulated the existing information and hints to additional links among the existing pieces of knowledge.

Apparently - if LLM is working in self-improving mode then it could be highly benefitial if LLM could actively seek specific additional information and fine-tune itself on it.

My question is - how to do such active seeking of specific additional information and is there any research efforts that have already done something like that?

Additional info: actually there is a least one paper https://parl.ai/projects/seeker/ (https://arxiv.org/abs/2203.13224) that uses this self-improvement scheme.



You must log in to answer this question.

Browse other questions tagged .