1
$\begingroup$

If the AI alignment problem is one of the most pressing issues of our time, could AI itself augment our (i.e., human) quest to solve the alignment problem? Or would AI itself actually be counter-productive for such a meta-level goal?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

Yes, this is one the most popular approaches today. For instance a narrow AI model is built to capture the preference of the user and then this model is used to dispense reward to a reinforcement learning agent.

The narrow AI model is often called Reward Model or Preference Model.

The recent RLHF training technique used for chatGPT uses this approach.

This can be automated further: https://arxiv.org/abs/2212.08073

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .