I have been reading some articles on AI safety and they almost always speak of AI Safety from the reinforcement learning (RL) perspective, i.e. where we have some artificially intelligent agent acting in an environment so to maximise some reward.

Is there a reason why the focus is very often from this RL POV?




1 Answer 1


I think what you are really asking is why don't we talk about alignment from the perspective of general deep learning?

In fact, we do talk about alignment for general AI system that's being used in production. For example, making large language model less toxic is an active area of research.

The reason you see a lot of references to RL in the context alignment is that one method that has been actively researched to make our AI systems aligned with our preference is through training them via RL. The reward function is, not surprisingly, good alignment with human preferences.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .