Questions tagged [reward-hacking]

For questions related to reward hacking, i.e. when an agent manipulates the reward system to get a lot of reward in unintended ways. The paper "Concrete Problems in AI Safety" by Dario Amodei et al. provides more details about this topic.

Filter by
Sorted by
Tagged with
4
votes
2answers
121 views

How can we prevent AGI from doing drugs?

I recently read some introductions to AI alignment, AIXI and decision theory things. As far as I understood, one of the main problems in AI alignment is how to define a utility function well, not ...