854
5 mins Sometimes an optimization loop may create agents that are themselves optimizers.
610
3 mins It’s difficult to create a reward function that precisely reflects what we want.
299
2 mins Intelligent systems may pursue a goal in unexpected ways, potentially leading to undesired outcomes.
201
1 min Almost any level of intelligence can in principle pursue almost any goal.
349
2 mins Humans or other entities will often adopt similar intermediate strategies to help them achieve their ultimate goals.