Specification Gaming: The Flip Side of AI Ingenuity
AI Safety Fundamentals: Alignment
The Importance of Correcting Mistakes in Task Specification Design
In the example above, the robot's task was misspecified because of incorrect assumptions about simulator physics. Analogously, a real-world traffic optimization task might be misspecified by incorrectly assuming that the traffic routing infrastructure does not have software bugs or security vulnerabilities. As tasks grow too complex to consider every detail, researchers are more likely to introduce incorrect assumptions during specification design. This poses the question, is it possible to design agent architectures that correct for such false assumptions instead of gaming them? And last, how do we avoid reward tampering?
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
