Debate Update: Obfuscated Arguments Problem
AI Safety Fundamentals: Alignment
The Importance of Deliberation in Machine Learning
In the original debate paper, it was shown that a debate like mechanism can provide a supervision signal for problems in peace base using a polynomial time judge. We extended this to any XP with the cross examination mechanism. This suggested that as long as there is some potentially exponentially large argument for the correct answer, where a human can understand each step, we can incentivize honestly reporting the correct answer. However, both of these results needed to assume that the debate is our computationally unbounded. If the honest debate is bounded, then we can essentially only trust conclusions that can be verified using the debate as computational resources. And when either the only human understandable arguments are inscrutable to the model
