Understanding AI Failures

AI systems can experience two types of failures: observable failures, which developers can detect through testing and red teaming, and unobservable failures, which are harder to identify and often missed during development. Observable failures can be addressed using standard machine learning techniques, as they can be discovered through typical evaluation processes. In contrast, unobservable failures can involve subtle biases or deceptive alignments that make them challenging to surface. This distinction highlights the limitations of current AI development practices, emphasizing the need for research focused on non-standard machine learning problems.