Challenges in Verification for Language Models

Verification for language models presents multiple challenges, particularly with public models that struggle to deliver accuracy in output. Merely crafting good prompts does not sufficiently enhance verification capabilities. The internal research has focused on improving verification and addressing ambiguity through reinforcement learning, which aids in refining both proprietary and open-source models. Verification is a multifaceted concept, involving user-level assessments, model-level enhancements via reinforcement learning to assess output accuracy, and compiler-level code evaluations. The current lack of structured datasets that relate user prompts to generated outputs and their evaluations hampers verification efforts. There exists a need for comprehensive tracing that connects user intents with model responses, including explanations for incorrect outputs, which could provide a foundation for model training and development.