State of Progress in Open Alignment Space

Progress in language model fine-tuning, specifically in online DPO variants, remains mostly stagnant despite new code bases, datasets, and papers. The speaker recently discussed the need for accelerating progress, highlighting challenges with open-source tools for training open-aligned models, such as TRL's various loss functions. Notable findings from a recent paper focused on unpacking DPO and PPO best practices, aiming to enhance proximal policy optimization's performance when compared to industry standards.