In this episode, we discuss DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models by Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, Daya Guo. The paper presents DeepSeekMath 7B, an advanced language model trained on 120 billion math-related tokens to improve mathematical reasoning. The model scores 51.7% on the MATH benchmark, and by using an approach called self-consistency, it reaches 60.9%, approaching the results of state-of-the-art models like Gemini-Ultra and GPT-4 without external aids. The success of DeepSeekMath is attributed to the use of an extensive web data collection and a novel optimization algorithm called Group Relative Policy Optimization (GRPO) that improves math reasoning while being memory-efficient.
AI Breakdown