This paper identifies 'cliff tokens' as the exact single-token triggers that cause large language models to diverge into reasoning failures during multi-step mathematical tasks. By introducing a taxonomy of these failures and a targeted preference optimization method (Cliff-DPO), it establishes a foundational approach to diagnosing and improving LLM reasoning reliability.
Short on time? Watch the video overview of this paper.
Listen on: Spotify · Apple Podcasts · Amazon Music · YouTube
Follow the show: YouTube · Medium · X · LinkedIn · Bluesky · Threads · Instagram · TikTok · Facebook · Telegram