The ICASSP 2026 HumDial Challenge paper introduces a standardized benchmark for evaluating human-like spoken dialogue systems in the era of advanced Audio-LLMs. While current models excel at task completion, measuring their ability to replicate the subtle nuances of natural human communication requires assessing deep emotional resonance and complex turn-taking. To address this gap, the authors created a sizable dataset using a hybrid approach of LLM-generated scripts performed by professional human actors to preserve authentic conversational dynamics.
The challenge evaluates systems across two core dimensions:
Key findings from the challenge submissions showed that while top systems are highly capable of analyzing emotional logic and reasoning, generating truly empathetic vocal and textual responses remains a significant difficulty. Furthermore, in full-duplex interactions, maintaining silence and distinguishing valid user turns from ambient background noise was identified as the primary hurdle for current systems.
Listen on: Spotify · Apple Podcasts · Amazon Music · YouTube
Follow the show: YouTube · Medium · X · LinkedIn · Bluesky · Threads · Instagram · TikTok · Facebook · Telegram