This paper introduces K-VEC, a coverage-aware KV-cache eviction strategy that resolves performance drops in long-context tasks by maximizing token retention across attention heads and layers. By theoretically linking token coverage to mutual information, it provides a foundational efficiency breakthrough for deploying long-context LLMs under resource constraints.
Short on time? Watch the video overview of this paper.
Listen on: Spotify · Apple Podcasts · Amazon Music · YouTube
Follow the show: YouTube · Medium · X · LinkedIn · Bluesky · Threads · Instagram · TikTok · Facebook · Telegram