This paper introduces a novel architectural primitive that integrates attention into a recurrent feedback path, achieving power-law memory tails for superior long-context information preservation. It represents a significant breakthrough by combining the strengths of Transformers and State-Space Models to enable flexible selective retrieval that does not decay with sequence distance.
Listen on: Spotify · Apple Podcasts · Amazon Music · YouTube
Follow the show: YouTube · Medium · X · LinkedIn · Bluesky · Threads · Instagram · TikTok · Facebook · Telegram