As artificial intelligence systems tackle increasingly complex tasks, they often need to process vast amounts of information—sometimes millions of words or “tokens”—to make smart decisions. However, these systems have a limited memory window, meaning they can’t keep all that information active at once. This limitation forces AI developers to find ways to compress or “compact” past information efficiently without losing important details. A newly published research paper introduces a novel technique called CliffCompaction that promises to reduce the computational cost of handling long information streams by up to half, while maintaining or even enhancing performance.
Key Takeaways
- CliffCompaction reduces the cost of processing long AI sessions by up to 50% without sacrificing accuracy.
- The technique improves efficiency in test-time scaling, enabling better performance for less computing power.
- It achieves state-of-the-art results on benchmarks like KernelBench, outperforming specialized algorithms.
- CliffCompaction maintains information integrity by only removing or truncating data, never rewriting it, preventing errors from accumulating over time.
Traditional AI models struggle with “context windows,” which limit how much past information they can consider simultaneously. When working on tasks that require understanding or reasoning over millions of tokens, this limitation becomes a bottleneck. To address this, systems often use “compaction” methods to summarize or compress previous data. However, many existing compaction approaches risk distorting the original information, leading to degraded performance over time.
CliffCompaction takes a different approach by focusing strictly on removing less relevant content rather than rephrasing or rewriting it. This strategy ensures the compacted information remains faithful to the original input. Additionally, the method discards any previously compacted data before each new compaction pass, avoiding the buildup of errors or “context drift” that can occur when summaries are repeatedly summarized. This careful handling allows AI agents to learn continuously over extremely long sessions without losing accuracy.
The researchers tested CliffCompaction on challenging benchmarks like Terminal-Bench and KernelBench, which simulate real-world coding and reasoning tasks. Notably, CliffCompaction enabled a CUDA kernel—a core component of GPU computing—to speed up by over 3.5 times after 400 steps, surpassing other specialized search algorithms and trained agents. It also made test-time scaling more cost-effective, boosting performance by more than 10 percentage points on Terminal-Bench for less than the cost of running two full-context sessions.
Furthermore, when running multiple tests in parallel, CliffCompaction allowed the AI model Kimi K2.6 to match or exceed the performance of larger, more expensive models like Opus 4.7 and GPT-5.3 Codex at a lower computational cost. The researchers have also made their approach accessible by open-sourcing a flexible software interface compatible with popular AI platforms such as Claude Code and Codex.
This development could have significant implications for AI applications that require long-term memory and reasoning, such as complex programming, scientific research, and multi-session dialogues. By cutting costs and improving efficiency without compromising accuracy, CliffCompaction may help make advanced AI capabilities more accessible and scalable. Future work may explore integrating this technique into a wider range of AI systems and further optimizing it for specific real-world tasks.
Based on research published on arXiv by Trang Nguyen, Eulrang Cho, Bingqing Chen et al..