Spool speeds up video question-answering with smarter memory for vision-language AI

Photo of author

By Sophia Chen

Researchers have developed a new technique called Spool that helps advanced AI models quickly answer questions about recent video footage. This breakthrough is important for applications like assistive technology for visually impaired users and robotics, where timely understanding of ongoing video is critical. The challenge lies in efficiently processing a continuous stream of video frames without repeatedly reanalyzing old footage, which can slow down responses. Spool offers a clever way to “remember” recent video content while discarding outdated information, enabling faster and more accurate answers.

Key Takeaways

  • Spool improves the speed of hybrid vision-language AI models, reducing the delay before the model starts generating answers by over six times compared to recent methods.
  • It achieves this without any additional training, by caching and efficiently updating internal model states as new video frames arrive.
  • The technique maintains exact removal of expired frames’ influence within each recurrent layer, ensuring up-to-date understanding of the video.
  • In tests across multiple models and video question-answering tasks, Spool also improved accuracy by an average of 2.3 percentage points.

Hybrid vision-language models (VLMs) combine image and text understanding, making them useful for interpreting video and answering related questions. These models often use recurrent layers, which process sequences by maintaining a state that summarizes all past inputs. However, when working with a sliding window of recent frames—say, the last two minutes of video—removing old frames from the model’s memory typically requires reprocessing the entire window, which is computationally expensive and slow.

The key insight behind Spool is recognizing that the recurrent update step in these models is an affine transformation with input-dependent coefficients. In simpler terms, each new video frame transforms the model’s internal state in a predictable, mathematically composable way. By caching these per-frame transformations, Spool can quickly combine or remove their effects as the window moves forward—without needing to feed the frames back through the entire model.

To achieve this, the researchers designed a specialized data structure and GPU-optimized routines that efficiently store and compose these transformations. This approach ensures that expired frames are exactly removed from each recurrent layer’s state, keeping the model’s memory focused only on the relevant recent video. While some residual influence from older frames may persist through earlier layers’ cached inputs, the overall system maintains high accuracy and responsiveness.

Tests on five different open-source hybrid VLMs across four video reasoning benchmarks showed that Spool consistently delivered the fastest time-to-first-token (the moment the model starts producing an answer), outperforming two recent state-of-the-art baselines. The reduction in latency was significant, with a geometric mean speedup of 6.6 times. Additionally, Spool improved the accuracy of answers by about 2.3 percentage points on average, demonstrating that faster inference did not come at the cost of quality.

This new method could have a meaningful impact on real-time AI applications that rely on interpreting video streams. For example, visual accessibility tools could respond more swiftly to questions about a user’s surroundings, and robots could better understand and react to dynamic environments. Since Spool does not require retraining models, it can be integrated into existing systems to enhance performance immediately. Future work may explore extending this approach to other forms of sequential data and further optimizing the balance between speed and accuracy in vision-language AI.

Based on research published on arXiv by Rui Ning, Om Chabra, Hari Balakrishnan.

Editor's note

This article focuses on the confirmed update first, then points readers to the competitive and policy context that shapes the beat.

Article briefing

Researchers have developed a new technique called Spool that helps advanced AI models quickly answer questions about recent video...

Story details

  • Author: Sophia Chen
  • Published: November 18, 2026
  • Category: AI

Key developments

  • Researchers have developed a new technique called Spool that helps advanced AI models quickly answer questions about recent video footage.
  • The challenge lies in efficiently processing a continuous stream of video frames without repeatedly reanalyzing old footage, which can slow down responses.
  • Spool offers a clever way to “remember” recent video content while discarding outdated information, enabling faster and more accurate answers.

Why this matters

While some residual influence from older frames may persist through earlier layers’ cached inputs, the overall system maintains high accuracy and responsiveness.

Impact and next steps

This new method could have a meaningful impact on real-time AI applications that rely on interpreting video streams.

Background

This breakthrough is important for applications like assistive technology for visually impaired users and robotics, where timely understanding of ongoing video is critical.

Source

This article is based on source material from arxiv.org.

About the author

Sophia Chen

Sophia Chen covers artificial intelligence and emerging technology. With a background in computer science and a decade of tech journalism, she specialises in AI policy, machine learning applications and the societal impact of automation.

editorial@peacknews.com

Categories AI