As artificial intelligence systems tackle more complex problems, they often need to sift through large amounts of diverse information to find the right answers. Traditional methods usually rely on simply retrieving the most relevant pieces of text based on similarity, but this approach can fall short when the answer isn’t plainly stated in any single source. A newly published research paper introduces a novel framework called RECAST that helps AI models actively piece together evidence by combining retrieval with computation, enabling more accurate and flexible problem-solving.
Key Takeaways
- RECAST treats evidence gathering as a step-by-step decision process, allowing the AI to filter, combine, and compute information from multiple sources rather than just retrieving similar text.
- The system uses a lightweight “RouterLM” model to decide which operations to perform, translating these into executable code for processing information.
- On six diverse benchmark tests, RECAST achieved a success rate of 75.6%, outperforming the best existing large-scale language model methods by nearly 16%.
- RECAST also demonstrated strong zero-shot generalization, meaning it performed well on new, unseen tasks without additional training.
In many real-world applications, answers aren’t found by simply pulling up a single relevant document or snippet. Instead, the solution may require synthesizing information scattered across multiple sources, performing calculations, or applying logical reasoning. Existing AI methods often rely heavily on retrieval, which limits their ability to handle these complex tasks effectively.
RECAST addresses this challenge by framing evidence collection as a sequential decision-making process. At its core is the RouterLM, a smaller language model trained to select what action to take next—whether to retrieve a piece of information, apply a computation, or combine evidence gathered so far. Instead of just searching for matching text, RouterLM can specify customized operations that are translated into executable code by a separate “CompilerLM.” This code then processes the information to derive new evidence. The system repeats this cycle, refining and expanding the evidence until it judges it has enough to answer the question. Finally, the accumulated evidence is passed to an AnswerLM, a frozen model responsible for generating the final response.
Training RECAST involves two stages: supervised fine-tuning (SFT), where RouterLM learns from examples of good decision sequences, followed by group relative policy optimization (GRPO), a reinforcement learning technique that helps RouterLM improve its decision-making policies. This combination enables RECAST to effectively navigate complex information sources that vary widely in format and content.
Tests on six different benchmark families—which include tasks with heterogeneous and long information sources—show that RECAST consistently outperforms traditional retrieval-based models and even advanced large language models. Notably, it achieves a 15.9% improvement over the strongest baseline and maintains strong performance on three new benchmarks it wasn’t trained on, highlighting its ability to generalize to new tasks and data types.
The introduction of RECAST is an important step toward AI systems that can more intelligently and flexibly handle real-world information challenges, where answers often require more than just looking up facts. By combining retrieval, computation, and adaptive decision-making, RECAST opens up possibilities for more reliable AI assistance in fields like research, data analysis, and complex problem-solving. Future work may explore scaling this approach further and integrating it into practical applications where understanding and synthesizing diverse information is critical.
Based on research published on arXiv by Yilun Hao, Krishna Sayana, Isabella Ye et al..