Automating routine business tasks often means repeating the same step-by-step instructions—known as Standard Operating Procedures (SOPs)—over and over. While advanced AI tools called Large Language Models (LLMs) can follow these SOPs written in natural language, they can be slow and expensive because they have to “think” through every step each time they run. A newly published research paper introduces ReTrace, a method that learns from a few past AI runs to create efficient, reusable workflows that don’t rely on calling the AI at every step. This approach promises to speed up automation and reduce costs without sacrificing much accuracy.
Key Takeaways
- ReTrace transforms repeated AI-driven SOP executions into compact, code-based workflows that decide the next action without calling the AI each time.
- On real-world business processes, ReTrace workflows achieved 93.8% task completion—close to the 96.8% success of full AI agents—but with 24% lower cost per task.
- In benchmark tests, ReTrace workflows ran in seconds instead of minutes, dramatically improving speed while maintaining comparable accuracy.
- When errors occurred, a process called “repair” improved workflow success rates, demonstrating that ReTrace can learn and adapt over time.
The key insight behind ReTrace is that AI agents don’t just perform actions—they also explain their reasoning at each step. These explanations reveal the conditions or “why” behind each decision, which the ReTrace system extracts and combines across multiple runs. By merging these decision rules into a concise document, ReTrace creates a traditional workflow where the sequence of steps is determined by clear, executable rules rather than by querying the AI every time.
To build these workflows, the researchers started with a handful of logged executions of an AI agent following an SOP. Each log included the actions taken and the accompanying explanations. ReTrace analyzed these logs to identify the conditions triggering each step—such as checking if a form is filled or a value exceeds a threshold. It then merged these conditions from different runs to form a generalized set of rules that cover various scenarios. Finally, these rules were compiled into workflows that could run on different automation platforms, making the approach flexible and scalable.
This method contrasts with the usual approach where the AI agent is called repeatedly during each task, which can cause delays and incur higher computational costs. By shifting decision-making from the AI to code, ReTrace workflows run faster and cheaper. The researchers tested their method on 79 business procedures from a real production platform, achieving nearly the same success rate as the full AI agent but at significantly reduced cost. Similar improvements were observed on public benchmarks designed to test tool-using AI agents.
Importantly, ReTrace isn’t perfect out of the box. Some workflows initially failed on certain tasks, but the researchers showed that applying a “repair” process—using labeled examples of failures to refine the rules—could boost success rates substantially. This suggests ReTrace workflows can improve over time with additional training data, combining the reliability of code with the adaptability of AI.
Looking ahead, ReTrace could help businesses scale AI automation more efficiently by reducing latency and cloud computing expenses. It offers a promising middle ground between fully manual coding of workflows and fully AI-driven processes, capturing the best of both worlds. Future research may explore how to automate the repair process further or extend ReTrace to more complex, dynamic tasks. For now, this new approach marks a step toward smarter, faster, and more cost-effective AI-powered automation.
Based on research published on arXiv by Kaiyuan Hou, Yang Ping Kuo, Indranil Bhattacharya et al..