Researchers have taken a closer look at how large language model (LLM) agents—advanced AI systems like those behind chatbots and virtual assistants—can improve by learning skills from past tasks and applying them to new challenges. This new study explores when and how these AI agents can reliably transfer skills they’ve developed, a key step toward making them more adaptable and efficient over time. Understanding this could help AI systems become better at handling a wider range of problems without starting from scratch each time.
Key Takeaways
- Skills learned at a smaller, subtask level help AI agents perform better on new tasks than skills learned from entire tasks.
- Skills represented in natural language text transfer more effectively than those encoded as computer code.
- Neither how specific a skill is to a task nor how broadly applicable it is alone predicts success; instead, a combined measure called “skill utility” better indicates useful skill transfer.
- The proposed skill utility score can be calculated without running new tasks, offering a practical way to evaluate the usefulness of stored skills before applying them.
Large language model agents operate by completing tasks using their training and experience. When these agents finish a task, they can “induce” or extract skills—essentially reusable pieces of knowledge—that might help in future tasks. However, not all skills transfer well. Sometimes, trying to reuse a previously learned skill can even reduce performance, making the agent less effective than if it had no memory of past tasks at all.
To investigate this problem, the researchers compared two main approaches to inducing skills: one that focuses on entire tasks (task-level) and another that breaks tasks down into smaller components (subtask-level). They also looked at two ways to represent these skills—either as plain text descriptions or as snippets of code. By running a series of controlled experiments, they observed how these different methods affected the AI agents’ ability to apply past knowledge to new tasks.
They found that skills learned at the subtask level consistently improved the agents’ performance, while task-level skills often did not. Additionally, skills expressed in text form transferred better than those in code form. To better understand why, the team introduced two concepts: specificity, which measures how closely a skill matches a particular task, and abstractness, which reflects how broadly a skill applies across many tasks. Interestingly, neither specificity nor abstractness alone was enough to predict successful skill transfer. Instead, a combination of the two—a “skill utility score”—was a reliable indicator of whether a skill would help on new tasks.
What makes this finding particularly useful is that the skill utility score can be calculated using only the skill descriptions and the new task details, without needing to actually run the task. This means AI systems could assess and select which skills to reuse ahead of time, potentially saving computational resources and avoiding performance drops.
Looking ahead, this research offers a promising direction for building AI agents that grow more capable by effectively reusing what they’ve learned. By focusing on breaking tasks into subtasks and representing skills in accessible text formats, future AI could become better at adapting to new challenges. The skill utility score also provides a practical tool for developers to monitor and improve skill transfer. While further work is needed to test these findings in real-world applications, this study lays important groundwork for more flexible and efficient AI agents.
Based on research published on arXiv by Yiyang Feng, Biddut Sarker Bijoy, Niranjan Balasubramanian et al..