Researchers have developed a new way to test whether artificial agents—computer programs that act in virtual worlds—can truly learn from their past experiences and improve their abilities in a lasting way. Using the popular game Minecraft as a testing ground, this study focuses on whether these agents can take what they’ve learned in one scenario and successfully apply it to new, related tasks. This kind of “self-improvement” is a key challenge in artificial intelligence, especially for agents that interact with complex environments and need to adapt over time.
Key Takeaways
- The team created MC-EvoBench, a carefully designed benchmark within Minecraft that tracks agents’ learning across 19 evolution chains—sequences of tasks meant to test skill growth and transferability.
- They found a consistent performance gap between simpler “controller” agents and more advanced “planner-backed” agents, with planners generally performing better.
- Experience gained in one task did not reliably help agents perform better in related tasks, suggesting limited transfer of learned skills.
- Early challenges, like collecting wood and crafting basic tools, often blocked agents from demonstrating more advanced capabilities later on.
To understand how well AI agents improve themselves through experience, the researchers needed a controlled environment where learning progress could be clearly measured and compared. Minecraft, with its open-ended gameplay and well-defined tasks, provides an excellent platform. MC-EvoBench represents tasks as “task-state graphs,” which map out the steps and possible states an agent encounters during a challenge. This structure allows the researchers to monitor how agents move through different stages and whether they acquire new, reusable skills.
The benchmark divides tasks into “source” and “target” roles. Agents first work through source tasks, then attempt related target tasks in a reset environment, with only specific information or artifacts allowed to carry over. This setup helps isolate whether improvements come from genuine skill transfer or just leftover resources or luck. The researchers tested agents under varied conditions—including no prior experience, related tasks, unrelated tasks, and randomized task exposure—to see how different prior knowledge affected learning outcomes. They also used multiple runs with different random seeds to ensure results were reliable.
Performance was measured using several metrics: TSR (task success rate), WGP (weighted gain percentage), and VMP (value of memory persistence), each tracking different aspects of agent improvement and transfer. The findings showed that while more sophisticated planner-backed agents generally outperformed simpler controllers, prior experience on related tasks did not consistently boost performance on new tasks. In some cases, minor gains appeared but were not statistically significant and depended heavily on chance. Additionally, bottlenecks early in the task chains—such as difficulty gathering wood or crafting tools—often prevented agents from progressing far enough to demonstrate higher-level skills.
This research, published as a new arXiv paper, provides a valuable tool for the AI community by offering a standardized way to audit and compare how embodied agents learn and improve over time in complex environments. The MC-EvoBench benchmark helps distinguish true skill acquisition from superficial gains due to resource carryover or uneven task exposure. Looking ahead, this controlled testing framework could guide the development of agents that better retain and apply knowledge across different situations, a crucial step toward more adaptable and intelligent AI systems. The researchers have made their code and data publicly available, inviting further exploration and improvement in this area.
Based on research published on arXiv by Jingwei Song, Zhisheng Chen, Bingfan Zeng et al..