Designing realistic cybersecurity training exercises, like capture-the-flag challenges, is often a painstaking process requiring experts to coordinate many technical details. A newly published research paper introduces ScenarioForge, an AI-driven system that automates the generation and validation of these complex cyber exercises. This innovation promises to make cybersecurity training more efficient, reliable, and scalable—helping organizations better prepare for real-world cyber threats.
Key Takeaways
- ScenarioForge automatically converts high-level training requirements into fully detailed cyber-range scenarios, including network setups, vulnerable services, and attack paths.
- The system validates each scenario by deploying it in a simulated environment called the Common Open Research Emulator (CORE), ensuring exercises are realistic and executable.
- Researchers developed a large dataset of 226 tested scenarios featuring hundreds of vulnerabilities and custom challenge components, all passing rigorous runtime and artifact checks.
- Benchmark tests show ScenarioForge achieves perfect success rates in generating network topologies and intent fields when combined with advanced AI models, outperforming direct generation without guided scaffolding.
Cybersecurity exercises often involve setting up virtual networks with vulnerable machines, simulated traffic, and step-by-step challenges that trainees must solve. Traditionally, experts manually design these elements—a time-consuming and error-prone task. ScenarioForge tackles this by using artificial intelligence to interpret broad training goals and automatically create detailed, consistent scenarios grounded in real resources. This includes defining network layouts, deploying vulnerable services, generating realistic network traffic, and sequencing challenges logically.
To verify that the AI-generated exercises are not only plausible descriptions but also practically deployable, the researchers integrated ScenarioForge with the Common Open Research Emulator (CORE). CORE is a platform that simulates complex network environments, allowing researchers to instantiate and test the generated scenarios in a controlled but realistic setting. Each scenario undergoes multiple validation steps, including construction checks, previewing, deployment, and live runtime validation to confirm that every challenge objective functions as intended.
At the heart of ScenarioForge’s success is its AI-assisted scaffolding approach. Instead of generating scenarios from scratch in one step—which can lead to inconsistencies—the system incrementally builds the exercise using structured guidance. This includes defining the scenario’s overall structure, using a specialized vocabulary for cyber components, applying policy rules, and providing container-image references for vulnerable services. This layered approach ensures that the final scenarios are coherent, executable, and aligned with the original training goals.
The research team also created an extensive ScenarioForge Execution Dataset, encompassing 226 fully materialized scenarios. These scenarios incorporate 294 different vulnerable service images and 85 custom-built challenge generators. Impressively, all scenarios passed strict validation criteria, confirming the system’s ability to produce high-quality, reproducible training exercises. In comparative tests, pairing ScenarioForge with advanced language models like Qwen3.8 and GPT5.6 consistently yielded perfect topology and intent generation results, highlighting the benefits of the scaffolded generation process over direct prompt-based scenario creation.
This research marks a significant step toward automating the complex task of cybersecurity exercise design. By ensuring that AI-generated scenarios are both resource-grounded and rigorously validated, ScenarioForge could help training providers rapidly develop diverse, realistic exercises tailored to evolving cyber threats. Future work may explore expanding the range of supported vulnerabilities, refining challenge sequencing, and integrating real-time feedback to further enhance training effectiveness.
Based on research published on arXiv by Jaime C. Acosta, Anantaa Kotal, Aritran Piplai.
