As artificial intelligence systems become more integrated into daily life, ensuring they act safely—even when given incomplete information—is crucial. A newly published research paper introduces a tool to test how well AI language models, especially those that use external tools or data, handle situations where important facts are missing. This matters because AI agents sometimes make decisions based on partial information, which can lead them to perform unsafe or prohibited actions despite all provided statements being true.
Key Takeaways
- The researchers created EpiSEM-Bench, a new benchmark with 300 paired tasks designed to test AI agents’ safety when critical evidence is missing.
- They found significant differences among ten popular language models in how well they detect missing facts and avoid unsafe actions.
- Some models actively seek out missing information before acting, while others proceed without verifying, increasing risk.
- Access to recovery mechanisms that help AI gather missing evidence reduces unsafe actions across all tested models, but not all agents use these tools effectively.
Large language models (LLMs) that use tools—like databases or external APIs—depend heavily on accurate and complete information from their environment to decide what actions are safe or allowed. However, in real-world settings, these tools may sometimes omit important safety-related facts. For example, an AI assistant might receive partial data about a user’s request and unknowingly carry out an action that should be prohibited. The challenge is that even if all the facts it does receive are true, missing critical evidence can lead to unsafe behaviors.
To better understand and improve AI safety in these scenarios, the authors developed EpiSEM-Bench. This benchmark tests AI agents by presenting them with carefully paired tasks—one safe and one unsafe—across six different domains. By systematically removing critical facts from the information provided, the benchmark measures how often and under what conditions the AI makes unsafe decisions. It also identifies the minimum number of failures observed, including situations where AI agents never fail even after exhaustive testing.
One innovative aspect of the benchmark is its independent audit tool, which evaluates whether an AI agent actively looks for missing facts before deciding on an action. This is important because good safety behavior involves not just avoiding unsafe actions but also recognizing when there isn’t enough information to proceed. The audit checks if the agent withholds prohibited actions in uncertain scenarios while still completing tasks that are allowed.
Testing ten different language models revealed notable variation. Some models were robust against missing evidence—they either detected the gaps or used recovery processes to obtain the missing facts. Others were less cautious, proceeding without sufficient information and increasing the risk of unsafe execution. Interestingly, having access to recovery tools that help retrieve missing facts consistently reduced unsafe actions, but only some agents effectively used these tools to improve their decision-making.
These findings highlight the importance of developing AI systems that not only rely on available data but also actively verify and seek out missing critical information before acting. The EpiSEM-Bench benchmark provides a valuable framework for researchers and developers to evaluate and enhance the safety of AI agents operating under incomplete evidence.
Looking ahead, this research could guide improvements in AI safety protocols and tool design, especially for applications in sensitive areas like healthcare, finance, and autonomous systems where missing information can have serious consequences. By encouraging AI models to be more cautious and evidence-aware, developers can help reduce unintended harmful actions and build more trustworthy intelligent assistants.
Based on research published on arXiv by Qijin Zhang, Hanzhen Wang, Yuehan Zhao et al..