Multimodal Image Editing Tools Struggle to Detect Unsafe Content in Both Text and Images

Photo of author

By Sophia Chen

As artificial intelligence continues to advance, tools that edit images based on user instructions have become increasingly popular. These systems combine understanding of both images and text prompts to modify pictures in creative ways. But a newly published research paper reveals that current AI image editing tools often miss potential safety risks because they focus on either the text instructions or the images—not both together. This gap could allow unsafe or harmful content to slip through, raising concerns about responsible AI use.

Key Takeaways

  • Researchers created EditSafetyBench, a new benchmark with nearly 6,000 examples pairing images and editing instructions, including safe and unsafe content across 11 risk categories.
  • Four advanced image editing systems were tested: Gemini, GPT, Qwen API, and Qwen Local, revealing inconsistent safety responses to risky inputs.
  • Systems were much more likely to block unsafe text prompts (over 80% refusal rate) than unsafe images combined with neutral instructions (only about 22-24% refusal).
  • Different models showed varying sensitivity to categories like political content and sexual material, with sexual content being the most reliably detected visually.

The researchers behind this study set out to understand how well current AI image editors can identify and block unsafe content when users upload images and provide text instructions for editing. “Unsafe” here refers to content that could be harmful, inappropriate, or sensitive, such as political misinformation, violent imagery, or sexual content. Prior safety measures often focused on just one input type—for example, screening only the text prompts users type or only the images they upload. But in instruction-guided image editing, the risk can come from either or both, so a more comprehensive approach is needed.

To investigate this, the team developed EditSafetyBench, a large test set containing 5,940 pairs of images and editing instructions. These pairs cover 11 different categories of risk and include three types of editing goals. By running these pairs through four popular image editing AI systems—Gemini, GPT, Qwen API, and Qwen Local—the researchers could measure how often and under what conditions each system refused to process potentially unsafe requests.

One of the key findings was a “text-vision safety mismatch.” This means that the systems were much better at detecting unsafe text prompts than unsafe images combined with neutral or harmless instructions. For instance, Gemini and GPT blocked about 87% and 82% of cases where the text prompt itself had unsafe editing intent. But when the image was unsafe and the instruction was neutral, these refusal rates dropped sharply to around 24%. This suggests that the systems do not sufficiently analyze the visual content in tandem with the text instructions to catch risks effectively.

The study also found that different AI models responded inconsistently across risk categories. For example, when it came to political content deemed unsafe, GPT allowed about 70% of editing attempts, while Gemini only allowed about 35%. Sexual content, on the other hand, was most reliably detected and blocked, likely because there are already well-developed tools to identify not-safe-for-work (NSFW) images. This uneven performance highlights the challenge of building safety mechanisms that work well across diverse types of sensitive content.

In practical terms, this research points to an important gap in current AI image editing tools: they need more sophisticated safety guardrails that jointly consider both the image and the text instruction before proceeding with edits. Without this, users could unintentionally or maliciously generate or modify images containing harmful content. The authors suggest that future development should focus on multimodal safety systems that analyze visual and textual data together to make more informed decisions.

As AI-powered image editing becomes more widespread—in social media, content creation, and professional design—ensuring these tools handle unsafe inputs responsibly is critical. This new benchmark and analysis provide a valuable foundation for improving safety in multimodal AI systems. Going forward, researchers and developers can use these insights to build more robust safeguards that better protect users and communities from harmful content.

Based on research published on arXiv by Ziqing Yang, Yixin Wu, Michael Backes et al..

Editor's note

Editors matched this AI update with related coverage to show where it sits in the broader race over models, regulation and product strategy.

Article briefing

As artificial intelligence continues to advance, tools that edit images based on user instructions have become increasingly...

Story details

  • Author: Sophia Chen
  • Published: November 22, 2026
  • Category: AI

Key developments

  • As artificial intelligence continues to advance, tools that edit images based on user instructions have become increasingly popular.
  • This gap could allow unsafe or harmful content to slip through, raising concerns about responsible AI use.
  • The researchers behind this study set out to understand how well current AI image editors can identify and block unsafe content when users upload images and provide text instructions for editing.

Why this matters

But a newly published research paper reveals that current AI image editing tools often miss potential safety risks because they focus on either the text instructions or the images—not both together.

Impact and next steps

"Unsafe" here refers to content that could be harmful, inappropriate, or sensitive, such as political misinformation, violent imagery, or sexual content.

Background

Sexual content, on the other hand, was most reliably detected and blocked, likely because there are already well-developed tools to identify not-safe-for-work (NSFW) images.

Source

This article is based on source material from arxiv.org.

About the author

Sophia Chen

Sophia Chen covers artificial intelligence and emerging technology. With a background in computer science and a decade of tech journalism, she specialises in AI policy, machine learning applications and the societal impact of automation.

editorial@peacknews.com

Categories AI