Checking whether civil infrastructure plans meet regulatory standards is a painstaking task traditionally done by engineers poring over detailed 2D drawings. These plans contain crucial geometric and layout information that is often lost when automated systems rely solely on text extraction methods like Optical Character Recognition (OCR). A newly published research paper introduces PlanSightRAG, an AI framework designed to understand and analyze engineering plans visually, aiming to automate compliance checking more accurately and efficiently.
Key Takeaways
- PlanSightRAG processes plan images directly, preserving geometric and layout details that OCR-based methods typically miss.
- The system achieved over 91% accuracy in retrieving relevant plan information without prior training on specific datasets, demonstrating strong zero-shot performance.
- On synthetic compliance drawings, the AI pipeline reached perfect accuracy in deciding compliance when given pre-defined rule thresholds, outperforming standard OCR baselines.
- PlanSightRAG can autonomously extract numeric rules from specification documents without human-provided thresholds, enabling more flexible and scalable compliance checking.
At the heart of PlanSightRAG is a “Visual-First Multimodal Retrieval-Augmented Generation” (RAG) approach. Unlike traditional systems that convert images to text before analysis, this framework indexes and reasons directly over the visual content of engineering plans. This means it understands the spatial layout, shapes, and graphical elements crucial for interpreting civil infrastructure designs. The system combines several AI components: a multi-vector retrieval engine called ColNomic-3B that searches plan imagery efficiently; an agentic framework named Planner-Retriever-Auditor-Synthesizer that coordinates plan interpretation and verification; and MaxSim heatmaps that provide transparent evidence trails showing how the AI reached its conclusions.
The researchers built a new benchmark dataset containing over 4,000 paired samples from five state Departments of Transportation standard plans, totaling nearly 1,900 pages. PlanSightRAG demonstrated strong performance on this dataset, retrieving relevant plan sections with high recall rates even without prior specific training (“zero-shot”). In tests using a separate Michigan DOT corpus, the system maintained similarly high accuracy, indicating its robustness across different regional standards.
To further evaluate compliance checking, the team generated synthetic drawings with parametrically controlled features. Using a large multimodal language model pipeline (Qwen2.5-VL-72B), PlanSightRAG achieved 100% accuracy in assessing whether plans met rules—when supplied with exact numeric thresholds from the regulations. This contrasts with a typical non-visual OCR-based baseline that reached only about 76% accuracy under the same conditions. Importantly, PlanSightRAG also showed the ability to autonomously identify these numeric limits directly from textual specification documents, eliminating the need for manually inputted rule thresholds. This capability marks a significant step toward fully automated and adaptable compliance verification.
The implications of this research are promising for civil engineering and infrastructure management. Automating compliance checks with a system that “sees” and understands plans as humans do could reduce errors, speed up project approvals, and lower costs. Moreover, by extracting rules directly from specifications, the approach can adapt to evolving standards without extensive reprogramming. Future work may focus on expanding the framework to other types of technical drawings, improving integration with engineering workflows, and refining the AI’s interpretability to build trust with human experts. As infrastructure projects grow in complexity, tools like PlanSightRAG may become essential allies in ensuring safety and regulatory compliance.
Based on research published on arXiv by Nabaraj Subedi, Shuvo Dip Datta, Ahmed Abdelaty et al..