Back to Feed
Multimodal / Benchmarks & Evals

Automating Civil Engineering Plan Compliance Checking

Original: PlanSightRAG: A Visual-First Multimodal RAG for Automating Question Answering and Compliance Checking for Civil Standard Plans
By Nabaraj Subedi, Shuvo Dip Datta, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar Published 2026-08-26

Listen to the summary

Uses a voice available on your device

Audio options
On this page 4 sections
Related concepts 4 concepts

Key Takeaways

  • The ColNomic-3B model achieves 91.47 percent Recall@5 on a new 4,056-pair benchmark across five DOT standard plans.
  • The retrieval performance significantly outperforms text-based baselines, scoring 55.90 percentage points higher than BGE-M3 combined with OCR.
  • An agentic compliance pipeline attains 100 percent accuracy on a 500-drawing single-doc CAD test set.
  • The system provides robust performance on held-out data, achieving 91.40 percent Recall@5 on the Michigan DOT corpus.

Summary & Methodology Analysis

The framework utilizes a visual-first approach to address the limitations of text-centric retrieval systems, which often lose spatial data in technical engineering documents. The core component, ColNomic-3B, employs a late-interaction retrieval mechanism that processes patch-level embeddings to perform similarity scoring. This design allows the system to capture spatial relationships directly from the imagery, bypassing the information loss typically associated with OCR-based pipelines. The retriever is validated against a 4,056-pair benchmark derived from 1,898 pages of standard plans from five state Departments of Transportation, where it demonstrates a 92.69 percent Recall@5 score, outperforming the text-based BGE-M3 plus OCR baseline by 55.90 percentage points.

Interactive System Flowchart

Click diagram to expand and zoom

Cross-Examination & FAQs

A deeper dive clarifying mechanics, constraints, and baseline evaluations.

Q1. What is the primary goal of PlanSightRAG?

The system aims to automate compliance checking for civil engineering standard plans, which is traditionally a manual and error-prone process.

Q2. How does this system improve upon existing methods?

It avoids the pitfalls of text-centric RAG systems by prioritizing visual information, leading to significantly higher retrieval accuracy for engineering documents.

Q3. Is this system ready for production use by licensed engineers?

While the agentic pipeline reaches 100 percent verdict accuracy on a specific CAD test set, the benchmark relies on machine-generated data rather than direct manual annotation by licensed engineers.

Q4. What is the performance of the ColNomic-3B model?

ColNomic-3B reaches 91.47 percent Recall@5 on the 4,056-pair benchmark and 91.40 percent Recall@5 on the Michigan DOT corpus.

Q5. How does the ColNomic-3B retriever compare to text-based models?

The ColNomic-3B retriever reaches 92.69 percent Recall@5, which is 55.90 percentage points higher than the strongest text-based baseline using BGE-M3 and OCR.

Q6. What role does Qwen2.5-VL-72B play in the architecture?

The Qwen2.5-VL-72B pipeline is used for compliance audits on synthetic, parametrically-generated drawings, where it reaches 100 percent verdict accuracy.

Q7. What are the known limitations of the current system?

The system is sensitive to low-quality scans due to its reliance on high-resolution visuals and the benchmark currently uses machine-generated QnA pairs.

Q8. How was the benchmark developed?

The authors introduced a 4,056-pair benchmark derived from 1,898 pages of standard plans from five state Departments of Transportation.

Q9. Does the system handle multi-drawing queries?

Yes, the agentic Planner-Auditor-Synthesizer pipeline is designed to decompose multi-plan compliance queries into structured verification steps.

Flag an issue

What is wrong with this summary?

Browse the archive →
Multimodal / Benchmarks & Evals Detecting Android Rotation Bugs Using AI RotDroid uses a vision-language model to detect GUI rotation bugs by comparing visual states between portrait and landscape orientations. Multimodal / Benchmarks & Evals Comparing Multimodal Pipelines for Document Retrieval The paper evaluates various retrieval-augmented generation pipelines, finding that multimodal vision-based approaches significantly outperform text-based methods despite introducing higher latency and storage costs. Multimodal / Benchmarks & Evals WeChat Multi-modal Embedding Technical Report The paper introduces WeMM-Embedding, a series of multimodal models based on Qwen3.5 that achieve state-of-the-art performance on retrieval benchmarks and demonstrate consistent gains in production applications.