How well can AI model from drawings? Reading CADBench V3

Originally published by AI驱动产品设计 on October 18, 2026. Results refer to the September 24, 2026 CADBench V3 snapshot and the original author's inspection of its public task material. This republication is not a new modeling experiment.

This article interprets gNucleus's Parametric CAD Bench V3, called CADBench here. Reported data come from its official pages and public task material, checked for the original article through October 18, 2026. The analysis and engineering judgments are the original author's.

An earlier article discussed Normal's CAD Arena: 18 drawings, five CAD platforms, and separate attention to geometry and editability. CADBench V3 asks a related question under a different contract. It uses FreeCAD; 40 of its 100 tasks provide an engineering drawing without a textual parameter specification. Geometry and dimensional consistency are combined, with failures scored zero 1[3]. In the September snapshot, image-to-CAD scores ranged from about 8 to 85, a much wider spread than text-only creation 2. This article focuses on those drawing tasks.

1. A drawing must become a feature-history model

The original author downloaded and inspected the public task package 5. Tasks use A3 engineering drawings with orthographic, sectional and isometric views, dimensions and hole callouts such as 4×Ø3.5, along with general tolerances and surface-finish symbols. Typical parts include servo mounts, brackets, pulleys, housings and links.

Original CADBench V3 drawing and reference-model task material. The original article illustrated task freecad-1dd4c42537, an MG90 servo mount. This edition links the dataset examples at source.

The drawing is the only geometric specification: no text description or parameter table is supplied. An agent writes a FreeCAD Python script and must produce a file with one PartDesign Body and a single solid constructed through feature history. Storing only a history-free shape does not meet that requirement 5. This is a sketch-and-feature task, not merely an exercise in making a plausible exterior.

Each task has a 9,000-second limit and one attempt. Agents have no network access except to the selected model provider, and reference geometry remains in an isolated verifier environment. Failed trials receive zero 3.

2. Scoring: shape and dimensions both matter

Each task receives a score from 0 to 1, combining geometric similarity and dimensional consistency with a harmonic mean. If either component is zero, the combined result is zero 5[6].

Geometry: an oriented bounding box checks the three principal extents; a 10% size discrepancy triggers zero. Subsequent comparisons include volume, surface area and spatial alignment of face centers to examine feature positions 6.

Dimensions: the hidden reference tables for the 40 drawing tasks contain 9–118 dimensions each, totaling 1,446. Roughly one tenth are tied to specific geometric locations. Most others are matched to the closest numerical value of a compatible type in sketch constraints, feature parameters or body geometry, with 1% relative tolerance. Each missing or wrong dimension deducts 0.1, so ten errors reduce that component to zero. The original author's inspection found no corresponding entries for general tolerance tables or surface roughness 5[6].

I read this as stricter than shape-only comparison: a misplaced hole can affect both components. But numerical matching also has limits; the same value occurring elsewhere can potentially satisfy a check without establishing the intended location.

Figure 1. AI-generated concept illustration of the geometric and dimensional checks. It is not a separate scoring specification.

3. The image-to-CAD scorecard

The September 24 snapshot contains 13 model-and-agent configurations 2. The selected rows below show image-to-CAD, text creation and the overall mean:

• Claude Opus 5.5 — Agent: Claude Code; Image-to-CAD: 84.66; Text creation: 50.87; Overall: 61.03
• Claude Opus 5.5 — Agent: mini-swe-agent; Image-to-CAD: 81.77; Text creation: 42.79; Overall: 57.96
• Claude Fable 5.1 — Agent: Claude Code; Image-to-CAD: 72.71; Text creation: 55.48; Overall: 56.75
• GPT-6 Astra — Agent: Codex; Image-to-CAD: 69.70; Text creation: 52.26; Overall: 56.87
• Claude Opus 5 — Agent: Claude Code; Image-to-CAD: 62.64; Text creation: 49.48; Overall: 50.86
• Gemini 3.8 Flash — Agent: mini-swe-agent; Image-to-CAD: 31.10; Text creation: 47.94; Overall: 36.19
• Grok 4.7 — Agent: Grok Build; Image-to-CAD: 26.91; Text creation: 45.57; Overall: 34.82
• Gemini 3.8 Flash — Agent: Antigravity; Image-to-CAD: 17.45; Text creation: 43.28; Overall: 27.56
• Kimi K3 — Agent: mini-swe-agent; Image-to-CAD: 15.36; Text creation: 39.11; Overall: 24.92
• Grok 4.6 — Agent: Grok Build; Image-to-CAD: 13.00; Text creation: 41.61; Overall: 27.72
• GPT-5.6 Terra — Agent: Codex; Image-to-CAD: 8.18; Text creation: 23.32; Overall: 17.24

Scores use a 0–100 scale. The 30 text-based editing tasks are not shown. Kimi K3 was the clearly identified Chinese-model entry in this snapshot; the original survey found no DeepSeek, Qwen or Doubao entry 2.

Score illustration derived from the table above, preserving the original historical snapshot.

Across all 13 rows, the original author calculated an image-to-CAD spread of 76.48 points, from 8.18 to 84.66. The text-creation spread was 32.16 points and the editing spread 21.14 points. The leading five configurations scored higher on drawing tasks than on text creation; the other eight showed the reverse. With 40 tasks, this category strongly influences the overall ranking. The official update cautions that the leading four overall 95% confidence intervals overlap; the point estimates do not establish decisive separation 2[4].

4. Visual reasoning and delivery of the image are different issues

One issue is whether a model can read views, small dimension labels and their associated features. Another is whether the agent actually passes the drawing's pixels into a multimodal model, rather than merely mentioning an image file.

Same agent, different model. With Claude Code, moving from Opus 5 to Opus 5.5 increased drawing scores by 22.02 points, compared with only 1.39 points for text creation. The total cohort cost fell by 43.3% 4. My interpretation is that drawing interpretation contributes substantially to the difference; this benchmark does not isolate every cause.

Same model, different agent. Gemini 3.8 Flash with mini-swe-agent's multimodal bridge scored 31.10 on drawings, versus 17.45 with Antigravity: a 13.65-point mean difference, with a paired 95% interval of 5.55–21.75. The report explicitly says prompts, tool protocols and serving paths differ, so this is not a controlled agent-only ablation 3. The two Opus 5.5 agent configurations differed by 2.90 points on drawings 4. Capability depends on both the model and the surrounding delivery-and-tool workflow.

The bridge introduced its own failure mode. In six Gemini drawing tasks, later shell-error output was marked as image data, the model API rejected the invalid payload, and the trials remained zero. The original drawing had been delivered correctly; the error concerned subsequently requested images 3. A May report also described stronger agents printing volume after recomputation as a check, while a weaker trajectory failed to notice a reversed cut direction 7.

Figure 2. AI-generated concept illustration: reading the drawing, delivering the image and checking the model are separate responsibilities.

5. What the results do and do not show

1. Useful drafts, not automatic release. The best configuration averaged 84.66 on these structured drawing tasks. That supports interest in feature-history reconstruction, not certification of tolerances, fits or robust modification 2[5].
2. Visual interpretation matters. The Opus comparison shows a much larger gain on drawings than on text creation. That is consistent with improved reading of views and dimensions, without proving a single causal mechanism 4.
3. The interface matters too. The Gemini comparison demonstrates different outcomes for the same named model under different agent workflows. It does not prove that every lower score resulted from an absent original image 3.
4. Checks belong inside the loop. Printing or measuring volume after recomputation is a minimal example of inspecting the result rather than assuming a successful command made a correct part 7.
5. Only the first part of a wider workflow is measured. The proposed Drawing-to-CAD workflow also includes source comparison, properties, drawings, managed storage and engineering approval. These tasks concentrate on scripted feature-history modeling.
6. CAD Arena and CADBench scores are not interchangeable. They use different tasks and evaluators. Similar leading model names do not make 0.75 and 84.66 directly comparable 2[8].
7. Do not generalize one Chinese-model row to all domestic models. Kimi K3 scored 15.36 on drawings and 24.92 overall in this snapshot, well below the leading row. Other named Chinese models were absent, so the table cannot establish their performance 2.

Related: why reliable CAD agents need checks after each step.

Illustrations are conceptual, not screenshots from a new product test. The English edition uses localized English illustrations; official dataset examples remain linked at source.