FrontierCAD: evaluating parts that fit and assemblies in place

Originally published by AI驱动产品设计 on October 10, 2026. This republication preserves the original analysis and the September 19, 2026 evaluation run. It does not present the results as a current model ranking.

This article interprets Normal's FrontierCAD research. Reported measurements belong to that study; the engineering judgments are the original author's analysis.

The workflow is detailed design → assembly and fit. Shape similarity cannot tell us whether bolt holes align or parts sit in the correct relative positions. FrontierCAD has one task for rebuilding a part from its drawing and another for positioning supplied parts from an assembly drawing. Its inspectable assembly-related regions come from reference geometry. That does not mean the AI understands a complete assembly's design intent.

The research index dates the article to October 5, 2026. The authors are Aayush Gupta, Pav Saini, Anson Yu and Hudhayfa Nazoordeen. The reported run evaluates seven models on 39 parts and 16 assemblies, with one attempt per task, on September 19 1[2][3].

1. Two tasks concerned with fit

The part task supplies an engineering drawing and requires a STEP solid. The assembly task supplies STEP parts and an assembly drawing with toleranced dimensions, a bill of materials and balloons; the model positions those existing parts. Normal explicitly says that selecting parts, adding mates and fasteners, and checking motion were not yet released 1.

Human reconstruction checked whether drawings contained sufficient information. CADQuests participants used Onshape and drawings alone in July and August 2026; references were accepted with a 3% comparison threshold. Under the same verifier, the human baseline averaged 0.691 on the 39 parts, with only one pass and a median active modeling time of 84 minutes. The report calls this a baseline, not a ceiling: drawings do not completely specify every blend, relief and internal detail 2.

2. Similar shapes can fail to assemble

Controlled perturbations expose two problems with existing scoring methods 1.

Orientation. A drawing generally does not require a part to occupy one absolute world orientation. Rotating a correct part should not make it wrong. Across four parts and rotations from 0.1° to 90°, CADGenBench fell to 0.81 at 1°, BenchCAD to 0.12 at 90°, while CADBench stayed between 0.96 and 0.99. FrontierCAD scored 1.000 at each angle. BenchCAD's feature component was set to zero, preventing a maximum score of 1.0. These controlled examples were not held out in advance; a previous verifier had scored one rotated example at 0.750.

Hole location. Moving a counterbored bolt hole along a plate leaves volume unchanged. At a 4 mm shift, the shape score remained 0.985, although a bolt could no longer pass through. The jig idea distinguishes space that must remain empty, or keep-out, from material that must exist, or keep-in. The moved hole's keep-out score fell to 0.3 and the combined score to 0.944, below the passing threshold. Shifts of 1.5 mm or less still passed because other jigs continued to match 1.

For me, this example matters more than a ranking. A part must fit its neighboring parts; looking similar is insufficient.

Figure 1. AI-generated concept illustration: a shape score of 0.985 can coexist with an unusable bolt interface.

3. How FrontierCAD scores a result

Part evaluation starts with rigid alignment: translation and rotation are allowed, reflection and scaling are not. The combined score is 0.4 × shape + 0.4 × jigs + 0.2 × topology. Shape averages volumetric intersection-over-union and surface F1. Topology compares solid bodies, through-holes and internal voids. Invalid solids score zero; a score of at least 0.97 passes 1[2].

The original report's scoring illustrations and interactive demonstrations.

Jiggen generates inspection volumes from the reference STEP using geometry algorithms rather than a language model. It places checks around locating and fastening regions such as holes, pins, slots and bosses. Bolt paths must remain empty; bosses must contain material. A jig is discarded when its intersection-over-union against the reference is below 0.98. The August 10 report describes 3,218 jigs across 39 parts, from 11 to 446 per part 1. This checks a candidate against a known reference; it is not autonomous inference of fits, fastening order or complete assembly intent.

Assembly evaluation pairs parts by shape, so swapping identical screws does not incur a penalty. File order and names do not determine correspondence. The assembly is globally aligned, leaving relative part locations to be assessed. Each part earns 1 / (1 + d). The technical report defines d using the larger of shape and pose discrepancies, with a translation or rotation unit of 0.1 mm or 6°, independent of part size. The sum of part scores is divided by the larger of the reference and candidate part counts, penalizing missing and extra parts. A score of 0.97 passes 1[2]. The overview and technical report describe combining translation and rotation differently; this article follows the technical report.

In an eight-part controlled assembly, moving the whole assembly changed the score by less than 2 × 10⁻¹⁰. Rotating the L-bracket by 12.8° produced a recorded score of 0.915. Rotating a hexagonal screw by 60° about its symmetry axis should not be penalized 1.

Here the AI generates a part or proposes assembly poses; geometric algorithms align, match and measure; the weights and passing threshold are benchmark rules. Separate shape and interface checks are useful, but they do not constitute a rule engine for product requirements, assembly process or design intent. Reference-derived jigs establish similarity to that reference, not suitability for your product.

Figure 2. AI-generated concept illustration of pose evaluation, showing the report's 0.1 mm / 6° normalization and the 12.8° example.

4. Reported results and their limits

The part passing threshold is 0.97. Values below use the technical report's three-decimal figures. The overview rounds the highest average to 0.63 and the best individual result to 0.96, with no model passing a part in this selected 39-part set 1[2].

• GPT-6 Astra — Harness / effort: codex · ultra; Mean score: 0.633; Scored trials: 39/39; Passes: 0/39
• Claude Fable 5.1 — Harness / effort: claude-code · max; Mean score: 0.582; Scored trials: 39/39; Passes: 0/39
• GLM 5.3 Flash — Harness / effort: opencode · max; Mean score: 0.333; Scored trials: 6/39; Passes: 0/6
• Gemini 3.8 Flash — Harness / effort: gemini-cli · high; Mean score: 0.332; Scored trials: 39/39; Passes: 0/39
• Muse Spark 1.3 — Harness / effort: muse-code · xhigh; Mean score: 0.332; Scored trials: 39/39; Passes: 0/39
• DeepSeek V4 Flash Vision — Harness / effort: opencode · max; Mean score: 0.305; Scored trials: 39/39; Passes: 0/39
• Grok 4.6 — Harness / effort: grok-build · xhigh; Mean score: 0.255; Scored trials: 39/39; Passes: 0/39

Astra had the highest assembly mean, about 0.71, and passed four of 16 assemblies. The report cautions that uncertainty is wide and small differences between models are not meaningful 1[2].

• GPT-6 Astra — Mean assembly score: 0.713; Scored trials: 16/16; Passes: 4/16
• Claude Fable 5.1 — Mean assembly score: 0.618; Scored trials: 16/16; Passes: 3/16
• Gemini 3.8 Flash — Mean assembly score: 0.371; Scored trials: 16/16; Passes: 2/16
• DeepSeek V4 Flash Vision — Mean assembly score: 0.339; Scored trials: 16/16; Passes: 2/16
• Grok 4.6 — Mean assembly score: 0.334; Scored trials: 16/16; Passes: 2/16
• Muse Spark 1.3 — Mean assembly score: 0.266; Scored trials: 15/16; Passes: 1/15
• GLM 5.3 Flash — Mean assembly score: 0.075; Scored trials: 16/16; Passes: 0/16

Assemblies contained 5–179 parts, with a median of 43.5. Part count had only a weak association with score: repeated bolts can be relatively easy to place. Mean scores were 0.44 for assemblies with at most 20 parts and 0.36 for those above 50 1.

Output limits prevented GLM 5.3 Flash from delivering CAD for 33 part trials. Its 0.333 mean covers six scored trials; assigning zero to the missing 33 yields 0.051. One Muse Spark 1.3 assembly ran out of memory; counting that as zero reduces its assembly mean from 0.266 to 0.250. Candidates rejected by the verifier already count as zero 1[2].

Task selection is a major qualification. The technical report says 59 tasks were removed after the scores were seen, including every part passed by any model. Zero part passes therefore partly follows from that selection; the scores are lower than on an unselected set. Eight of the retained drawings were incomplete. Some drawings were corrected on October 5, but the reported run was not repeated. Assembly scoring also remained provisional: exact B-Rep comparisons had not been performed; 11 of 14 passing trials clustered on two tasks, sometimes with scores equal to six or more decimal places, so exact coincidence of parts was not established 1[2].

Those findings do not justify handing over complex reconstruction or complete assembly release to AI. Equally, “none of 39 passed” should not be presented without the selection procedure.

5. What I take from the study

The measured scope: reconstruct STEP parts from drawings and position supplied parts from assembly drawings. Geometry jigs and relative poses are assessed. Full assembly relationships, mate constraints, joints, motion, feature history, parameters and design intent are outside this evaluation; some of those questions are closer to CAD Arena's scope 1[2].

Normal's findings: shape-only similarity can miss unusable interfaces; jig-based geometric checks add useful information; the selected complex parts remained difficult; assembly results were somewhat better but small, uncertain and provisional. Post-result task selection affects the apparently stark part pass rate.

My judgment: AI can propose a reconstruction or arrangement, but an engineer still needs to check mating faces, hole-and-shaft interfaces and locating shoulders. A rendering or shape score is not final approval. The study does not demonstrate a complete autonomous workflow that selects independent parts, establishes all assembly constraints and releases a validated assembly.

Related: why step-by-step verification matters for CAD agents.

Illustrations are conceptual, not screenshots from a new product test. The English edition uses separately localized English illustrations with matching captions.