57 fixtures. One answer, every time.
A geometry kernel must give the same body for the same input, and say so when it cannot. Our build of OpenCascade runs a fixture suite in fresh processes, and a gate refuses a regression and a lie: a success whose body is wrong.
The gate, as it ran
A fixture that passed must still pass.
The gate runs every fixture on OpenCascade and on the npm build of SMLib, three fresh processes each. The OpenCascade column is held to baseline.json. The local CI runs it as the job geometry-conformance whenever a kernel or the suite changes. When baseline.json was told F08 passes, the gate stopped:
F08 fail heat sink, fillet overlapping the hole by 0.5, hole first
F19 lie boolean identities on seeded pairs of boxes and cylinders
GATE: occt F08 passed in baseline.json and is now fail
exit 1Case study · the fixture suite
50 of 57 pass. The 7 others are named.
Each fixture is a body with a known answer: a closed-form volume, a boolean identity, a seam that must not move, a refusal that must be honest. OpenCascade 8.0.1, through our occt-wasm 5.7.1-commandagi.3, is the kernel the 3D service ships.
- pass on OpenCascade
- 50 of 57
- fail: F08 (a fillet over a hole), F15 (blade roots swept through a pitch), F35 (face history of a fillet and a move)
- 3
- refused: F09 (a rounded box shelled 2 mm inward), where the npm SMLib answers with a wrong body
- 1
- lie: F19 (boolean identities on seeded boxes and cylinders), recorded in
baseline.jsonas a known defect - 1
- not supported: F04 and F05, G2 and G3 blends
- 2
packages/applets/3d-design/conformance/src/fixtures.tsOutput case-studies/geometry-conformance/stats.jsonTool npm run full (src/run.ts --runs 100 --sweeps full --parallel 8)Ran on This computer (dell-gb10-1, 20 Arm cores, Node 24.22.1)When Case study · the same input, 100 times
One result in 100 processes. npm's SMLib gave 23.
Every fixture ran in 100 fresh processes on each kernel, the first in order and the others shuffled: 17,100 runs in 59 minutes. Each run's body is hashed. A kernel that gives two hashes for one input is not deterministic on it, whatever its status says.
The fork's patches seed every random choice of the kernel (packages/kernels/smlib-fork). The fork is not the shipped kernel: OpenCascade is. The run shared this computer with other work (load 20 to 60 on 20 cores), so its time is not a benchmark.
- result per fixture in 100 processes on OpenCascade, all 57 fixtures, 0 hangs
- 1
- result per fixture in 100 processes on our SMLib fork, all 57 fixtures
- 1
- results for F09 on the npm SMLib in the same 100 processes: 95 wrong bodies and 5 lies
- 23
- results for F09 on the npm SMLib on every one of 29 nights
- 21 to 24
- nights of 29 when our fork gave 2 results for one fixture (F19 on 10-21, F18 on 10-23 and 10-24); none since
- 3
Over 29 nightly runs from 2026-10-16 to 2026-11-13, the npm build of SMLib gave 24 distinct results for F09 on the first 7 nights, 21 on 2026-10-23, 23 on the next 14 nights and 21 on the last 7. Our SMLib fork gave 1 result for F09 every night.
The numbers
| Night | SMLib 0.3.0 (npm) | Our SMLib fork |
|---|---|---|
| 10-16 | 24 | 1 |
| 10-17 | 24 | 1 |
| 10-18 | 24 | 1 |
| 10-19 | 24 | 1 |
| 10-20 | 24 | 1 |
| 10-21 | 24 | 1 |
| 10-22 | 24 | 1 |
| 10-23 | 21 | 1 |
| 10-24 | 23 | 1 |
| 10-25 | 23 | 1 |
| 10-26 | 23 | 1 |
| 10-27 | 23 | 1 |
| 10-28 | 23 | 1 |
| 10-29 | 23 | 1 |
| 10-30 | 23 | 1 |
| 10-31 | 23 | 1 |
| 11-01 | 23 | 1 |
| 11-02 | 23 | 1 |
| 11-03 | 23 | 1 |
| 11-04 | 23 | 1 |
| 11-05 | 23 | 1 |
| 11-06 | 23 | 1 |
| 11-07 | 21 | 1 |
| 11-08 | 21 | 1 |
| 11-09 | 21 | 1 |
| 11-10 | 21 | 1 |
| 11-11 | 21 | 1 |
| 11-12 | 21 | 1 |
| 11-13 | 21 | 1 |
packages/applets/3d-design/conformance/results/{occt,smlib,smlib-fork}.jsonOutput case-studies/geometry-conformance/stats.jsonTool npm run full, and the nightly timer (make geometry-nightly)Ran on This computer (dell-gb10-1)When Hold a kernel to known answers.
The suite, the gate and the fork are in the repository. A kernel change that drops a passing fixture does not land.