Start with the engineering
An engineering brief, operating conditions and competing constraints. Vary the design and duty so the model has to reason about the physics.
Engineering Judgement Benchmark
Simulation-grounded tasks, engineering rubrics and verifiable rewards for teams training models and agents for the physical world. EJBench is our public benchmark for measuring that judgement.

Actual FluxCore simulation from a training design. The model reasons from numerical evidence; these images illustrate the underlying physics.
01 / EJBench model performance
Our public benchmark puts engineering judgement to the test. Same frozen tasks. Same scoring rules.
v0.1 · Updated 11 Nov 2026| Overall rank | Model / organisation | Overall score 0 — 100% | 95% CI | API cost / task |
|---|---|---|---|---|
| 01 | GPT-6 AstraOpenAI | 71.0% | 65.1–77.3% | $0.727 |
| 02 | GPT-6.1 SolOpenAI | 68.9% | 62.4–74.7% | $0.147 |
| 03 | Claude Opus 5.5Anthropic | 64.1% | 58.1–70.1% | $0.426 |
| 04 | GPT-6.1 TerraOpenAI | 58.8% | 52.7–65.7% | $0.036 |
| 05 | Muse Spark 1.3Meta | 55.0% | 49.2–61.4% | $0.186 |
| 06 | Claude Sonnet 5.5Anthropic | 51.8% | 46.0–57.9% | $0.559 |
| 07 | Mistral Large 4Mistral AI | 50.3% | 44.0–57.2% | $0.134 |
| 08 | DeepSeek V4.1 FlashDeepSeek | 44.9% | 38.4–51.8% | $0.052 |
| 09 | Claude Haiku 5.5Anthropic | 37.6% | 31.0–44.1% | $0.026 |
| 10 | GPT-6 LunaOpenAI | 36.3% | 30.0–42.9% | $0.007 |
| 11 | Qwen3.5-9B + GRPO on 25% of trainGradient Dynamics · fine-tuned baseline | 7.8% | 4.2–12.0% | — |
| 12 | Qwen3.5-9BQwen (open weights) | 5.9% | 2.7–9.7% | — |
Overall scores use equal weighting across task types. Each model ran once. Intervals reflect task sampling, not run-to-run variation; close scores should be compared using paired task results. API costs use list prices at the time of each run; unreported costs are shown as —.
The leaderboard uses a fixed subset of 228 tasks from the 272-task test set. All 12 models scored all 228 tasks. API models used high reasoning effort; Qwen baselines used thinking on at temperature 0.6. Reply limits did not truncate any response.
No external tools or new simulations are allowed. The benchmark harness can reveal stored simulation results where a task permits it. Every task scores 1 only when every required answer is correct; otherwise it scores 0. Confidence does not affect success.
gpt-6-astrahigh · 2026-11-11gpt-6.1-solhigh · 2026-11-10claude-opus-5-5high · 2026-11-10gpt-6.1-terrahigh · 2026-11-10muse-spark-1.3high · 2026-11-10claude-sonnet-5-5high · 2026-11-10mistral-large-4-0high · 2026-11-11deepseek-flashhigh · 2026-11-10claude-haiku-5-5high · 2026-11-10gpt-6-lunahigh · 2026-11-10qwen9b-grpo-25pctthinking on, temperature 0.6 · 2026-11-10qwen9bthinking on, temperature 0.6 · 2026-11-0902 / Engineering RL environments
For AI labs and teams building engineering agents, we develop tasks grounded in physical systems, with evidence to work from and a measurable definition of success.
An engineering brief, operating conditions and competing constraints. Vary the design and duty so the model has to reason about the physics.
Define the available evidence, permitted actions and tool budget. EJBench already supports multi-step decisions over stored simulation results.
Reset the task, capture the interaction and score the outcome consistently. Use the same checks to compare model versions and measure learning progress.
Our starting point is simulation-grounded tasks, stored-result environments and deterministic reward functions. Work with us to scope custom task distributions, tool integrations, solver access and delivery into your training stack.
03 / Rubrics, verifiers & rewards
A useful training signal starts with a clear standard. We translate engineering requirements into explicit criteria, then check decisions against simulation-derived reference results.
For custom programmes, agree the hard constraints, quality criteria and reward structure with our engineering team before training begins.
Success requires every mandatory check to pass.
Our FluxCore solver provides conjugate heat transfer simulations that connect task requirements to numerical reference results.
04 / For AI labs & engineering-agent teams
Bring us the engineering capability you want to train. We can scope the tasks, evidence, rubrics and feedback around your model and workflow.
Work with our team on an engineering environment for your training or evaluation programme, built around the physical systems and decisions that matter to your model.
Companies can also evaluate their models with us on EJBench. Compare under the benchmark protocol, inspect results and discuss failure cases with our team.
Your model appears on the public leaderboard only with your approval.
Dataset & harness accessAsk for the v0.1 release and evaluation harness. The release is licensed CC BY-NC 4.0; contact us about commercial use. Evaluation data must not be used for training.