BenchOne by MPG ONE
The same 12 original business tasks for every model. Every answer graded by script, every run priced and timed. No tools, no retries, first answer only.
Leaderboard
Higher is better, further left is cheaper or faster. Switch what the chart compares, rank by one category, or spotlight one maker. Hover or tap a dot for its numbers, click a row to highlight it, and open any row with + to see all 12 task scores. Vertical lines show how much a model moved between its runs.
Hollow dots ran once; filled dots are the average of three runs. When dots crowd together some names hide; hover or tap a dot to see it. Swipe sideways to see the whole chart.
| # | Model | Score | Cost per run | Time per run | First word | Output tokens |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 Anthropic · 3 runs · medium effort | 96.6 ±0.0 | $1.91 | 9.3 min | 29 s | 71k |
| 1 | Claude Fable 5.1 Anthropic · 3 runs · medium effort | 96.6 ±0.0 | $4.82 | 12 min | 34 s | 72k |
| 3 | GPT-6.1 Sol OpenAI · 3 runs · medium effort | 95.4 ±0.6 | $0.53 | 8.5 min | 24 s | 37k |
| 4 | Grok 4.7 xAI · 1 run · default effort | 95.3 | $2.17 | 1 h 15 min | 3.5 min | 329k |
| 5 | Claude Sonnet 5.5 Anthropic · 3 runs · medium effort | 94.0 ±1.4 | $0.81 | 5.9 min | 21 s | 56k |
| 6 | DeepSeek V4.1 Flash DeepSeek · 3 runs · default effort | 93.6 ±4.5 | $0.17 | 30 min | 1.8 min | 286k |
| 7 | Claude Haiku 5.5 Anthropic · 3 runs · medium effort | 92.4 ±2.8 | $0.10 | 10 min | 31 s | 165k |
| 8 | Gemini 4 Argon Google · 3 runs · medium effort | 91.7 ±1.6 | $0.71 | 9.7 min | 8 s | 54k |
| 9 | Gemini 3.8 Flash Google · 1 run · medium effort | 91.1 | $1.05 | 23 min | 54 s | 263k |
| 10 | Mistral Large 4 Mistral · 3 runs · medium effort | 89.8 ±2.7 | $0.59 | 25 min | 1.5 min | 256k |
| 11 | Claude Sonnet 5 Anthropic · 3 runs · medium effort | 88.0 ±2.5 | $2.60 | 34 min | 1.9 min | 235k |
| 12 | DeepSeek V4 Pro DeepSeek · 3 runs · default effort | 86.5 ±0.4 | $1.54 | 1 h 47 min | 7.4 min | 361k |
| 13 | Kimi K3.1 Pro Moonshot AI · 3 runs · medium effort | 82.4 ±5.1 | $0.75 | 31 min | 1.3 min | 59k |
| 14 | GLM 5.3 Flash Z.ai · 3 runs · default effort | 80.1 ±6.1 | $0.17 | 1 h 27 min | 5.5 min | 321k |
| 15 | GLM 5.3 Z.ai · 3 runs · default effort | 80.0 ±12.6 | $1.81 | 39 min | 2.5 min | 385k |
| 16 | GPT-6 Luna OpenAI · 3 runs · medium effort | 78.7 ±4.3 | $0.06 | 15 min | 37 s | 93k |
| 17 | Kimi K2.7 Code Moonshot AI · 3 runs · default effort | 78.1 ±5.2 | $1.18 | 1 h 44 min | 5.3 min | 277k |
| 18 | Claude Opus 5 Anthropic · 3 runs · medium effort | 76.1 ±4.5 | $2.60 | 15 min | 49 s | 79k |
| 19 | Kimi K2.6 Moonshot AI · 3 runs · default effort | 71.5 ±8.5 | $1.65 | 1 h 14 min | 3.9 min | 394k |
| 20 | gpt-oss-120b OpenAI · 3 runs · default effort | 67.0 ±4.6 | $0.14 | 22 min | 1.5 min | 155k |
| 21 | Mistral Small 3.1 Mistral · 3 runs · default effort | 34.9 ±3.9 | $0.04 | 7.1 min | 1 s | 19k |
| 22 | Llama 4 Scout Meta · 3 runs · default effort | 28.7 ±5.4 | $0.03 | 5.3 min | 1 s | 14k |
Click a column title to sort. First word is the median wait before a model starts writing its answer. Output tokens include thinking. Model names link to our head to head tests where we have one. On small screens the last two columns are hidden.
What the results show
Claude Opus 5.5 and Claude Fable 5.1 got every task right in all three runs except the questions about events from July to October 2026, which every model honestly answered with UNKNOWN. 96.6 is the ceiling for any model without that knowledge.
Opus 5.5 matches Fable 5.1 for $1.91 a run instead of $4.82. GPT-6.1 Sol lands one point lower for $0.53, and Claude Sonnet 5.5 scores 94.0 in under six minutes a run, the fastest of any model above 90.
Claude Haiku 5.5 scores 92.4 for nine cents a run, and DeepSeek V4.1 Flash scores 93.6 for 17 cents. DeepSeek pays for it in time: 30 minutes a run, and once it thought for 64,000 tokens without answering.
Fable 5.1 gave the same score in all three runs and Opus 5.5 stayed within 0.1 points. GLM 5.3 swung from 69.6 to 94.7 between identical runs, and Kimi K2.6 from 61 to 78.
A 435 row subscription log decided more ranks than any other task. Ten of the 22 models averaged under 50 on it and five scored 0 in every run, while Opus 5.5, Fable 5.1, GPT-6.1 Sol and Grok 4.7 scored 100 every time.
Kimi K2.6, Kimi K2.7 Code, GLM 5.3, GLM 5.3 Flash, DeepSeek V4 Pro and Claude Sonnet 5 sometimes thought until they hit the 64,000 token limit without answering, most often on the subscription log. Claude Opus 5's API safety filter blocked most of its long document answers as cyber content, which left it at 0 on Long context. Opus 5.5 answered every task.
By category
Average score per category across all runs. The redder the cell, the more points the model dropped there. Facts and honesty tops out at 80 for every model tested so far because of the questions about recent events explained in the method. Click a category title to rank the board by it.
| Model | ||||||
|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 100 | 100 | 100 | 80 | 100 | 100 |
| Claude Fable 5.1 | 100 | 100 | 100 | 80 | 100 | 100 |
| GPT-6.1 Sol | 100 | 99 | 97 | 80 | 100 | 98 |
| Grok 4.7 | 100 | 100 | 97 | 76 | 100 | 100 |
| Claude Sonnet 5.5 | 96 | 98 | 95 | 78 | 97 | 100 |
| DeepSeek V4.1 Flash | 100 | 83 | 99 | 80 | 100 | 100 |
| Claude Haiku 5.5 | 95 | 90 | 93 | 78 | 98 | 100 |
| Gemini 4 Argon | 98 | 86 | 94 | 77 | 96 | 100 |
| Gemini 3.8 Flash | 99 | 79 | 100 | 80 | 100 | 90 |
| Mistral Large 4 | 100 | 69 | 92 | 78 | 100 | 100 |
| Claude Sonnet 5 | 97 | 65 | 95 | 76 | 100 | 95 |
| DeepSeek V4 Pro | 100 | 82 | 99 | 72 | 100 | 67 |
| Kimi K3.1 Pro | 78 | 64 | 93 | 80 | 97 | 83 |
| GLM 5.3 Flash | 88 | 50 | 88 | 73 | 99 | 83 |
| GLM 5.3 | 95 | 67 | 75 | 77 | 99 | 67 |
| GPT-6 Luna | 75 | 57 | 77 | 80 | 100 | 83 |
| Kimi K2.7 Code | 83 | 44 | 82 | 77 | 100 | 83 |
| Claude Opus 5 | 83 | 99 | 0 | 76 | 99 | 100 |
| Kimi K2.6 | 80 | 33 | 73 | 77 | 99 | 67 |
| gpt-oss-120b | 80 | 33 | 57 | 63 | 97 | 72 |
| Mistral Small 3.1 | 8 | 8 | 29 | 78 | 60 | 26 |
| Llama 4 Scout | 1 | 10 | 27 | 45 | 62 | 27 |
The tasks
Two tasks in each of six categories. Prompts and answer keys stay private; this is what each task tests.
A generated Python program that runs 45 warehouse operations through FIFO stock layers, bulk and weekend discounts, returns and backorders. Without running it, the model must write the program's exact output: 9 lines, each checked character for character.
Three small tables (customers, orders, refunds) and five SQLite queries with inner and left joins, NULLs, NOT IN, integer division, HAVING, window functions and ROUND. The model writes the exact result rows of each query; the key comes from running SQLite.
About 400 rows of raw subscription events for 150 customers: starts, plan changes, per seat Team plans, pauses and resumes, cancels, voided rows and an annual plan. The model computes MRR, expansion, churn and net revenue retention by hand: 12 exact answers.
Business days with a holiday calendar, weekend observance and a collision rule; meeting times across daylight saving changes in six zones including half and three quarter hour offsets; and a 60 month loan schedule rounded to the cent: 19 exact answers.
A nine month archive of about 170 internal memos (roughly 60k tokens). Eight facts change over time through unapproved proposals, corrections, cancelled plans and lookalikes; six more questions need exact counting across the whole archive (memos per sender, mentions, weekdays).
Twenty vendor contracts in legal prose, signed amendments that change rates, caps and end dates part way through, two unsigned drafts, and about 200 invoice emails. The model applies rates, monthly caps, contract periods and duplicate rules: 12 exact answers.
Twenty four short questions: eight long tail facts, eight events from July to October 2026 (after most training cutoffs) and eight about people, works or things that do not exist. A right answer earns a point, UNKNOWN earns nothing, a wrong or invented answer loses a point.
Thirteen requests for the exact URL of a specific documentation page, standard or news page, including three pages that do not exist. The grader loads every URL: a real page earns a point, NONE earns nothing, and a broken or invented URL loses a point.
A 200 word blog post for a small bakery under 17 rules at once: an acrostic, a paragraph of exactly 52 words, a paragraph without the letter e, sentences that must grow in length, exact word counts, banned words, no digits and no dashes. 23 checks, all by script.
About 40 messy finance emails: mixed date and number formats, four currencies, VAT exclusive totals, amounts written in words, two invoices in one email, credit notes, corrections, forwarded old copies and payment receipts that are not invoices. About 180 fields checked one by one.
A freshly generated logic grid puzzle: eight houses, six attributes, a minimal set of clues with exactly one solution (checked by a solver). 48 cells graded.
Twenty candidate projects with cost, value and design and engineering weeks, a budget, two team capacities, dependencies, conflicts and a must have rule. A valid plan earns 40 points; the full 100 needs the single best plan, proved by checking all 1,048,576 combinations.
Method
The honesty tasks include eight questions and one source request about events from July to October 2026. Every model tested so far answered UNKNOWN to those eight questions instead of guessing, which costs no points but earns none. A model that gets everything else right therefore scores 96.6. A model trained on those months could go higher.
BenchOne tests single answers to well defined tasks. It does not test long agent sessions with tools, image or video work, or chat style. Results are for the effort setting shown; higher effort settings cost more and can score differently.
Before this version we ran three pilots. Four earlier tasks were retired because every model scored 100 on them, and one task was reworded after a label in it was ambiguous.
Changelog
BenchOne results are published under the Creative Commons Attribution 4.0 license. You may quote, chart and republish them, as long as you credit MPG ONE and link to this page.
Source: BenchOne by MPG ONE, https://mpgone.com/benchone/ (v2.2, November 2026)
FAQ
BenchOne is MPG ONE's own benchmark for AI models. Every model gets the same 12 original business tasks (code, data and math, long documents, facts and honesty, writing rules and reasoning), graded by script, with the real cost and time of every run recorded.
Most leaderboards report a score. BenchOne reports the score next to what the same work cost and how long it took, uses only the first answer with no tools or retries, and uses tasks written for business work rather than exam questions.
Two of the 12 tasks ask about events from July to October 2026. Models whose training data ends earlier lose those points but keep their honesty by answering UNKNOWN. A model that got everything else right lands on 96.6, which is the ceiling for any model without that knowledge.
On BenchOne, Claude Opus 5.5 and Claude Fable 5.1 share the top score, and Opus 5.5 gets there for $1.91 a run instead of $4.82. GPT-6.1 Sol is one point lower for $0.53, and Claude Sonnet 5.5 scores 94.0 in under six minutes a run. Claude Haiku 5.5 scores 92.4 for about nine cents a run, so it is the pick when cost matters most.
We also tested Qwen 3.8 27B, Nemotron 3 120B, gpt-oss-20b and Gemma 4 26B on Cloudflare Workers AI, but the host stops a request after 2 to 10 minutes, before these models finished most of the longer tasks. Ranking them would measure the host instead of the model, so they are left off until we can run them without that limit.
New models are added on the same 12 tasks when they launch, so their scores compare directly. A new version of the task set is published only when the current one stops separating models, and every change is listed in the changelog.