NONOGRAMS BENCHMARK · REASONING MEDIUM

Claude Haiku 5.5

Anthropic · released 2026-10-07 · route claude-haiku-5-5 · Model profile · Chess results · Sudoku results · Mines results · FreeCell results · Starjar results · Nations Royale results · Wildhaven results · Hustle Sim results · Elemental Armory results · Micro Mouse results · Ravelin results · Minecraft results

Results

Score
5895%: 46 to 70
Puzzles solved
21 of 36
Cells that match the solution
51.2%
Rows and columns whose clue it satisfies
56.3%
Wrong cells / left cells unknown / no grid
4 / 6 / 5
Near misses (three cells or fewer wrong)
3
Average stated confidence
69%
Average confidence when it had not solved it
33%
Sure but wrong (90% or more, unsolved)
5
Calibration (Brier, lower is better)
0.127
API-equivalent cost per solved puzzle
$0.070
Tokens per puzzle (in / out)
2,257 / 80,666
Reasoning tokens per puzzle
80,366
Retries for an unreadable reply
12
API-equivalent cost per puzzle
$0.041
Seconds per puzzle
349.9
Replies cut off at the length limit
0.0%
Aborted on our side (not counted)
55

By tier

TierNeedsSolvedOutput tokensSeconds
5 by 5Line logic, a round or two6 of 68906.4
10 by 10Line logic, about 3 rounds4 of 68,06136.2
15 by 15Line logic, about 5 rounds4 of 637,823160.2
20 by 20Line logic, about 8 rounds5 of 6133,609564.0
25 by 25Line logic, about 11 rounds0 of 695,016416.2
BeyondLine logic stalls; needs contradiction or search2 of 6208,599916.7

Cells by the depth of logic they need

Each cell is tagged with the round of line logic that settles it. Beyond line logic means only contradiction or search does.

DepthCellsRight
First pass2,46376.2%
Rounds 2-33,52061.3%
Rounds 4-71,61133.3%
Round 8 on1,0116.6%
Beyond line logic99527.6%

Every puzzle

5 by 5 #1Solved 25 of 25 cells97% sure · 716 tokens · $0.00055 by 5 #2Solved 25 of 25 cells97% sure · 599 tokens · $0.00055 by 5 #3Solved 25 of 25 cells97% sure · 649 tokens · $0.00055 by 5 #4Solved 25 of 25 cells97% sure · 604 tokens · $0.00055 by 5 #5Solved 25 of 25 cells97% sure · 989 tokens · $0.00075 by 5 #6Solved 25 of 25 cells97% sure · 1,783 tokens · $0.001110 by 10 #1Solved 100 of 100 cells97% sure · 7,600 tokens · $0.004010 by 10 #2Wrong cells 97 of 100 cells97% sure · 8,867 tokens · $0.004710 by 10 #3Wrong cells 98 of 100 cells97% sure · 9,290 tokens · $0.004910 by 10 #4Solved 100 of 100 cells95% sure · 5,295 tokens · $0.002910 by 10 #5Solved 100 of 100 cells97% sure · 6,812 tokens · $0.003610 by 10 #6Solved 100 of 100 cells95% sure · 10,501 tokens · $0.005515 by 15 #1Solved 225 of 225 cells92% sure · 77,752 tokens · $0.03915 by 15 #2Wrong cells 224 of 225 cells95% sure · 54,618 tokens · $0.02815 by 15 #3No readable grid 0 of 225 cells93% sure · 25,424 tokens · $0.01315 by 15 #4Solved 225 of 225 cells92% sure · 23,928 tokens · $0.01215 by 15 #5Solved 225 of 225 cells95% sure · 24,269 tokens · $0.01215 by 15 #6Solved 225 of 225 cells95% sure · 20,945 tokens · $0.01120 by 20 #1Solved 400 of 400 cells90% sure · 104,545 tokens · $0.05320 by 20 #2Solved 400 of 400 cells88% sure · 120,217 tokens · $0.06020 by 20 #3No readable grid 0 of 400 cells92% sure · 236,987 tokens · $0.11920 by 20 #4Solved 400 of 400 cells95% sure · 60,525 tokens · $0.03120 by 20 #5Solved 400 of 400 cells92% sure · 179,141 tokens · $0.09020 by 20 #6Solved 400 of 400 cells90% sure · 100,238 tokens · $0.05025 by 25 #1No readable grid 0 of 625 cells2% sure · 9,674 tokens · $0.005625 by 25 #2Left cells unknown 113 of 625 cells2% sure · 161,012 tokens · $0.08225 by 25 #3Left cells unknown 39 of 625 cells1% sure · 136,116 tokens · $0.06925 by 25 #4No readable grid 0 of 625 cells1% sure · 148,826 tokens · $0.07525 by 25 #5Left cells unknown 18 of 625 cells1% sure · 107,416 tokens · $0.05425 by 25 #6Wrong cells 371 of 625 cells2% sure · 7,049 tokens · $0.0043Beyond #1Solved 225 of 225 cells95% sure · 71,231 tokens · $0.036Beyond #2Solved 225 of 225 cells90% sure · 251,171 tokens · $0.126Beyond #3Left cells unknown 9 of 225 cells2% sure · 149,777 tokens · $0.075Beyond #4Left cells unknown 28 of 225 cells2% sure · 200,913 tokens · $0.101Beyond #5Left cells unknown 14 of 225 cells3% sure · 176,828 tokens · $0.089Beyond #6No readable grid 0 of 225 cells3% sure · 401,675 tokens · $0.202