NONOGRAMS BENCHMARK · REASONING MEDIUM

Claude Sonnet 5.5

Anthropic · released 2026-09-28 · route claude-sonnet-5-5 · Model profile · Chess results · Sudoku results · Mines results · FreeCell results · Starjar results · Nations Royale results · Wildhaven results · Hustle Sim results · Elemental Armory results · Micro Mouse results · Ravelin results

Results

Score
4795%: 35 to 59
Puzzles solved
17 of 36
Cells that match the solution
48.6%
Rows and columns whose clue it satisfies
55.1%
Wrong cells / left cells unknown / no grid
9 / 5 / 5
Near misses (three cells or fewer wrong)
6
Average stated confidence
73%
Average confidence when it had not solved it
54%
Sure but wrong (90% or more, unsolved)
10
Calibration (Brier, lower is better)
0.260
API-equivalent cost per solved puzzle
$0.562
Tokens per puzzle (in / out)
1,657 / 25,863
Reasoning tokens per puzzle
25,414
Retries for an unreadable reply
7
API-equivalent cost per puzzle
$0.265
Seconds per puzzle
187.9
Replies cut off at the length limit
0.0%
Aborted on our side (not counted)
47

By tier

TierNeedsSolvedOutput tokensSeconds
5 by 5Line logic, a round or two6 of 66649.0
10 by 10Line logic, about 3 rounds2 of 63,50828.8
15 by 15Line logic, about 5 rounds5 of 610,90079.8
20 by 20Line logic, about 8 rounds2 of 650,596358.7
25 by 25Line logic, about 11 rounds0 of 62,00517.6
BeyondLine logic stalls; needs contradiction or search2 of 687,504633.4

Cells by the depth of logic they need

Each cell is tagged with the round of line logic that settles it. Beyond line logic means only contradiction or search does.

DepthCellsRight
First pass2,46366.4%
Rounds 2-33,52050.1%
Rounds 4-71,61125.7%
Round 8 on1,01112.7%
Beyond line logic99572.6%

Every puzzle

5 by 5 #1Solved 25 of 25 cells99% sure · 339 tokens · $0.00715 by 5 #2Solved 25 of 25 cells99% sure · 635 tokens · $0.0105 by 5 #3Solved 25 of 25 cells99% sure · 644 tokens · $0.0105 by 5 #4Solved 25 of 25 cells99% sure · 693 tokens · $0.0115 by 5 #5Solved 25 of 25 cells99% sure · 714 tokens · $0.0115 by 5 #6Solved 25 of 25 cells99% sure · 956 tokens · $0.01310 by 10 #1Solved 100 of 100 cells96% sure · 3,433 tokens · $0.03910 by 10 #2Wrong cells 98 of 100 cells97% sure · 6,099 tokens · $0.06510 by 10 #3Wrong cells 99 of 100 cells96% sure · 3,027 tokens · $0.03410 by 10 #4Solved 100 of 100 cells96% sure · 2,674 tokens · $0.03110 by 10 #5Wrong cells 98 of 100 cells96% sure · 2,384 tokens · $0.02810 by 10 #6Wrong cells 98 of 100 cells90% sure · 3,432 tokens · $0.03915 by 15 #1Solved 225 of 225 cells93% sure · 12,368 tokens · $0.12815 by 15 #2Solved 225 of 225 cells93% sure · 13,248 tokens · $0.13715 by 15 #3Solved 225 of 225 cells93% sure · 11,117 tokens · $0.11615 by 15 #4Solved 225 of 225 cells88% sure · 8,297 tokens · $0.08815 by 15 #5Wrong cells 212 of 225 cells5% sure · 10,273 tokens · $0.10815 by 15 #6Solved 225 of 225 cells93% sure · 10,100 tokens · $0.10620 by 20 #1Solved 400 of 400 cells96% sure · 106,685 tokens · $1.0720 by 20 #2Wrong cells 396 of 400 cells96% sure · 76,579 tokens · $0.77120 by 20 #3Left cells unknown 27 of 400 cells1% sure · 1,179 tokens · $0.01720 by 20 #4No readable grid 0 of 400 cells60% sure · 43,824 tokens · $0.45120 by 20 #5No readable grid 0 of 400 cells90% sure · 44,903 tokens · $0.46220 by 20 #6Solved 400 of 400 cells88% sure · 30,404 tokens · $0.31025 by 25 #1Left cells unknown 44 of 625 cells1% sure · 1,094 tokens · $0.01725 by 25 #2Left cells unknown 70 of 625 cells2% sure · 2,169 tokens · $0.02825 by 25 #3Left cells unknown 22 of 625 cells1% sure · 2,180 tokens · $0.03625 by 25 #4Left cells unknown 40 of 625 cells1% sure · 964 tokens · $0.01625 by 25 #5Wrong cells 289 of 625 cells1% sure · 2,176 tokens · $0.03625 by 25 #6No readable grid 0 of 625 cells1% sure · 3,448 tokens · $0.050Beyond #1No readable grid 0 of 225 cells90% sure · 37,499 tokens · $0.386Beyond #2No readable grid 0 of 225 cells96% sure · 198,854 tokens · $2.01Beyond #3Wrong cells 223 of 225 cells97% sure · 118,474 tokens · $1.19Beyond #4Solved 225 of 225 cells97% sure · 50,139 tokens · $0.507Beyond #5Wrong cells 223 of 225 cells97% sure · 46,744 tokens · $0.473Beyond #6Solved 225 of 225 cells96% sure · 73,314 tokens · $0.738