NONOGRAMS BENCHMARK · REASONING MEDIUM

Gemini 4 Neon

Google · released 2026-11-18 · route gemini-4-neon · Model profile · Chess results · Sudoku results · Mines results · FreeCell results · Starjar results · Nations Royale results · Wildhaven results · Hustle Sim results · Elemental Armory results · Micro Mouse results · Ravelin results

Results

Score
9295%: 74 to 100 · 12 of 36 base puzzles so far
Puzzles solved
11 of 12
Cells that match the solution
99.9%
Rows and columns whose clue it satisfies
99.4%
Wrong cells / left cells unknown / no grid
1 / 0 / 0
Near misses (three cells or fewer wrong)
1
Average stated confidence
100%
Average confidence when it had not solved it
100%
Sure but wrong (90% or more, unsolved)
1
Calibration (Brier, lower is better)
0.083
API-equivalent cost per solved puzzle
$0.670
Tokens per puzzle (in / out)
2,596 / 40,424
Reasoning tokens per puzzle
40,299
Retries for an unreadable reply
0
API-equivalent cost per puzzle
$0.614
Seconds per puzzle
287.7
Replies cut off at the length limit
0.0%
Aborted on our side (not counted)
0

By tier

TierNeedsSolvedOutput tokensSeconds
5 by 5Line logic, a round or two2 of 293310.3
10 by 10Line logic, about 3 rounds2 of 27,04352.8
15 by 15Line logic, about 5 rounds2 of 216,916126.0
20 by 20Line logic, about 8 rounds2 of 240,967299.2
25 by 25Line logic, about 11 rounds1 of 2116,466820.4
BeyondLine logic stalls; needs contradiction or search2 of 260,220417.4

Cells by the depth of logic they need

Each cell is tagged with the round of line logic that settles it. Beyond line logic means only contradiction or search does.

DepthCellsRight
First pass84599.9%
Rounds 2-31,13699.9%
Rounds 4-7709100.0%
Round 8 on241100.0%
Beyond line logic269100.0%

Every puzzle