NONOGRAMS BENCHMARK · REASONING MEDIUM

Gemini 4 Hydrogen

Google · released 2026-11-12 · route gemini-4-hydrogen · Model profile · Chess results · Sudoku results · Mines results · FreeCell results · Starjar results · Nations Royale results · Wildhaven results · Hustle Sim results · Elemental Armory results · Micro Mouse results · Ravelin results

Results

Score
4795%: 36 to 58
Puzzles solved
17 of 36
Cells that match the solution
71.9%
Rows and columns whose clue it satisfies
60.5%
Wrong cells / left cells unknown / no grid
7 / 12 / 0
Near misses (three cells or fewer wrong)
4
Average stated confidence
66%
Average confidence when it had not solved it
38%
Sure but wrong (90% or more, unsolved)
7
Calibration (Brier, lower is better)
0.202
API-equivalent cost per solved puzzle
$1.14
Tokens per puzzle (in / out)
3,312 / 71,281
Reasoning tokens per puzzle
71,138
Retries for an unreadable reply
2
API-equivalent cost per puzzle
$0.540
Seconds per puzzle
291.2
Replies cut off at the length limit
0.0%
Aborted on our side (not counted)
0

By tier

TierNeedsSolvedOutput tokensSeconds
5 by 5Line logic, a round or two6 of 61,1418.8
10 by 10Line logic, about 3 rounds6 of 67,39834.1
15 by 15Line logic, about 5 rounds4 of 618,91683.0
20 by 20Line logic, about 8 rounds1 of 687,092364.1
25 by 25Line logic, about 11 rounds0 of 6206,194817.6
BeyondLine logic stalls; needs contradiction or search0 of 6106,945439.4

Cells by the depth of logic they need

Each cell is tagged with the round of line logic that settles it. Beyond line logic means only contradiction or search does.

DepthCellsRight
First pass2,46392.2%
Rounds 2-33,52086.1%
Rounds 4-71,61173.9%
Round 8 on1,01121.9%
Beyond line logic99519.0%

Every puzzle

5 by 5 #1Solved 25 of 25 cells100% sure · 910 tokens · $0.0115 by 5 #2Solved 25 of 25 cells100% sure · 1,019 tokens · $0.0125 by 5 #3Solved 25 of 25 cells100% sure · 1,116 tokens · $0.0135 by 5 #4Solved 25 of 25 cells100% sure · 927 tokens · $0.0115 by 5 #5Solved 25 of 25 cells100% sure · 990 tokens · $0.0125 by 5 #6Solved 25 of 25 cells100% sure · 1,882 tokens · $0.01810 by 10 #1Solved 100 of 100 cells100% sure · 10,099 tokens · $0.08010 by 10 #2Solved 100 of 100 cells100% sure · 8,029 tokens · $0.06510 by 10 #3Solved 100 of 100 cells100% sure · 6,678 tokens · $0.05410 by 10 #4Solved 100 of 100 cells100% sure · 4,632 tokens · $0.03910 by 10 #5Solved 100 of 100 cells100% sure · 7,124 tokens · $0.05810 by 10 #6Solved 100 of 100 cells100% sure · 7,824 tokens · $0.06315 by 15 #1Wrong cells 223 of 225 cells100% sure · 20,605 tokens · $0.15915 by 15 #2Solved 225 of 225 cells100% sure · 26,573 tokens · $0.20415 by 15 #3Solved 225 of 225 cells100% sure · 15,243 tokens · $0.11915 by 15 #4Solved 225 of 225 cells100% sure · 14,988 tokens · $0.11715 by 15 #5Wrong cells 223 of 225 cells100% sure · 15,933 tokens · $0.12415 by 15 #6Solved 225 of 225 cells50% sure · 20,151 tokens · $0.15620 by 20 #1Wrong cells 392 of 400 cells100% sure · 90,981 tokens · $0.68720 by 20 #2Wrong cells 396 of 400 cells100% sure · 51,142 tokens · $0.38820 by 20 #3Solved 400 of 400 cells100% sure · 103,086 tokens · $0.77820 by 20 #4Wrong cells 398 of 400 cells100% sure · 52,573 tokens · $0.39920 by 20 #5Left cells unknown 251 of 400 cells0% sure · 43,556 tokens · $0.33220 by 20 #6Wrong cells 396 of 400 cells100% sure · 181,213 tokens · $1.3725 by 25 #1Left cells unknown 533 of 625 cells0% sure · 469,653 tokens · $3.5325 by 25 #2Left cells unknown 538 of 625 cells10% sure · 197,557 tokens · $1.4925 by 25 #3Left cells unknown 333 of 625 cells0% sure · 177,114 tokens · $1.3325 by 25 #4Left cells unknown 0 of 625 cells0% sure · 85,161 tokens · $0.64425 by 25 #5Left cells unknown 308 of 625 cells0% sure · 185,102 tokens · $1.3925 by 25 #6Left cells unknown 443 of 625 cells5% sure · 122,576 tokens · $0.925Beyond #1Left cells unknown 62 of 225 cells0% sure · 34,681 tokens · $0.265Beyond #2Wrong cells 223 of 225 cells100% sure · 83,920 tokens · $0.634Beyond #3Left cells unknown 22 of 225 cells0% sure · 163,248 tokens · $1.23Beyond #4Left cells unknown 40 of 225 cells0% sure · 146,203 tokens · $1.10Beyond #5Left cells unknown 15 of 225 cells0% sure · 51,891 tokens · $0.394Beyond #6Left cells unknown 57 of 225 cells0% sure · 161,725 tokens · $1.22