NONOGRAMS BENCHMARK · REASONING MEDIUM

GPT-6 Astra

Openai · released 2026-09-03 · route gpt-6-astra · Model profile · Chess results · Sudoku results · Mines results · FreeCell results · Starjar results · Nations Royale results · Wildhaven results · Hustle Sim results · Elemental Armory results · Micro Mouse results

Results

Score
6795%: 45 to 88 · 6 of 36 base puzzles so far
Puzzles solved
4 of 6
Cells that match the solution
60.2%
Rows and columns whose clue it satisfies
56.7%
Wrong cells / left cells unknown / no grid
0 / 2 / 0
Near misses (three cells or fewer wrong)
0
Average stated confidence
67%
Average confidence when it had not solved it
0%
Sure but wrong (90% or more, unsolved)
0
Calibration (Brier, lower is better)
0.000
API-equivalent cost per solved puzzle
$0.527
Tokens per puzzle (in / out)
21,674 / 5,058
Reasoning tokens per puzzle
4,405
Retries for an unreadable reply
0
API-equivalent cost per puzzle
$0.351
Seconds per puzzle
196.0
Replies cut off at the length limit
0.0%
Aborted on our side (not counted)
0

By tier

TierNeedsSolvedOutput tokensSeconds
5 by 5Line logic, a round or two1 of 111710.7
10 by 10Line logic, about 3 rounds1 of 12,02789.1
15 by 15Line logic, about 5 rounds1 of 14,730164.9
20 by 20Line logic, about 8 rounds1 of 114,017560.9
25 by 25Line logic, about 11 rounds0 of 16,166219.9
BeyondLine logic stalls; needs contradiction or search0 of 13,294130.7

Cells by the depth of logic they need

Each cell is tagged with the round of line logic that settles it. Beyond line logic means only contradiction or search does.

DepthCellsRight
First pass40391.8%
Rounds 2-361367.2%
Rounds 4-733254.5%
Round 8 on1590.0%
Beyond line logic930.0%

Every puzzle