SUDOKU BENCHMARK · LOCAL, THINKING LOW

gpt-oss 20B

Openai · open weights · released 2025-08-05 · route gpt-oss:20b · Model profile · Chess results

Results

Score
2995%: 8 to 49 · 7 of 42 grids so far
Grids solved
2 of 7
Blank cells filled correctly
53.8%
Wrong cells / empty cells / changed a given / no grid
0 / 0 / 3 / 2
Broke the rules (complete grid repeating a digit)
0
Near misses (three cells or fewer wrong)
0
Average stated confidence
80%
Average confidence when it had not solved it
67%
Sure but wrong (90% or more, unsolved)
2
Calibration (Brier, lower is better)
0.400
API-equivalent cost per grid
n/a
API-equivalent cost per solved grid
n/a
Seconds per grid
3,050
Tokens per grid (in / out)
445 / 29,731
Reasoning tokens per grid
n/a
Retries for an unreadable reply
2
Aborted on our side (not counted)
110

By tier

TierNeedsSolvedOutput tokensSeconds
CozySingles only0 of 18,212753
GentleScanning1 of 118,046653
SteadyLocked candidates1 of 116,6613,346
TrickyHidden pairs and triples0 of 164,8415,313
DeviousWings and fish0 of 115,9413,445
ExpertDeep wings and fish0 of 164,6155,250
BeyondNo human technique finishes it0 of 119,8002,587

Cells by the technique they needed

Each blank cell is tagged with the simplest technique that fills it, given everything a solver could fill before it.

TechniqueCellsFilled correctly
Last free cell6093.3%
Scanning12478.2%
Locked candidates37100.0%
Wings and fish2070.0%
Search3312.1%

Every answer