SUDOKU BENCHMARK · LOCAL, NO THINKING

Qwen3.8 27B

Alibaba · open weights · released 2026-08-14 · route qwen3.8:27b · Model profile · Chess results

Results

Score
095%: 0 to 16 · 14 of 42 grids so far
Grids solved
0 of 14
Blank cells filled correctly
29.7%
Wrong cells / empty cells / changed a given / no grid
3 / 0 / 11 / 0
Broke the rules (complete grid repeating a digit)
3
Near misses (three cells or fewer wrong)
0
Average stated confidence
72%
Average confidence when it had not solved it
72%
Sure but wrong (90% or more, unsolved)
7
Calibration (Brier, lower is better)
0.619
API-equivalent cost per grid
n/a
API-equivalent cost per solved grid
n/a
Seconds per grid
1,356
Tokens per grid (in / out)
527 / 11,808
Reasoning tokens per grid
n/a
Retries for an unreadable reply
2
Aborted on our side (not counted)
1

By tier

TierNeedsSolvedOutput tokensSeconds
CozySingles only0 of 215,6001,232
GentleScanning0 of 211,450542
SteadyLocked candidates0 of 210,994753
TrickyHidden pairs and triples0 of 210,5242,122
DeviousWings and fish0 of 211,3942,090
ExpertDeep wings and fish0 of 210,0671,703
BeyondNo human technique finishes it0 of 212,6231,051

Cells by the technique they needed

Each blank cell is tagged with the simplest technique that fills it, given everything a solver could fill before it.

TechniqueCellsFilled correctly
Last free cell12160.3%
Scanning25028.0%
Locked candidates8825.0%
Hidden pairs and triples10422.1%
Wings and fish6320.6%
Deep wings and fish5411.1%
Search8123.5%

Every answer