NONOGRAMS BENCHMARK · LOCAL, NO THINKING

Ministral 3 8B

Mistral · open weights · released 2025-12-02 · route ministral-3:8b-instruct-2512-q4_K_M · Model profile · Chess results · Sudoku results · Mines results · FreeCell results · Starjar results · Nations Royale results · Wildhaven results · Hustle Sim results · Elemental Armory results · Micro Mouse results · Ravelin results · Minecraft results

Results

Score
095%: 0 to 22 · 6 of 36 base puzzles so far
Puzzles solved
0 of 6
Cells that match the solution
8.6%
Rows and columns whose clue it satisfies
1.1%
Wrong cells / left cells unknown / no grid
2 / 0 / 4
Near misses (three cells or fewer wrong)
0
Average stated confidence
100%
Average confidence when it had not solved it
100%
Sure but wrong (90% or more, unsolved)
5
Calibration (Brier, lower is better)
0.992
API-equivalent cost per solved puzzle
n/a
Tokens per puzzle (in / out)
1,605 / 5,944
Reasoning tokens per puzzle
n/a
Retries for an unreadable reply
5
API-equivalent cost per puzzle
n/a
Seconds per puzzle
1,653.8
Replies cut off at the length limit
0.0%
Aborted on our side (not counted)
0

By tier

TierNeedsSolvedOutput tokensSeconds
5 by 5Line logic, a round or two0 of 140349.8
10 by 10Line logic, about 3 rounds0 of 1595105.9
15 by 15Line logic, about 5 rounds0 of 169365.9
20 by 20Line logic, about 8 rounds0 of 115,1286,212.6
25 by 25Line logic, about 11 rounds0 of 110,0852,712.5
BeyondLine logic stalls; needs contradiction or search0 of 18,757776.2

Cells by the depth of logic they need

Each cell is tagged with the round of line logic that settles it. Beyond line logic means only contradiction or search does.

DepthCellsRight
First pass4035.2%
Rounds 2-36135.1%
Rounds 4-73329.0%
Round 8 on1590.6%
Beyond line logic9358.1%

Every puzzle