Gemma 3 4B
Google · open weights · released 2025-03-12 · route gemma3:4b · Model profile · Chess results · Sudoku results · Mines results · FreeCell results · Starjar results · Nations Royale results · Wildhaven results · Hustle Sim results · Elemental Armory results · Micro Mouse results · Ravelin results · Minecraft results
Results
- Score
- 095%: 0 to 22 · 6 of 36 base puzzles so far
- Puzzles solved
- 0 of 6
- Cells that match the solution
- 0.0%
- Rows and columns whose clue it satisfies
- 0.0%
- Wrong cells / left cells unknown / no grid
- 0 / 0 / 6
- Near misses (three cells or fewer wrong)
- 0
- Average stated confidence
- 95%
- Average confidence when it had not solved it
- 95%
- Sure but wrong (90% or more, unsolved)
- 5
- Calibration (Brier, lower is better)
- 0.902
- API-equivalent cost per solved puzzle
- n/a
- Tokens per puzzle (in / out)
- 1,380 / 1,198
- Reasoning tokens per puzzle
- n/a
- Retries for an unreadable reply
- 6
- API-equivalent cost per puzzle
- n/a
- Seconds per puzzle
- 80.9
- Replies cut off at the length limit
- 0.0%
- Aborted on our side (not counted)
- 0
By tier
| Tier | Needs | Solved | Output tokens | Seconds |
|---|---|---|---|---|
| 5 by 5 | Line logic, a round or two | 0 of 1 | 106 | 17.6 |
| 10 by 10 | Line logic, about 3 rounds | 0 of 1 | 208 | 23.9 |
| 15 by 15 | Line logic, about 5 rounds | 0 of 1 | 2,685 | 82.6 |
| 20 by 20 | Line logic, about 8 rounds | 0 of 1 | 0 | 16.5 |
| 25 by 25 | Line logic, about 11 rounds | 0 of 1 | 3,354 | 312.8 |
| Beyond | Line logic stalls; needs contradiction or search | 0 of 1 | 832 | 31.7 |
Cells by the depth of logic they need
Each cell is tagged with the round of line logic that settles it. Beyond line logic means only contradiction or search does.
| Depth | Cells | Right |
|---|---|---|
| First pass | 403 | 0.0% |
| Rounds 2-3 | 613 | 0.0% |
| Rounds 4-7 | 332 | 0.0% |
| Round 8 on | 159 | 0.0% |
| Beyond line logic | 93 | 0.0% |
Every puzzle
5 by 5 #1No readable grid 0 of 25 cells95% sure · 106 tokens · local10 by 10 #1No readable grid 0 of 100 cells95% sure · 208 tokens · local15 by 15 #1No readable grid 0 of 225 cells95% sure · 2,685 tokens · local20 by 20 #1No readable grid 0 of 400 cells0 tokens · local25 by 25 #1No readable grid 0 of 625 cells95% sure · 3,354 tokens · localBeyond #1No readable grid 0 of 225 cells95% sure · 832 tokens · local