FreeCell benchmark
Classic numbered deals refereed with the rules of FreeCell on Crest Arcade, the same engine you can play. The tiers take free cells away. Compare deals won, how far lost games got, the moves the referee refused, and the cost and time of a game.
FreeCell leaderboard
Select a column to sort. Scroll sideways for all metrics →
| # | Model | Score | 95% interval | Won per tierEas4c3c2c1c | Cards home | Refused / game | Moves / reply | Plan | Games | API cost / game | Sec / game |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Solreasoning medium | 205 of 30 | 10000 | 15.4 | 5.005.1% of moves sent | 11.3 | 0 of 1 | 5 | $0.381 | 735.2 | |
| 2 | GPT-6 Astrareasoning medium | 205 of 30 | 10000 | 14.2 | 4.405.5% of moves sent | 10.6 | n/a | 5 | $1.25 | 659.4 | |
| 3 | GPT-6.1 Solreasoning medium | 205 of 30 | 01000 | 14.0 | 3.603.8% of moves sent | 11.9 | 0 of 1 | 5 | $0.267 | 523.5 | |
| 4 | GPT-6.1 Terrareasoning medium | 05 of 30 | 00000 | 8.4 | 6.406.7% of moves sent | 9.4 | 0 of 1 | 5 | $0.063 | 113.3 | |
| 5 | GPT-6 Lunareasoning medium | 05 of 30 | 00000 | 2.8 | 5.0022.3% of moves sent | 2.1 | 0 of 1 | 5 | $0.0089 | 85.5 |
No models match your search.
Score is the win rate averaged over the five tiers, each tier weighted the same, from 0 to 100. Cards home is how many of the 52 cards reached the foundations, averaged per tier. Plan is the one-reply variant, scored apart. A model ranks once it has played the first 6 games of every tier; until then it shows how many of those it has played and ranks after the models that have. Past that the bench keeps playing further games in the same order for every model, and every finished one counts, so n grows and the intervals narrow.
The Pareto frontier shows where no other model scores at least as well for less cost or time, with one strict improvement. Partial sets are included and may cover different items. Use completed sets for direct comparisons.
By tier
Deals won of games played, then the win rate with its 95% interval and the cards home on average. The count after each result is n, the games finished in that tier.
| Model | Easy | Four cells | Three cells | Two cells | Beyond |
|---|---|---|---|---|---|
| GPT-6 Solreasoning medium | 1 of 1100% (17 to 100) · 52.0 home | 0 of 10% (0 to 83) · 10.0 home | 0 of 10% (0 to 83) · 10.0 home | 0 of 10% (0 to 83) · 3.0 home | 0 of 10% (0 to 83) · 2.0 home |
| GPT-6 Astrareasoning medium | 1 of 1100% (17 to 100) · 52.0 home | 0 of 10% (0 to 83) · 11.0 home | 0 of 10% (0 to 83) · 4.0 home | 0 of 10% (0 to 83) · 4.0 home | 0 of 10% (0 to 83) · 0.0 home |
| GPT-6.1 Solreasoning medium | 0 of 10% (0 to 83) · 10.0 home | 1 of 1100% (17 to 100) · 52.0 home | 0 of 10% (0 to 83) · 3.0 home | 0 of 10% (0 to 83) · 4.0 home | 0 of 10% (0 to 83) · 1.0 home |
| GPT-6.1 Terrareasoning medium | 0 of 10% (0 to 83) · 25.0 home | 0 of 10% (0 to 83) · 8.0 home | 0 of 10% (0 to 83) · 3.0 home | 0 of 10% (0 to 83) · 5.0 home | 0 of 10% (0 to 83) · 1.0 home |
| GPT-6 Lunareasoning medium | 0 of 10% (0 to 83) · 3.0 home | 0 of 10% (0 to 83) · 4.0 home | 0 of 10% (0 to 83) · 7.0 home | 0 of 10% (0 to 83) · 0.0 home | 0 of 10% (0 to 83) · 0.0 home |
Every game
Every game
One square per deal, filled when it won. The one-reply plan games are on each model's page. Columns run tier by tier, in the order every model meets them. Click a square to open that one.
| Model | Eas | 2 | 3 | 4 | 5 | 6 | 4c | 2 | 3 | 4 | 5 | 6 | 3c | 2 | 3 | 4 | 5 | 6 | 2c | 2 | 3 | 4 | 5 | 6 | 1c | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Sol | ||||||||||||||||||||||||||||||
| GPT-6 Astra | ||||||||||||||||||||||||||||||
| GPT-6.1 Sol | ||||||||||||||||||||||||||||||
| GPT-6.1 Terra | ||||||||||||||||||||||||||||||
| GPT-6 Luna |
MethodologyProtocol, scoring & limitations
The deals
Classic numbered deals, dealt the way the original Windows game deals them, and refereed with the rules and messages of FreeCell on Crest Arcade. A solver checked every deal winnable with the free cells its tier allows. The tiers change the number of free cells, not the rules:
| Tier | Free cells | The deals |
|---|---|---|
| Easy | 4 | Deals winnable with one free cell, played with four |
| Four cells | 4 | Deals that need two free cells, played with four |
| Three cells | 3 | The same deals with three free cells |
| Two cells | 2 | The same deals with two: not one cell to spare |
| Beyond | 1 | The easy deals with a single free cell |
Four cells, Three cells and Two cells use the same deals, so only the space changes. Easy and Beyond share another pool. Every model meets the same deals in the same order from set v1. A model ranks once it has played the first 6 of every tier. After that the bench keeps playing further deals, up to 100 per tier, so n grows and the intervals narrow.
A game
Every request is fresh: the rules, the notation, the whole position as text, the current limit on how many cards can move together, how many replies are left, and what happened to the last reply. The model sends any number of moves in standard notation: columns 1 to 8, free cells a to d, h for the foundations. The referee plays them in order until one is not allowed, then says how many were played and why that one was refused, in one plain sentence. The refused move and everything after it are not played.
There are no automatic moves: what the model writes is exactly what happens. Once the rest of the game is only moves to the foundations, the referee finishes it and the game is won. A game is lost at 30 replies, after 8 replies in a row with no new card home, when no legal move is left, or on resignation. The one retry in a turn is for a reply with no moves tag.
Plan
The plan variant deals the Easy tier with one reply only: the whole solution, planned with no feedback. It separates planning from correcting course, which is what the main game mostly measures, and it is scored apart from the tiers.
Score
The score is the win rate in each tier, averaged so every tier counts the same, from 0 to 100, with a 95% interval from each tier's Agresti-Coull uncertainty. Cards home is partial credit: how far a lost game got, out of 52. Refused counts the moves the referee would not play.
Cost and time
Same as the chess and sudoku benchmarks: Claude models run through Claude Code and OpenAI models through the Codex CLI on a subscription, with every tool switched off, and local models on our own DGX Sparks. Cost is API-equivalent: what the tokens of every request in the game, retries included, would cost at the lab's list price. Local models have no list price, so they show n/a, not zero. Time is the wall-clock wait for the replies, including any queue on our side, so compare it within a route rather than across them. A request that fails on our side (a timeout, an outage, a usage limit) aborts the game, and an aborted game never counts against the model.