Hustle Sim benchmark
Hustle Sim, the idle game about side hustles, played one short list of purchases at a time with the clock paused while the model thinks. The bench runs the game's own economy, so every price and payout is the one you play against. Compare how fast each model grows a run to its goal against the game's own balance strategies, the purchases the game refused, and the cost and time of a decision. Hustle Sim is made with Chach16 and Oliver.
Hustle Sim leaderboard
Select a column to sort. Scroll sideways for all metrics →
| # | Model | Score | 95% interval | Score per tierNewTkoEmp | In time | Slower | Beat the best | Refused / decision | Decisions / run | Runs | API cost / run | Sec / decision |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astrareasoning medium | 653 of 6 | 855952 | 3 of 3 | 1.6x | 0 of 3 | 0.000.0% of actions | 21.0 | 3 | $1.65 | 17.0 | |
| 2 | GPT-6 Solreasoning medium | 623 of 6 | 884554 | 3 of 3 | 1.7x | 0 of 3 | 0.000.0% of actions | 20.7 | 3 | $0.328 | 20.9 | |
| 3 | GPT-6.1 Terrareasoning medium | 423 of 6 | 317818 | 3 of 3 | 2.8x | 0 of 3 | 0.000.0% of actions | 19.7 | 3 | $0.101 | 4.9 | |
| 4 | GPT-6.1 Solreasoning medium | 353 of 6 | 79196 | 3 of 3 | 4.7x | 0 of 3 | 0.000.0% of actions | 11.0 | 3 | $0.228 | 15.4 | |
| 5 | GPT-6 Lunareasoning medium | 293 of 6 | 60271 | 3 of 3 | 7.7x | 0 of 3 | 0.113.9% of actions | 12.7 | 3 | $0.011 | 6.5 |
No models match your search.
Score is the best game strategy's time to the goal over the model's, averaged per tier with every tier weighted the same. Slower is the model's game time over the best line's, as a geometric mean over its runs. A model ranks once it has played the first 2 runs of every tier; until then it shows how many of those it has played and ranks after the models that have. Past that the bench keeps playing further runs in the same order for every model, and every finished one counts, so n grows and the intervals narrow.
The Pareto frontier shows where no other model scores at least as well for less cost or time, with one strict improvement. Partial sets are included and may cover different items. Use completed sets for direct comparisons.
By tier
Score on that tier's runs, then how many reached the goal before the replies ran out, and its median game time against the best line's. The count after each result is n, the runs finished in that tier.
| Model | Fresh start | Takeover | Empire |
|---|---|---|---|
| GPT-6 Astrareasoning medium | 85n 1 · 1 in time · median 20m 38s vs 17m 30s | 59n 1 · 1 in time · median 14h 20m vs 8h 24m | 52n 1 · 1 in time · median 11d 8h vs 5d 20h |
| GPT-6 Solreasoning medium | 88n 1 · 1 in time · median 19m 59s vs 17m 30s | 45n 1 · 1 in time · median 18h 38m vs 8h 24m | 54n 1 · 1 in time · median 10d 20h vs 5d 20h |
| GPT-6.1 Terrareasoning medium | 31n 1 · 1 in time · median 55m 40s vs 17m 30s | 78n 1 · 1 in time · median 10h 48m vs 8h 24m | 18n 1 · 1 in time · median 32d 15h vs 5d 20h |
| GPT-6.1 Solreasoning medium | 79n 1 · 1 in time · median 22m 03s vs 17m 30s | 19n 1 · 1 in time · median 1d 19h vs 8h 24m | 6n 1 · 1 in time · median 90d 2h vs 5d 20h |
| GPT-6 Lunareasoning medium | 60n 1 · 1 in time · median 29m 25s vs 17m 30s | 27n 1 · 1 in time · median 1d 7h vs 8h 24m | 1n 1 · 1 in time · median 1.2 years vs 5d 20h |
The game's own strategies on the same runs
The ten strategies the game's balance simulator ships, each played from the same starts to the same goals under the same paused-time protocol and scored the same way. The best of the seven deliberate ones on a run is the bar; the three unguided players are the floor.
| Strategy | Score | Fresh start | Takeover | Empire |
|---|---|---|---|---|
| Best income per dollarAlways buys whatever raises income most per dollar spent. The optimiser's answer. | 100 | 100median 17m 30s | 100median 8h 24m | 100median 5d 14h |
| Deepen firstDrives the newest hustle to the level-25 speed milestone before looking for another. | 94 | 88median 19m 49s | 93median 9h 07m | 100median 5d 14h |
| Cheapest thing availableBuys the lowest-priced option on the board. The default behaviour of someone not optimising. | 71 | 59median 29m 31s | 70median 12h 03m | 84median 6d 16h |
| Chase speed milestonesPushes its best earner to levels 25, 50 and 100 ahead of anything else. | 24 | 39median 44m 28s | 16median 4d 2h | 16median 47d 14h |
| Perks before levelsBuys every affordable perk before spending on levels. | 3 | 0median 3d 23h | 1median 62d 7h | 8median 73d 1h |
| Automate everythingManagers and boosts on every hustle before any levels. | 2 | 2median 13h 55m | 2median 14d 10h | 0median 3.3 years |
| Collect firstBrainstorms the moment it can afford to, upgrading only enough to pay for the next one. | 1 | 2median 17h 15m | 2median 17d 13h | 0median 4.1 years |
| Unguided (seed 3)Third random control. (control) | 1 | 2median 12h 31m | 0median 65d 1h | 1median 1.6 years |
| Unguided (seed 1)Picks at random among affordable options. The control: strategy should beat this. (control) | 1 | 2median 18h 31m | 1median 57d 12h | 1median 1.3 years |
| Unguided (seed 2)Second random control. (control) | 1 | 1median 1d 15h | 1median 28d 15h | 1median 2.6 years |
Every run
Every run
One square per run, filled when it reached the goal within twice the best line's game time (a score of 50 or more). Columns run tier by tier, in the order every model meets them. Click a square to open that one.
| Model | New | 2 | Tko | 2 | Emp | 2 |
|---|---|---|---|---|---|---|
| GPT-6 Astra | ||||||
| GPT-6 Sol | ||||||
| GPT-6.1 Terra | ||||||
| GPT-6.1 Sol | ||||||
| GPT-6 Luna |
MethodologyProtocol, scoring & limitations
The runs
Hustle Sim is an idle game about side hustles. You start with the $100 your parents gave you, pay a rising fee to brainstorm a new hustle, and grow each one with levels, a helper who runs it for you, a boost and perks. Hustle Sim is made with Chach16 and Oliver. The bench runs the game's own economy, pinned to commit a295bad, through its own balance simulator, so a later change to the game cannot move anyone's result. Every run stores its actions in order, and the same seed, commit and actions give the same run.
| Tier | Start | Goal | Replies | Best line's time | Best line's replies |
|---|---|---|---|---|---|
| Fresh start | A new run: the $100 gift, no hustles | $1M | 30 | 17m 30s | 5 |
| Takeover | An unguided player's run at its first $1M | $1B | 60 | 8h 24m | 22 |
| Empire | An unguided player's run at its first $1B | $1T | 80 | 5d 14h | 44 |
Each tier is a goal in lifetime earnings. Fresh start is a new run. Takeover and Empire hand the model a run that the game's own unguided player (its random strategy, under the run's seed) has taken to its first $1M or $1B, so each start is a different, untidy board. Every model plays the same runs in the same order from set v1. A model ranks once it has played the first 2 of every tier. After that the bench keeps playing further runs, up to 60 per tier, so n grows and the intervals narrow. Fresh start runs differ only in the shuffle of the risky hustles' flops and jackpots, so they are close to the same run each time.
A decision
Play is in paused time, the way the game's balance simulator plays. Every decision is a fresh request with the whole board: the game clock, cash, lifetime earnings against the goal, income, and every hustle owned with its level, pay per job, seconds per job, helper and boost, what its next level, ten levels and next speed milestone cost, the perks it can buy and the brainstorm fee. Each price says how long current income takes to pay it. A brainstorm's ideas show only their name and pitch, as on the game's own idea cards.
The model answers with up to 20 actions, run in order: LEVEL (1 to 100 levels, to the next speed milestone, or as many as cash buys), HELPER, BOOST, PERK, BRAINSTORM, PICK an idea, and WAIT. A purchase it cannot afford yet moves the clock exactly as long as its income needs to pay for it, which is how the simulator moves between purchases. The clock never runs while it thinks. Before the first helper, every hustle is worked by hand without a break; after it, only hustles with helpers earn, as in the simulator. A purchase the game refuses is not retried: it is counted and reported back on the next decision. The one retry is for a reply with no readable actions tag; two decisions in a row with none forfeit the run.
Each tier allows a fixed number of replies. When they run out, or a run is forfeited, the clock keeps running with what the model owns and no more purchases until the goal is reached, so its economy still decides the time. Only helpers earn then, so a run with none never gets there.
Score
A run scores the best game strategy's time to the goal from the same start, divided by the model's. As fast as the game's best line is 1, twice as slow is 0.5, faster is above 1, and a run that would never get there is 0. The score is the average per tier, every tier weighted the same, times 100. Its 95% interval adds up each tier's Agresti-Coull-adjusted uncertainty. Slower is the model's time over the best line's, a geometric mean over its runs.
The game's strategies
The game's balance simulator ships ten strategies: best income per dollar, collect first, deepen first, cheapest thing available, automate everything, perks before levels, chase speed milestones, and three unguided players that pick at random. Each played every run of the set from the same start to the same goal through the same driver. The best of the seven deliberate ones on a run is its bar; the unguided players are the floor. The strategies read the catalog's numbers when they pick an idea, which a player and the model cannot.
Cost and time
Same as the chess and sudoku benchmarks: Claude models run through Claude Code and OpenAI models through the Codex CLI on a subscription, with every tool switched off, and local models on our own DGX Sparks. Cost is API-equivalent: what the tokens of every request in the run, retries included, would cost at the lab's list price. Local models have no list price, so they show n/a, not zero. Time is the wall-clock wait for the replies, including any queue on our side, so compare it within a route rather than across them. A request that fails on our side (a timeout, an outage, a usage limit) aborts the game, and an aborted game never counts against the model.
What it doesn't show
Paused time removes tapping speed on purpose, and the bench, like the simulator, never taps a hustle by hand once a helper is hired. Cosmetics, boosters bought with Hustlecoin and pulling the rug are left out, because none of them helps a run reach an earnings goal. These scores compare with each other and with the game's strategies, not with a person playing the web game.