Elemental Armory benchmark

The matches

Bounce Duels is the main mode of Elemental Armory, made with Chach16. Each side brings four of eight elemental balls. A ball fights with the ability cards in its slots: they attack on their own, on a timer, on contact or on a wall bounce, and a plain collision does no damage. Each round both sides pick one of their remaining balls and aim its opening launch, then the round runs by itself until a ball is knocked out, or for 45 seconds, when the larger share of health left wins. A ball that fought a decided round is spent, and the first side to two round wins takes the match. The bench runs the game's own simulation and its own match rules, pinned to commit 78afdf2, so a later change to the game cannot move anyone's result.

The opponent is the game's local rival. It deals itself a deck, equips random cards, and seals a random ball and a random aim each round before the player picks, never looking at the player's choice. Every model plays the same matches in the same order from set v1: match k is the same seed at every level, so the rival brings the same deck and makes the same picks for everyone. The levels are the game's ball levels, and the rival plays at the model's level, as the game sets it. Each level adds health, damage and, at levels 5 and 8, a card slot.

A model ranks once it has played the first 50 matches of every level. After that the bench keeps playing further matches, up to 200 per level, so n grows and the intervals narrow. The wins in the table count all of a level's matches.

The decisions

Play is in paused time: the game's 15-second planning clock is gone, and nothing moves while the model thinks. First one request for the deck: the rules, every ball's nine cards with the game's own descriptions and ratings, the counters and the level. As in the game, the deck is built before the rival's is known. Then one request per round, with both decks and their cards, which balls are spent, the score, the counters among the balls left, and every round so far: who launched what at which angle, who won, the health left, and each card's damage, healing, blocks and effects, the game's own battle details. The model answers with a ball and an angle. A drawn round replays with the same balls, so it only chooses a new angle.

One retry follows a reply the game cannot use (no answer tag, a card that is not that ball's, a spent ball), with the reason. A second one takes a recorded default: for the deck, whatever was legal, filled the way the game fills a short deck, with a core card for a ball that has none; for a round, the first ball left and the game's own starting aim, straight at the rival.

Score

The score is the match win rate at each level, averaged so every level counts the same, from 0 to 100. Its 95% interval adds up each level's Agresti-Coull uncertainty. Between evenly matched sides a match is close to a coin flip, so the intervals stay wide until many matches are in.

The game's own players

The game's own players played every match of the set on the same pinned commit. Rival in your seat. The game's own rival in your seat: a random deck, random cards, a random ball each round, aimed anywhere. The rival plays uniformly at random, so this is also the random baseline. Starter player. A new player who changes nothing: the starter deck with its core cards, the first ball in hand, launched straight at the rival. Sharp player. The starter deck with three-star cards in every slot, the ball that counters most of the rival's balls left, launched straight at the rival. Tuned player. A player who measured all 72 cards on the game's simulation: the four elements with the best cards, those cards, counters first.

Cost and time

Same as the chess and sudoku benchmarks: Claude models run through Claude Code and OpenAI models through the Codex CLI on a subscription, with every tool switched off, and local models on our own DGX Sparks. Cost is API-equivalent: what the tokens of every request in the match, retries included, would cost at the lab's list price. Local models have no list price, so they show n/a, not zero. Time is the wall-clock wait for the replies, including any queue on our side, so compare it within a route rather than across them. A request that fails on our side (a timeout, an outage, a usage limit) aborts the game, and an aborted game never counts against the model.

What it doesn't show

Paused time removes the planning clock on purpose. A model reads the cards as text and never sees the arena move, so it cannot learn how a card plays by watching it; it only gets each round's battle details. The rival plays at random, so this measures how well a model builds and picks against a fair but careless opponent, not against a strong person.