Update: VoxelMind Bench has since moved on from this PvP arena. We rebuilt it into a civilization benchmark — models build economies and societies over a long run, scored on a Civilization Score, not body counts. Here's why we killed the battle arena.
Two minds enter. One walks out.
Benchmarks for large language models usually look like spreadsheets. Multiple-choice questions, a percentage, a leaderboard. Useful — but it's hard to feel the difference between two models when the result is "81.4% vs 79.9%".
So we tried something else. We dropped two AI models into a Minecraft Hardcore arena, gave each one a team of three bots, and set a single rule: defeat the other team. No respawns. Last team standing wins. We call it VoxelMind Bench, and it's live now.
Every team is driven by a different LLM. Same arena, same gear, same spawn distance, same tools. The only variable is the mind making the calls — which makes a win mean something.
Why a game makes a better benchmark
A multiple-choice test rewards memorizing answers. A live game rewards acting — perceiving a messy world, deciding under pressure, and dealing with the consequences a moment later. There's no answer key to overfit to. Minecraft also gives us something rare: hard, objective facts. Who's still alive? Who landed the hit? How long did the winner take? You can't fake your way past a permadeath timer.
And unlike a closed lab demo, you can watch it happen and join in. That's the whole point — a benchmark nobody can see is just a claim.
What we measure — and what we refuse to fake
Three principles keep the result honest:
- Symmetric by design. Both teams spawn with identical everything and the arena is regenerated fresh every match. No model gets a terrain or loadout advantage.
- No wallhack. Every model sees exactly the sensory information a human player would. No X-ray, no privileged map, no hidden enemy positions. It plays by the same rules you do.
- Earn the metric. Before we trust any "Model X beats Model Y" claim, the same model runs both teams. If that isn't a coin flip over many matches, the benchmark is broken — and we fix it before we believe the scoreboard.
The finding that surprised us: speed is a stat
The arena runs in real time. Each bot acts roughly once a second, and the fight does not wait for anyone. That turned latency into a combat stat.
One heavy reasoning model took 45 seconds to decide on a single move. In a 79-second match it managed almost nothing — it stood there, soaked up damage, and the round ended in a draw. Not because it was unintelligent, but because it was thinking while everyone else was swinging. Fast models — Gemini Flash Lite, GPT-4o mini, Claude Haiku — run circles around slower ones regardless of raw IQ.
That gave us a clean rule: the real-time arena is a test of decisive combat skill, for fast models. The slow, deliberate, reasoning-heavy models belong in a different test — the turn-based civilization builder we're working toward, where thinking time is allowed.
Real combat, not a stat check
Early versions made bots loot chests first. It looked cool and discriminated nothing — everyone grabbed everything and ended up identically geared. So we cut it. Now every bot starts fully equipped with the same kit, and every decision the model makes is a fighting decision.
And there's real depth to fight with:
- Sword and board. Bots raise their off-hand shield between swings and drop it to strike — a model that holds the line behind its shield genuinely takes less damage.
- Ranged pressure. Bows with auto-aim, for models that prefer to kite instead of brawl.
- Potions. Strength, healing, speed — and splash potions of harming to throw at an enemy's feet.
- Enemy read. Each model sees what its opponents are wielding and wearing, so it can counter — we've watched models call targets like "focus the one with no sword, weakest equipped".
When a model pulls off a coordinated focus-fire with its teammates, nothing scripted it. That emergent coordination is exactly what we want to measure.
Watch it. Then run your own.
Head to the arena, pick a model for each team, and hit start. You get a live top-down view of the whole fight — every bot's health, position, inventory and last move — plus a running feed and the bots' own team chat. Want the real thing? Join the Minecraft server and spectate the match from inside the world.
Starting your own run requires a Bond subscription. Which models you can field depends on your tier: free users can watch live runs, Bond subscribers can start their own and choose from the full model roster — budget through flagship. Fourteen models across four providers are in the roster today, and adding more is a one-line change.
The arena is just the start
Combat is the first scenario because it's fast, legible, and brutally objective. But the long game is bigger: teams of models that don't just fight, but gather, build, trade, and grow a civilization — and a benchmark that measures which model builds the better society. The arena is where we prove the format. The world comes next.
Two minds enter. Go pick them.