Intro
Catan looks simple from the other side of the table. It is not simple for a program.
Every turn mixes dice luck, hidden cards, a board that never looks the same twice, and a legal-action list that can explode once trading is allowed. A bot does not only have to choose “build a road.” It has to choose which road, now, given what it cannot see.
That is why almost nobody trains a Catan bot the way you train a photo classifier. Supervised learning needs a label for every state — “this was the right move.” In Catan that label is expensive, ambiguous, and often wrong the moment the next roll lands. A few projects still go that way (self-play with a delayed outcome, imitation from human logs, DAgger on Catanatron). Most do not.
What people actually ship falls into a small number of algorithm families. They overlap. The strongest public agents usually stack two of them.
Algorithm families
Hand-crafted heuristics
A heuristic is a rule you can point to in the code: prefer the intersection with the most pips, diversify numbers so two settlements do not share the same 6 and 8, expand toward a missing resource, plan a path to 10 victory points.
This is how JSettlers played twenty years ago, how Catanatron’s Value Function and AlphaBeta players still play today.
Search (MCTS and friends)
Monte-Carlo Tree Search does not learn a policy offline. At decision time it grows a tree: pick a move, simulate the rest of the game, back up the result, repeat until the clock runs out.
Plain MCTS (Szita et al., Monte Catano, many course projects) can already beat random and some scripted bots. Two extra ideas matter in Catan:
- Information-set / belief search, because you do not see the other hands.
- Move grouping or action types, because “offer wood for brick” is one idea with hundreds of concrete offers.
Search is not a rival to heuristics. Catanatron’s strong built-in players are search on top of a heuristic evaluator. That combination is still the thing most new agents have to beat.
Model-free reinforcement learning
DQN, PPO, A2C, SAC and their cousins learn from reward by playing millions of games against copies of themselves (or against a heuristic).
They need three unglamorous pieces first: a fast rules engine, a vector or tensor for the board, and a mask so the network cannot pick an illegal action. That is why so many GitHub projects sit on Catanatron or a custom Gym wrapper.
Honest results in this family are mixed. With a stripped ruleset (no player trading, 1v1, first to 7) PPO and DQN learn real strategy. On the full four-player game with trades, several public attempts barely clear random. Reward design and the trading action space are the usual failure points — not the choice between PPO and DQN.
Expert iteration / AlphaZero-style
Here search and learning sit in a loop. A neural net proposes moves and values positions. MCTS uses that net as a prior. Self-play games become the next training set. Repeat.
Read the ruleset before the win rate. Many of the impressive numbers are 1v1, no domestic trades, or a shortened race to 7 VP. That is still real progress. It is not “solved four-player Catan.”
Supervised and imitation.
As outlined in the intro, labeling by hand is miserable. So people cheat in useful ways: train on the agent’s own later predictions (temporal-difference self-play, Justin Asher), clone a search player (DAgger / behavioral cloning), or evolve a small network as a position evaluator (genetic algorithms on Catanatron).
Large language models
There are now two different uses:
- LLM-as-player — the model reads the state and picks a move (CatanBench and the other prompt-only arenas). Strong models win a lot of those tables and still hallucinate development-card rules.
- LLM-as-programmer — the model writes or rewrites a compiled Catanatron player, then the code plays thousands of games (HexMachina and follow-ups). The artifact is a bot, not a chat transcript.
Neither replaces search or a fast simulator.
References
The GitHub table on this site tags each project with these families; the Papers table is the same map with citations.