Belote AI
PausedGame-playing agents for Belote on OpenSpiel, built as a ladder of measured rungs: hand-built rules, Monte Carlo search, networks that replace the search, and a language model at the table.
Highlights
- +38.7 pts/dealMonte Carlo policy improvement over the best hand-written rule, the largest gain on the ladder.
- 148x fasterA search-free network agent that ties Monte Carlo search over a 6,000-deal match.
- Fix merged upstreamFound and fixed a seeding bug in OpenSpiel's Python ISMCTS.
- Every claim measuredPaired, dealer-swapped matches with confidence intervals, sized in advance to detect 5 points a deal.
Overview
Game-playing agents for Belote, a four-player French trick-taking card game, built on OpenSpiel. The project is a ladder: each rung asks one question, answers it with a measured comparison, and hands the result to the next, from fixed rules through Monte Carlo search to neural networks and a local language model.
The strongest card play contains no network. One step of Monte Carlo policy improvement over a hand-written rule is the largest gain on the ladder, and the networks buy speed rather than strength: a search-free agent ties that search at 148x the speed. Building the game’s resampler also surfaced a seeding bug in OpenSpiel’s own ISMCTS, fixed upstream in open_spiel#1584.
Problem
- Belote is not an OpenSpiel game. There is no engine to build agents on, and “Belote” is many rule sets: this is classic à la retourne, with a take-or-pass auction, not Coinchée or Contrée.
- One deal swings by roughly ±162 points, and not like a bell curve. The all-or-nothing contract piles margins up at the extremes: one dealer orientation alone has a standard deviation of 123 points a deal, so “I won fifty deals” carries almost no information.
- The hands are hidden. A search agent has to reason over worlds consistent with what its seat actually knows, never over the real deal.
- Bidding and card play confound each other. A margin between two agents that differ in both halves can be attributed to neither, and a search agent that searches every action silently bids by search too.
- A language model is too slow for a match. At 7.7 to 16.7 seconds a decision, the thousands of deals a 5-point comparison needs are out of reach.
Approach
Every claim of the form A beats B is a paired match: each deal is played twice with the dealer swapped, the margin carries a 95% confidence interval, and the deal count is solved in advance to detect 5 points a deal at 80% power. An interval containing zero is reported as a tie, which is a bound (smaller than about five points), not a claim of equality. A comparison meant to isolate card play pins bidding, and a guard verifies it rather than trusting the configuration.
Three instruments carry that rule to players of very different cost:
matches for whole agents, decision grading for a player too slow to
play hundreds of deals, and a judge that screens a card-play change too
small for a match, as a paired difference at the same stored positions. Every
rung writes its configuration, rows and findings to results/ladder/, so
each number on this page traces to a record in the repo.
What I built
The agents the ladder converged on:
| Agent | Role | Result | Cost |
|---|---|---|---|
BidNet + Monte Carlo | Strongest agent | +5.50 over Monte Carlo with rule bidding | ~120 ms/deal |
BidNet + PlayNet | Fastest strong agent | Level with Monte Carlo, -0.63 [-2.06, +0.81] | ~0.9 ms/deal |
| Monte Carlo, 32 worlds | Strongest card play | +38.68 over the best rule, win_or_dump | ~120 ms/deal |
BidNet | Bidding | +5.59 over the best rule, hand_strength 62+ |
The gameDoneBelote as an OpenSpiel game, its scoring and resampler checked against the public API.
GoalImplement Belote as an OpenSpiel game whose rules, scoring and resampling behave exactly as the agents built on it assume.
python_belote lives in a fork of OpenSpiel: chance nodes deal the cards,
the framework enforces imperfect information, and one flat action space of
38 actions covers the auction and card play. The fork carries 42 rule tests;
the ladder’s own first rung checks the contract with the agents instead:
returns zero-sum over 200 deals, trick points exactly 162 or 252, and
resampled worlds that keep the actor’s hand, never resurrect a played card
and respect every inferred void.
Unit of play
- Chose
- One deal is one game, returning the signed point margin
- Instead of
- A race to 501 or 1000 across deals
- Why
- Every result becomes a mean of independent per-deal margins, and the game is zero-sum by construction.
Declarations
- Chose
- Belote-rebelote only, granted automatically
- Instead of
- Announcement actions, plus tierce, cinquante, cent and carré
- Why
- Keeps the deck worth a fixed 162 (252 on a capot), which every confidence interval depends on, without adding a bluffing action.
Dealer
- Chose
- A game parameter that rotates on each redeal
- Instead of
- A fixed first seat
- Why
- It is what makes dealer-swapped paired matches possible, the most important measurement device in the project.
MeasurementDonePairing removes 28% more noise than averaging, and a true null reads as a tie.
GoalBuild the instrument before any agent worth pointing it at, and prove it on cases where the right answer is known.
Who deals decides who bids first, which is worth real points, so an unpaired comparison partly measures seat luck. The fix borrows duplicate bridge’s trick:
One seeded shuffle
the identical cards are dealt both times
Dealer at seat 0
one team bids first: this orientation's margin
Dealer at seat 1
the other team bids first: the mirror margin
Paired margin
seat-order advantage cancels out
The rung measures the instrument with bots too simple to have opinions:
| Check | Result |
|---|---|
| Spread, one dealer orientation | sd 123.0 |
| Spread expected from averaging two draws alone | sd 87.0 |
| Spread, dealer-swapped pair | sd 63.0, 28% tighter than averaging |
| Random vs random, a known null | -0.04 [-3.52, +3.45] over 1,199 deals: tie |
| Sized to detect 5 points a deal | 1,153 deals, achieved half-width 3.54 |
| Bidding guard, always-take vs always-pass | caught at deal 0 |
Hand-built rulesDoneRules beat random by +12.6 a deal, and counting trumps beats counting points by +4.7.
GoalFind the strongest player that can be written by hand, with bidding and card play tuned separately and in the right order.
The first agent splits into two independent halves, a bid rule and a card-play rule, so every later comparison can freeze one and vary the other. That split paid off immediately: the whole agent beats random by +12.55, but isolated, bidding carries +20.86 while playing the highest-value card loses to random card play by 3.84.
Card play was ranked first, bidding pinned, against highest_value:
Only then was the bid threshold tuned, against win_or_dump rather than
against random. A second bid rule, hand_strength, adds trump length,
honours and voids to raw card points, and beats the tuned points rule by
+4.73 [+1.28, +8.19].
Monte CarloDoneOne step of policy improvement over the best rule, no training: +38.7 a deal.
GoalTest whether one step of policy improvement over the best rule beats that rule, with no network, no tree and no training.
At each real choice the agent samples complete deals consistent with what
its seat knows, plays every legal card out in each under win_or_dump, and
keeps the best average. Two details do most of the work:
- Common random numbers. Every candidate card is scored on the same worlds, the pairing trick moved inside the agent.
- Forced decisions are skipped. About a third of card-play decisions have one legal card.
Against win_or_dump, bidding pinned on both sides:
Four worlds already capture 67% of the 32-world margin, so most of the gain is the method. But every smaller budget also loses to 32 head to head, so 32 worlds is a floor, not an optimum, at 385x the rule’s cost.
PlayNetDone86% of Monte Carlo's margin at 1/152 of the cost; no lever closes the last 5.17 points.
GoalAmortize the Monte Carlo agent into a network fast enough to play against, and find out what limits the copy.
MCPlayBot.evaluate() already values every legal card, so 25,000 played
deals yield 846,086 labels for free. PlayNet predicts each card’s advantage
over the average legal card, on a trump-relative encoding, with the split
grouped by deal because ~34 decisions share each deal’s outcome.
Input (145)
132 for the position, 13 for the candidate card
Hidden 1 (256)
Linear, ReLU
Hidden 2 (128)
Linear, ReLU
Output (1)
the card's advantage, in points
Chosen over 128 × 64 by held-out R² (+0.411 against +0.405). Capacity is not the limit.
It beats win_or_dump by +33.81 against the search’s +39.12, at 0.79 ms a
deal against 120, and loses to the search head to head, -5.17 [-6.59,
-3.74]. Every lever on that gap was measured:
| Lever | Result |
|---|---|
| 128 label worlds instead of 32 | +0.018 R² |
| Twice the data | +1.14 in play, tie |
| Who played each card | -0.76 in play, tie |
| A listwise ranking loss | -0.08 [-0.63, +0.46], tie |
| As the search’s rollout policy | +1.48, tie, at 1.4x the cost |
| A value net as the search’s leaf evaluator | -4.03 at depth 8, -23.29 at depth 0 |
BidNet and the search-free agentDoneBidNet out-bids the rule by +5.59; with PlayNet it ties Monte Carlo at 148x the speed.
GoalReplace the hand-written bid rule with a network, then assemble both networks into an agent with no search at run time.
The auction has no teacher, so each bid’s value is measured by playing it
out. Every legal bid is cloned and played to the end with PlayNet, in 24
re-dealt worlds consistent with the bidder’s hand, on the same worlds for
every bid. Resampling cuts label noise within a trump-length bucket from
147.2 to 46.5, and the labels read the way a player would expect:
| Round-1 trump length | Take minus pass |
|---|---|
| 0-2 | -54.9 |
| 3 | +28.1 |
| 4+ | +77.2 |
BidNet is also less selective than the rule (0.20 redeals a deal against
0.55), so its margin is not bought by declining to play.
Decision gradingDoneOne decision becomes one measurement: 411 positions rank every player in match order.
GoalMeasure a player one decision at a time, so that a player too slow or too costly for a match lands on a scale that already means something.
Monte Carlo at 128 worlds values every legal card at a position, so a choice is scored exactly: cost = best card’s value - chosen card’s value. 411 decisions from 40 deals, each stored with the action history that rebuilds the state, grade every player already on the ladder:
| Player | Points given up per decision | Picks a best card |
|---|---|---|
| Random legal card | 8.25 | 42.6% |
win_or_dump | 4.01 | 61.6% |
PlayNet | 2.69 | 71.3% |
| Monte Carlo, 32 worlds | 0.92 | 83.2% |
| The teacher, 128 worlds | 0.48 | 86.1% |
A player can also state claims, checked at three levels: public (“my partner is winning”), hidden (“an opponent holds the ace”) and future (“this ace takes the trick”). The checker is tested against the hands it must not look at, and agrees on all 120 positions.
A language model at the tableDoneA local 12B model gives up more per decision than a random card, and loses by 35.9 a deal.
GoalPut a local language model on the same scale as every other player: first graded one decision at a time, then over whole deals.
gemma4:12b, served by Ollama, sees only what its seat can see and answers
with a card, a reason and any claims, under a schema that deliberately does
not enumerate the legal cards. On 150 graded decisions it gives up 9.28
points per decision against 6.13 for a random legal card, and thinking
first does not help.
A LangGraph pipeline then hands it the counts it got wrong (trumps out, master cards, what beats the trick) and splits the decision into four steps:
| One call | Four steps | |
|---|---|---|
| Points given up per decision | 9.15 | 9.23 |
| “My side wins this trick”, correct | 62.1% | 78.7% |
| Model calls / seconds per decision | 1 / 7.7 | 3 / 24.2 |
It describes the position better and chooses no better: what limits it is judgement, not information. Over 24 whole deals against Monte Carlo it loses -35.92 [-52.54, -19.29] a deal.
Playing the agentDoneA person plays BidNet + Monte Carlo in a browser, then reviews and replays every decision.
GoalPut a person at the table against the strongest agent, the only opponent from outside the project that any agent here faces.
The browser page is a view over the real python_belote state served from
Python, not a JavaScript copy of the rules, so a deal played there is the
same game the matches measure. A terminal version plays the same agent, and
both keep a running record of the human’s margin, with its interval.
Larger models and a new harnessPlannedTarget: a language model that gives up less per decision than a random legal card.
GoalFind out whether what holds the language model back is its size or the way it is asked, measured on the same 150 graded decisions and then in whole deals.
The 12B model reads the board well and chooses badly, and handing it more facts changed nothing, so the next steps test judgement rather than information. The bar is fixed in advance: 6.13 points given up per decision, a random legal card on the same positions.
- Larger models.
gemma4:26band the frontier models, graded first, then in whole deals. That includes whether a stronger model forms beliefs about the hidden hands; the 12B model made 3 reads beyond what the play proves in 150 decisions. - A harness aimed at the choice. The model proposes a few candidate cards for Monte Carlo to check, or plays through a tool-using agent loop, instead of receiving more facts.
- Bidding by the model. A rule bids for it today. Take-or-pass on five cards is a smaller test of judgement, and decision grading extends to it.
Tech stack
What each piece of the stack is actually doing, and where to look for more:
Languages
- Python
The agents, the training, the measurement layer and both front ends.
AI/ML
- OpenSpiel
Belote is python_belote in a fork of OpenSpiel, with a C++ twin verified identical over 40 seeded deals. The game's resampler is what every search agent stands on.
- PyTorch
PlayNet (145 → 256 → 128 → 1) and BidNet (81 → 128 → 64 → 1), trained on Monte Carlo labels with train/validation splits grouped by deal.
- LangGraph
The four-step pipeline (facts computed in code, then infer, evaluate, choose) that tests whether decomposing a decision helps the language model.
- Ollama
Serves gemma4:12b locally for the language-model rungs; the MLX build runs 1.6x faster than GGUF for the same weights.
Future improvements
The game, the hand-built player and the networks are settled; the language
model’s next steps are the planned phase above. Beyond those, these are the
known gaps, and the repo’s
docs list each part’s
measured dead ends alongside them:
- Worlds sampled from a model of the hidden hands. Monte Carlo draws the opponents’ hands uniformly from every deal consistent with the play and ignores what the auction and discards say. A network predicting who holds which card would make a stronger teacher without changing anything downstream; Monte Carlo that sees the real deal bounds what it can buy, and is not yet measured.
- Partner signalling. No agent here signals on purpose, so tactical plays like an appel (calling for a suit through a discard) or an impasse (a finesse) are neither produced nor measured.