From 3b527d21ed30b0a05f52c831cdfc9c432e5ebd81 Mon Sep 17 00:00:00 2001 From: Owen Date: Tue, 3 Nov 2026 14:49:07 +0100 Subject: [PATCH] Update domain docs for local tablebase routing (#32) Redefine Tablebase and Cooldown in GLOSSARY.md for the local<=5 / Lichess 6-7 / cache-only-API split, and add Probe and Position key terms. Add ADR-0005 recording the source split and its latency-vs-footprint trade-off, and mark ADR-0001 superseded. --- GLOSSARY.md | 23 ++++++++++++++----- docs/adr/0001-tablebase-via-lichess-api.md | 3 +++ docs/adr/0005-tablebase-source-split.md | 26 ++++++++++++++++++++++ 3 files changed, 47 insertions(+), 5 deletions(-) create mode 100644 docs/adr/0005-tablebase-source-split.md diff --git a/GLOSSARY.md b/GLOSSARY.md index d1d85dc..2f740a1 100644 --- a/GLOSSARY.md +++ b/GLOSSARY.md @@ -63,13 +63,26 @@ promotion (still winning) for conversion puzzles, mate otherwise. **Tablebase**: A database of precomputed optimal outcomes for every position with at most -seven pieces, queried via the lichess API and cached. The trainer caches only -the outcome data it grades on — enough to tell a winning move from a losing -one — not every metric the tablebase reports, and refetches anything else. +seven pieces. Positions with five or fewer pieces are answered from local +Syzygy files; six and seven piece positions come from the lichess API. The +trainer keeps only the outcome data it grades on — enough to tell a winning +move from a losing one — and caches API answers only; local answers are cheap +enough to refetch. + +**Probe**: +A single lookup of one position in the tablebase, returning its result and its +legal moves. A probe is answered locally or by the API. + +**Position key**: +The identity of a tablebase position — board, side to move, castling and +en-passant — excluding the move clocks. FENs that differ only in their clocks +share a position key. +_Avoid_: Full FEN **Cooldown**: -Rate-limit state entered after a 429 from the lichess tablebase API, during -which the trainer blocks play. +Rate-limit state entered after a 429 from the lichess API, during which the +trainer blocks play for positions that need the API (six or seven pieces). +Local positions are unaffected. **Cache sweep**: The periodic halving of LFU cache frequencies (weekly) that lets rarely-used diff --git a/docs/adr/0001-tablebase-via-lichess-api.md b/docs/adr/0001-tablebase-via-lichess-api.md index c50d58b..f888806 100644 --- a/docs/adr/0001-tablebase-via-lichess-api.md +++ b/docs/adr/0001-tablebase-via-lichess-api.md @@ -1,5 +1,8 @@ # Tablebase via the lichess API instead of self-hosting +_Superseded by ADR-0005: five-or-fewer-piece positions are now probed locally; +Lichess remains for six and seven pieces._ + The trainer grades moves against Syzygy tablebases, but self-hosting them is not possible yet, so we query the lichess tablebase API (`tablebase.lichess.ovh`) over HTTP and cache responses. This couples the core play loop to a remote diff --git a/docs/adr/0005-tablebase-source-split.md b/docs/adr/0005-tablebase-source-split.md new file mode 100644 index 0000000..a4cd31d --- /dev/null +++ b/docs/adr/0005-tablebase-source-split.md @@ -0,0 +1,26 @@ +# Answer five-or-fewer-piece positions from local Syzygy, keep six and seven on Lichess, and cache only the API path + +_Status: accepted. Supersedes ADR-0001's "all lookups remote" assumption._ + +Relying entirely on the Lichess tablebase API put network latency and rate +limits (and a cooldown on 429) in front of every lookup. Hosting everything is +not viable either: the 3-4-5-piece Syzygy set is about 939 MB, six pieces about +150 GB, seven pieces about 18 TB, and eight or more pieces are outside every +tablebase. We route by piece count instead: five or fewer are probed locally +from Syzygy WDL+DTZ files, six and seven go to Lichess, and eight or more are +rejected without a lookup. A local failure — missing or partial files, probe +error — falls back to Lichess, so play only blocks when both sources are +unavailable. + +Only the Lichess path is cached. Local probes read from the OS page cache and +are cheap, so an application cache would spend memory on answers we can +regenerate. Cache entries are keyed by position rather than the full FEN: the +result depends only on board, side to move, castling and en-passant, so +clock-only differences are the same position, and the 50-move rule is already +encoded in the WDL/DTZ values rather than the running halfmove clock. + +The cost is a native build: the local prober is an MIT-licensed C library +vendored behind a Rust NIF, so a release build needs the Rust toolchain, and +each host points `TABLEBASE_PATH` at its own copy of the files. In exchange the +common (≤5-piece) case is answered without the network or rate limits, while +the enormous 6-7-piece sets stay off our infrastructure.