Compare commits

...
4 Commits
Author SHA1 Message Date
owenrees 3b527d21ed Update domain docs for local tablebase routing (#32)
Redefine Tablebase and Cooldown in GLOSSARY.md for the local<=5 / Lichess 6-7 / cache-only-API split, and add Probe and Position key terms. Add ADR-0005 recording the source split and its latency-vs-footprint trade-off, and mark ADR-0001 superseded.
2026-11-03 14:49:07 +01:00
owenrees 8ba87d8efe Document TABLEBASE_PATH and local tablebase setup (#31)
Add docs/tablebases.md covering the local/Lichess routing, the TABLEBASE_PATH environment variable (dev vs VPS), the fetch/check mix tasks, boot logging, and the release build requirement. Link it from the README.
2026-11-03 14:36:14 +01:00
owenrees f37242ed29 Add local tablebase path config, fetch and check tasks (#31)
Read the local Syzygy directory from TABLEBASE_PATH so a dev machine and a VPS can point at their own copies. Add mix tablebase.fetch (downloads the 3/4/5 WDL+DTZ set, resumable) and mix tablebase.check (verifies it loads), plus a boot log reporting whether local tablebases are ready and the largest piece count.

Covers the codeable part of #31; installing on the VPS and setting TABLEBASE_PATH there remains a manual step.
2026-11-03 14:26:30 +01:00
owenrees cb35351f8c Wire Fathom into the local tablebase backend (#30)
Add Backend.Fathom: probes positions with five or fewer pieces via the Fathom NIF, maps WDL/DTZ to the trainer structs (flipping per-move perspective), generates SAN, sorts moves best-first, and falls back to Lichess on any failure. The local directory is configured with TABLEBASE_PATH; local answers are no longer cached.

Add Tablebase.normalize_wdl/1 and flip/1, plus tests for the WDL mapping, SAN, checkmate, fallback, and a cache-free end-to-end local probe.
2026-11-03 14:23:46 +01:00
20 changed files with 758 additions and 11 deletions
+18 -5
View File
@@ -63,13 +63,26 @@ promotion (still winning) for conversion puzzles, mate otherwise.
**Tablebase**:
A database of precomputed optimal outcomes for every position with at most
seven pieces, queried via the lichess API and cached. The trainer caches only
the outcome data it grades on — enough to tell a winning move from a losing
one — not every metric the tablebase reports, and refetches anything else.
seven pieces. Positions with five or fewer pieces are answered from local
Syzygy files; six and seven piece positions come from the lichess API. The
trainer keeps only the outcome data it grades on — enough to tell a winning
move from a losing one — and caches API answers only; local answers are cheap
enough to refetch.
**Probe**:
A single lookup of one position in the tablebase, returning its result and its
legal moves. A probe is answered locally or by the API.
**Position key**:
The identity of a tablebase position — board, side to move, castling and
en-passant — excluding the move clocks. FENs that differ only in their clocks
share a position key.
_Avoid_: Full FEN
**Cooldown**:
Rate-limit state entered after a 429 from the lichess tablebase API, during
which the trainer blocks play.
Rate-limit state entered after a 429 from the lichess API, during which the
trainer blocks play for positions that need the API (six or seven pieces).
Local positions are unaffected.
**Cache sweep**:
The periodic halving of LFU cache frequencies (weekly) that lets rarely-used
+14
View File
@@ -10,6 +10,20 @@ Now you can visit [`localhost:4000`](http://localhost:4000) from your browser.
Ready to run in production? Please [check our deployment guides](https://phoenix.hexdocs.pm/deployment.html).
## Local tablebases (optional)
Positions with five or fewer pieces are answered locally from Syzygy tablebases
when `TABLEBASE_PATH` points at a directory of `.rtbw`/`.rtbz` files; otherwise
lookups fall back to Lichess.
```sh
mix tablebase.fetch --path /path/to/syzygy/3-4-5 # once per host (~939 MB)
export TABLEBASE_PATH=/path/to/syzygy/3-4-5 # set per environment
```
See [docs/tablebases.md](docs/tablebases.md) for the full reference — dev vs
VPS paths, `mix tablebase.check`, and boot logging.
## Learn more
* Official website: https://www.phoenixframework.org/
+7
View File
@@ -20,6 +20,13 @@ if System.get_env("PHX_SERVER") do
config :chesstrainer, ChesstrainerWeb.Endpoint, server: true
end
# Local Syzygy tablebases: set TABLEBASE_PATH to the directory holding the
# .rtbw/.rtbz files. When unset, the local backend stays disabled and lookups
# fall back to Lichess.
if tablebase_path = System.get_env("TABLEBASE_PATH") do
config :chesstrainer, :tablebase_path, tablebase_path
end
# Tablebase cache: size and the UTC time of the daily backup. Environment
# variables win over the defaults in config/config.exs so deployments can tune
# the cache without a rebuild. Invalid values are reported and ignored rather
+3
View File
@@ -1,5 +1,8 @@
import Config
# Point the local Fathom backend at the committed 3-piece fixtures.
config :chesstrainer, :tablebase_path, Path.expand("../test/fixtures/tablebase", __DIR__)
# Configure your database
#
# The MIX_TEST_PARTITION environment variable can be used
@@ -1,5 +1,8 @@
# Tablebase via the lichess API instead of self-hosting
_Superseded by ADR-0005: five-or-fewer-piece positions are now probed locally;
Lichess remains for six and seven pieces._
The trainer grades moves against Syzygy tablebases, but self-hosting them is not
possible yet, so we query the lichess tablebase API (`tablebase.lichess.ovh`)
over HTTP and cache responses. This couples the core play loop to a remote
+26
View File
@@ -0,0 +1,26 @@
# Answer five-or-fewer-piece positions from local Syzygy, keep six and seven on Lichess, and cache only the API path
_Status: accepted. Supersedes ADR-0001's "all lookups remote" assumption._
Relying entirely on the Lichess tablebase API put network latency and rate
limits (and a cooldown on 429) in front of every lookup. Hosting everything is
not viable either: the 3-4-5-piece Syzygy set is about 939 MB, six pieces about
150 GB, seven pieces about 18 TB, and eight or more pieces are outside every
tablebase. We route by piece count instead: five or fewer are probed locally
from Syzygy WDL+DTZ files, six and seven go to Lichess, and eight or more are
rejected without a lookup. A local failure — missing or partial files, probe
error — falls back to Lichess, so play only blocks when both sources are
unavailable.
Only the Lichess path is cached. Local probes read from the OS page cache and
are cheap, so an application cache would spend memory on answers we can
regenerate. Cache entries are keyed by position rather than the full FEN: the
result depends only on board, side to move, castling and en-passant, so
clock-only differences are the same position, and the 50-move rule is already
encoded in the WDL/DTZ values rather than the running halfmove clock.
The cost is a native build: the local prober is an MIT-licensed C library
vendored behind a Rust NIF, so a release build needs the Rust toolchain, and
each host points `TABLEBASE_PATH` at its own copy of the files. In exchange the
common (≤5-piece) case is answered without the network or rate limits, while
the enormous 6-7-piece sets stay off our infrastructure.
+70
View File
@@ -0,0 +1,70 @@
# Tablebases
The trainer answers endgame positions from two sources, chosen by piece count
(kings included):
| Pieces | Source |
| ------ | ------ |
| 5 or fewer | Local Syzygy files, probed through the Fathom NIF |
| 6–7 | Lichess tablebase API, cached |
| 8 or more | Not covered — rejected before any lookup |
If a local probe fails (files missing, partial set, probe error), the lookup
falls back to Lichess, so play only blocks when both sources are unavailable.
## `TABLEBASE_PATH`
The local backend reads its directory from the **`TABLEBASE_PATH`** environment
variable at boot (`config/runtime.exs`). It is environment-specific: a dev
machine and a VPS will point at different copies.
```sh
# Development
export TABLEBASE_PATH=/home/you/Downloads/tablebase
mix phx.server
# Production (release)
TABLEBASE_PATH=/data/syzygy/3-4-5 bin/chesstrainer start
```
The value must reach the process. A bare `TABLEBASE_PATH=...` line in bash
only sets a shell variable and is **not** exported to `mix` or the release —
use `export`, or prefix the command. For a release, systemd unit or container,
set it in that service's environment, not an interactive shell.
When `TABLEBASE_PATH` is unset the local backend is disabled and every lookup
falls back to Lichess (subject to its rate limits).
## Installing the files
Fathom expects the `.rtbw` (WDL) and `.rtbz` (DTZ) files **flat** in the
directory; both sets are required (WDL for the result, DTZ for optimal-move
ranking and the 50-move-rule categories).
```sh
mix tablebase.fetch --path /path/to/syzygy/3-4-5 # ~939 MB, resumable
mix tablebase.fetch --path /path/to/syzygy/3-4-5 --dry-run
mix tablebase.check --path /path/to/syzygy/3-4-5 # verifies it loads
```
`mix tablebase.check` exits non-zero when the path is unset, contains no files,
or Fathom cannot initialise it.
## Boot logging
On startup the app logs one of:
- `local Syzygy tablebases ready at <path> (largest=5)`
- `local Syzygy tablebases disabled (TABLEBASE_PATH unset)`
- `local Syzygy tablebases at <path> contain no files`
- `local Syzygy tablebases failed: <reason>`
## Deployment note
The Fathom NIF is compiled to `priv/native/fathom_nif.so` and ships inside the
release. Whichever machine *builds* the release therefore needs the Rust
toolchain (and crates.io access); the runtime host only needs the release
artefact plus the tablebase files.
See `docs/adr/0001-tablebase-via-lichess-api.md` (predates local probing) and
`docs/adr/0004-tablebase-cache-shape-and-scaling.md`.
+2 -1
View File
@@ -20,7 +20,8 @@ defmodule Chesstrainer.Application do
ChesstrainerWeb.Endpoint,
{Registry, keys: :unique, name: Chesstrainer.CacheRegistry},
{Task.Supervisor, name: Chesstrainer.Cache.BackupSupervisor},
{Chesstrainer.Cache, {:tablebase, cache_opts}}
{Chesstrainer.Cache, {:tablebase, cache_opts}},
Chesstrainer.Endgames.Tablebase.Local
]
# See https://elixir.hexdocs.pm/Supervisor.html
+25 -1
View File
@@ -44,6 +44,29 @@ defmodule Chesstrainer.Endgames.Tablebase do
def normalize_category(_category), do: :unknown
@doc """
Normalises a Fathom WDL atom into a trainer result.
Like `normalize_category/1`, cursed-win and blessed-loss are draws because the
50-move rule prevents the decisive result.
"""
@spec normalize_wdl(atom()) :: result()
def normalize_wdl(:win), do: :win
def normalize_wdl(:cursed_win), do: :draw
def normalize_wdl(:draw), do: :draw
def normalize_wdl(:blessed_loss), do: :draw
def normalize_wdl(:loss), do: :loss
def normalize_wdl(_wdl), do: :unknown
@doc """
Flips an outcome to the other side's perspective.
"""
@spec flip(atom()) :: result()
def flip(:win), do: :loss
def flip(:loss), do: :win
def flip(:draw), do: :draw
def flip(_outcome), do: :unknown
@spec new(String.t()) ::
{:ok, t()}
| {:cooldown, non_neg_integer()}
@@ -68,7 +91,8 @@ defmodule Chesstrainer.Endgames.Tablebase do
{:error, :invalid_endgame}
{:ok, tablebase} ->
Tablebase.Cache.put(key, tablebase)
# Local probes are cheap; only remote (Lichess) answers are cached.
if Tablebase.Router.remote?(fen), do: Tablebase.Cache.put(key, tablebase)
{:ok, tablebase}
end
end
@@ -0,0 +1,101 @@
defmodule Chesstrainer.Endgames.Tablebase.Backend.Fathom do
@moduledoc """
Local Syzygy backend powered by the Fathom NIF.
Handles positions with up to five pieces. Falls back to the remote backend
(Lichess by default) when the tablebases are not configured, the probe fails,
or Fathom cannot answer the position, so play never blocks.
The backend is overridable for tests via
`config :chesstrainer, :tablebase_fallback, Module`.
"""
@behaviour Chesstrainer.Endgames.Tablebase.Backend
alias Chesstrainer.Endgames.Tablebase
alias Chesstrainer.Endgames.Tablebase.Backend.Lichess
alias Chesstrainer.Endgames.Tablebase.Fathom, as: Nif
alias Chesstrainer.Endgames.Tablebase.Local
alias Chesstrainer.Endgames.Tablebase.Move
alias Chesstrainer.Endgames.Tablebase.San
@impl true
def probe(fen) do
with {:ok, _largest} <- Local.check(),
{:ok, {root, moves}} <- Nif.probe(fen) do
case root do
:failed -> fallback(fen)
_ -> {:ok, to_tablebase(fen, root, moves)}
end
else
_ -> fallback(fen)
end
end
defp fallback(fen) do
(Application.get_env(:chesstrainer, :tablebase_fallback) || Lichess).probe(fen)
end
defp to_tablebase(_fen, :checkmate, _moves) do
%Tablebase{category: :loss, checkmate: true, moves: []}
end
defp to_tablebase(_fen, :stalemate, _moves) do
%Tablebase{category: :draw, checkmate: false, moves: []}
end
defp to_tablebase(fen, {wdl, _from, _to, _promotes, _ep, _dtz}, moves) do
%Tablebase{
category: Tablebase.normalize_wdl(wdl),
checkmate: false,
moves: fen |> map_moves(moves) |> sort_moves()
}
end
defp map_moves(fen, moves) do
Enum.map(moves, fn {wdl, from, to, promotes, _ep, dtz} ->
uci = uci(from, to, promotes)
%Move{
category: move_category(wdl),
dtz: dtz,
uci: uci,
san: San.for_uci(fen, uci)
}
end)
end
# Best move first: highest mover outcome, then sharpest DTZ (lowest for a
# mover who is winning, highest for a mover who is losing).
defp sort_moves(moves) do
Enum.sort_by(moves, fn %Move{category: category, dtz: dtz} ->
rank = mover_rank(Tablebase.flip(category))
{-rank, if(rank == 0, do: -(dtz || 0), else: dtz || 0)}
end)
end
# Fathom reports a move's WDL from the mover's perspective; the trainer stores
# the resulting position's category (the opponent's perspective).
defp move_category(wdl), do: Tablebase.flip(Tablebase.normalize_wdl(wdl))
defp mover_rank(:win), do: 2
defp mover_rank(:draw), do: 1
defp mover_rank(:loss), do: 0
defp mover_rank(_), do: -1
defp uci(from, to, promotes) do
square(from) <> square(to) <> promotion_suffix(promotes)
end
defp square(square) do
file = rem(square, 8)
rank = div(square, 8)
<<?a + file>> <> Integer.to_string(rank + 1)
end
defp promotion_suffix(1), do: "q"
defp promotion_suffix(2), do: "r"
defp promotion_suffix(3), do: "b"
defp promotion_suffix(4), do: "n"
defp promotion_suffix(_), do: ""
end
@@ -0,0 +1,68 @@
defmodule Chesstrainer.Endgames.Tablebase.Local do
@moduledoc """
Local Syzygy tablebase configuration and verification.
The directory is read from `TABLEBASE_PATH` (see `config/runtime.exs`), so the
same release can point at a different location on a dev machine and on a VPS
without a rebuild. When it is unset the local backend stays disabled and
lookups fall back to Lichess.
On boot this logs whether the local tablebases loaded and how many pieces they
cover, so a misconfigured path is visible in the logs.
"""
use GenServer
require Logger
alias Chesstrainer.Endgames.Tablebase.Fathom
@spec path() :: String.t() | nil
def path do
case Application.get_env(:chesstrainer, :tablebase_path) do
path when is_binary(path) and path != "" -> path
_ -> nil
end
end
@doc """
Initialises Fathom against the configured directory.
Returns `{:ok, largest}` where `largest` is the biggest piece count available
(`0` when the directory has no tablebase files), or `{:error, :not_configured}`
when `TABLEBASE_PATH` is unset.
"""
@spec check() :: {:ok, non_neg_integer()} | {:error, term()}
def check do
with path when is_binary(path) <- path(),
{:ok, largest} <- Fathom.init(path) do
{:ok, largest}
else
nil -> {:error, :not_configured}
{:error, _reason} = error -> error
end
end
def start_link(_opts), do: GenServer.start_link(__MODULE__, :ok, name: __MODULE__)
@impl true
def init(:ok) do
log_status()
{:ok, %{}}
end
defp log_status do
case check() do
{:ok, 0} ->
Logger.warning("local Syzygy tablebases at #{path()} contain no files")
{:ok, largest} ->
Logger.info("local Syzygy tablebases ready at #{path()} (largest=#{largest})")
{:error, :not_configured} ->
Logger.info("local Syzygy tablebases disabled (TABLEBASE_PATH unset)")
{:error, reason} ->
Logger.warning("local Syzygy tablebases failed: #{inspect(reason)}")
end
end
end
@@ -11,14 +11,14 @@ defmodule Chesstrainer.Endgames.Tablebase.Router do
"""
alias Chesstrainer.Endgames.Tablebase
alias Chesstrainer.Endgames.Tablebase.Backend.Fathom, as: FathomBackend
alias Chesstrainer.Endgames.Tablebase.Backend.Lichess
alias Chesstrainer.FEN
@max_local_pieces 5
@max_pieces 7
# The local Syzygy backend will replace Lichess here once it exists.
@local_backend Lichess
@local_backend FathomBackend
@remote_backend Lichess
@spec backend(String.t()) :: {:ok, module()} | {:error, :not_covered | :invalid_endgame}
@@ -31,6 +31,12 @@ defmodule Chesstrainer.Endgames.Tablebase.Router do
end
end
@doc """
True when the position is answered by the remote (Lichess) backend.
"""
@spec remote?(String.t()) :: boolean()
def remote?(fen), do: backend(fen) == {:ok, @remote_backend}
@spec probe(String.t()) ::
{:ok, Tablebase.t()} | {:cooldown, non_neg_integer()} | {:error, term()}
def probe(fen) do
+140
View File
@@ -0,0 +1,140 @@
defmodule Chesstrainer.Endgames.Tablebase.San do
@moduledoc """
Generates Standard Algebraic Notation for a UCI move in a position.
The local Fathom backend returns moves as board squares, so this turns them
into the SAN the trainer displays. Returns `nil` when the move cannot be
parsed or is not legal in the position.
"""
alias Chex.Game
alias Chex.Piece
@files ~w(a b c d e f g h)a
@piece_letters %{king: "K", queen: "Q", rook: "R", bishop: "B", knight: "N"}
@spec for_uci(String.t(), String.t()) :: String.t() | nil
def for_uci(fen, uci) do
with {:ok, game} <- game(fen),
{:ok, from, to, promote} <- parse_uci(uci),
{name, color, _start} <- Map.get(game.board, from),
{:ok, next} <- Game.move(game, {from, to}, promote || :queen) do
build(game, name, color, from, to, promote) <> suffix(next)
else
_ -> nil
end
end
defp game(fen) do
Game.new(fen)
rescue
_ -> {:error, :invalid_fen}
end
defp build(game, name, color, from, to, promote) do
if name == :king and abs(file_index(from) - file_index(to)) == 2 do
if file_index(to) > file_index(from), do: "O-O", else: "O-O-O"
else
normal(game, name, color, from, to, promote)
end
end
defp normal(game, name, color, from, to, promote) do
capture = capture?(game, color, from, to)
capture_marker = if capture, do: "x", else: ""
prefix =
if name == :pawn do
if capture, do: file(from), else: ""
else
Map.get(@piece_letters, name, "") <> disambiguation(game, name, color, from, to)
end
prefix <> capture_marker <> square_string(to) <> promotion_suffix(promote)
end
defp capture?(game, color, from, to) do
case Map.get(game.board, to) do
{_name, ^color, _start} -> false
{_name, _other, _start} -> true
nil -> piece_name(game, from) == :pawn and file_index(from) != file_index(to)
end
end
# Adds a file/rank/both only when another same-type piece could also reach `to`.
defp disambiguation(game, name, color, from, to) do
others =
game.board
|> Enum.filter(fn {square, {pname, pcolor, _start}} ->
square != from and pname == name and pcolor == color and
to in Piece.possible_moves(game, square)
end)
|> Enum.map(fn {square, _piece} -> square end)
cond do
others == [] ->
""
Enum.all?(others, &(file_index(&1) != file_index(from))) ->
file(from)
Enum.all?(others, &(rank(&1) != rank(from))) ->
Integer.to_string(rank(from))
true ->
file(from) <> Integer.to_string(rank(from))
end
end
defp suffix(next) do
cond do
Game.checkmate?(next) -> "#"
Game.in_check?(next, next.active_color) -> "+"
true -> ""
end
end
defp promotion_suffix(nil), do: ""
defp promotion_suffix(:queen), do: "=Q"
defp promotion_suffix(:rook), do: "=R"
defp promotion_suffix(:bishop), do: "=B"
defp promotion_suffix(:knight), do: "=N"
defp piece_name(game, square) do
case Map.get(game.board, square) do
{name, _color, _start} -> name
nil -> nil
end
end
defp parse_uci(uci) when is_binary(uci) do
with <<ff, fr, tf, tr, promotion::binary>> <- uci,
{:ok, from} <- parse_square(ff, fr),
{:ok, to} <- parse_square(tf, tr),
{:ok, promote} <- parse_promotion(promotion) do
{:ok, from, to, promote}
else
_ -> :error
end
end
defp parse_uci(_uci), do: :error
defp parse_square(file, rank) when file in ?a..?h and rank in ?1..?8 do
{:ok, {String.to_existing_atom(<<file>>), rank - ?0}}
end
defp parse_square(_file, _rank), do: :error
defp parse_promotion(""), do: {:ok, nil}
defp parse_promotion("q"), do: {:ok, :queen}
defp parse_promotion("r"), do: {:ok, :rook}
defp parse_promotion("b"), do: {:ok, :bishop}
defp parse_promotion("n"), do: {:ok, :knight}
defp parse_promotion(_promotion), do: :error
defp file_index({file, _rank}), do: Enum.find_index(@files, &(&1 == file))
defp file({file, _rank}), do: Atom.to_string(file)
defp rank({_file, rank}), do: rank
defp square_string({file, rank}), do: Atom.to_string(file) <> Integer.to_string(rank)
end
+37
View File
@@ -0,0 +1,37 @@
defmodule Mix.Tasks.Tablebase.Check do
use Mix.Task
@shortdoc "Verify the local Syzygy tablebases"
@moduledoc """
Verifies that the configured local Syzygy tablebases load.
TABLEBASE_PATH=/data/syzygy/3-4-5 mix tablebase.check
mix tablebase.check --path /data/syzygy/3-4-5
Exits non-zero when the path is unset, empty, or Fathom fails to initialise.
"""
@impl Mix.Task
def run(args) do
{opts, _rest, _invalid} = OptionParser.parse(args, strict: [path: :string])
path =
opts[:path] || System.get_env("TABLEBASE_PATH") ||
Chesstrainer.Endgames.Tablebase.Local.path()
if is_nil(path) do
Mix.raise("set TABLEBASE_PATH or pass --path")
end
case Chesstrainer.Endgames.Tablebase.Fathom.init(Path.expand(path)) do
{:ok, 0} ->
Mix.raise("no tablebase files found in #{path}")
{:ok, largest} ->
Mix.shell().info("local tablebases at #{path}: largest=#{largest}")
{:error, reason} ->
Mix.raise("failed to initialise Fathom: #{inspect(reason)}")
end
end
end
+91
View File
@@ -0,0 +1,91 @@
defmodule Mix.Tasks.Tablebase.Fetch do
use Mix.Task
@default_mirror "http://tablebase.sesse.net/syzygy/3-4-5/"
@shortdoc "Download the 3/4/5-piece Syzygy tablebases"
@moduledoc """
Downloads the complete 3/4/5-piece Syzygy WDL + DTZ set (~939 MB).
mix tablebase.fetch --path /data/syzygy/3-4-5
TABLEBASE_PATH=/data/syzygy/3-4-5 mix tablebase.fetch
Existing files are skipped, so re-running resumes an interrupted download.
Use `--mirror` to override the source (defaults to `#{@default_mirror}`), or
`--dry-run` to see how many files are available without downloading.
Verify the result with `mix tablebase.check`.
"""
@impl Mix.Task
def run(args) do
{opts, _rest, _invalid} =
OptionParser.parse(args,
strict: [path: :string, mirror: :string, dry_run: :boolean]
)
path =
opts[:path] || System.get_env("TABLEBASE_PATH") ||
Mix.raise("pass --path or set TABLEBASE_PATH, e.g. --path /data/syzygy/3-4-5")
mirror = opts[:mirror] || @default_mirror
path = Path.expand(path)
{:ok, _apps} = Application.ensure_all_started(:req)
files = list_files(mirror)
if opts[:dry_run] do
Mix.shell().info("Would fetch #{length(files)} files into #{path}")
else
File.mkdir_p!(path)
Mix.shell().info("Fetching 3/4/5 Syzygy tablebases into #{path}")
Mix.shell().info("#{length(files)} files to check")
{downloaded, skipped} =
Enum.reduce(files, {0, 0}, fn file, {downloaded, skipped} ->
target = Path.join(path, file)
if present?(target) do
{downloaded, skipped + 1}
else
download(mirror <> file, target, file)
{downloaded + 1, skipped}
end
end)
Mix.shell().info("Done: #{downloaded} downloaded, #{skipped} already present")
Mix.shell().info("Verify with: mix tablebase.check --path #{path}")
end
end
defp list_files(mirror) do
%{status: 200, body: body} = Req.get!(mirror)
~r/href="([^"]+\.rtb[zw])"/
|> Regex.scan(body)
|> Enum.map(fn [_, file] -> file end)
|> Enum.uniq()
end
defp present?(target), do: File.exists?(target) and File.stat!(target).size > 0
defp download(url, target, file) do
partial = target <> ".part"
Mix.shell().info(" #{file}")
case Req.get(url, into: File.stream!(partial)) do
{:ok, %{status: 200}} ->
File.rename!(partial, target)
{:ok, %{status: status}} ->
File.rm(partial)
Mix.raise("HTTP #{status} while fetching #{url}")
{:error, reason} ->
File.rm(partial)
Mix.raise("failed to fetch #{url}: #{inspect(reason)}")
end
end
end
@@ -0,0 +1,85 @@
defmodule Chesstrainer.Endgames.Tablebase.Backend.FathomTest.Stub do
@moduledoc false
alias Chesstrainer.Endgames.Tablebase
def probe(_fen), do: {:ok, %Tablebase{category: :draw, checkmate: false, moves: []}}
end
defmodule Chesstrainer.Endgames.Tablebase.Backend.FathomTest do
# Not async: the local backend reads app config and Fathom keeps global state.
use ExUnit.Case, async: false
alias Chesstrainer.Endgames.Tablebase
alias Chesstrainer.Endgames.Tablebase.Backend.Fathom, as: FathomBackend
alias Chesstrainer.Endgames.Tablebase.Cache, as: TablebaseCache
alias Chesstrainer.Endgames.Tablebase.Local
alias Chesstrainer.FEN
@krk "4k3/8/8/8/8/8/8/R3K3 w - - 0 1"
@mate "7k/6Q1/5K2/8/8/8/8/8 b - - 0 1"
test "probes a KRvK position locally" do
assert {:ok, %Tablebase{} = tablebase} = FathomBackend.probe(@krk)
assert tablebase.category == :win
refute tablebase.checkmate
assert length(tablebase.moves) == 15
assert Enum.all?(tablebase.moves, &(&1.category == :loss))
best = hd(tablebase.moves)
assert best.uci == "a1a7"
assert best.dtz == 23
assert best.san == "Ra7"
end
test "detects checkmate" do
assert {:ok, %Tablebase{category: :loss, checkmate: true, moves: []}} =
FathomBackend.probe(@mate)
end
test "Local.check/0 reports the configured fixture set" do
assert is_binary(Local.path())
assert {:ok, 3} = Local.check()
end
test "answers through Tablebase.new without a cached entry" do
# A draw-returning fallback proves the win came from Fathom, not Lichess.
previous = Application.get_env(:chesstrainer, :tablebase_fallback)
on_exit(fn ->
if previous,
do: Application.put_env(:chesstrainer, :tablebase_fallback, previous),
else: Application.delete_env(:chesstrainer, :tablebase_fallback)
end)
Application.put_env(:chesstrainer, :tablebase_fallback, __MODULE__.Stub)
TablebaseCache.delete(FEN.position_key(@krk))
assert {:ok, %Tablebase{category: :win, moves: moves}} = Tablebase.new(@krk)
assert length(moves) == 15
end
describe "fallback" do
setup do
previous_path = Application.get_env(:chesstrainer, :tablebase_path)
previous_fallback = Application.get_env(:chesstrainer, :tablebase_fallback)
on_exit(fn ->
Application.put_env(:chesstrainer, :tablebase_path, previous_path)
if previous_fallback do
Application.put_env(:chesstrainer, :tablebase_fallback, previous_fallback)
else
Application.delete_env(:chesstrainer, :tablebase_fallback)
end
end)
end
test "falls back when the tablebases are not configured" do
Application.put_env(:chesstrainer, :tablebase_path, nil)
Application.put_env(:chesstrainer, :tablebase_fallback, __MODULE__.Stub)
assert {:ok, %Tablebase{category: :draw}} = FathomBackend.probe(@krk)
end
end
end
@@ -1,6 +1,7 @@
defmodule Chesstrainer.Endgames.Tablebase.RouterTest do
use ExUnit.Case, async: true
alias Chesstrainer.Endgames.Tablebase.Backend.Fathom, as: FathomBackend
alias Chesstrainer.Endgames.Tablebase.Backend.Lichess
alias Chesstrainer.Endgames.Tablebase.Router
@@ -11,7 +12,7 @@ defmodule Chesstrainer.Endgames.Tablebase.RouterTest do
describe "backend/1" do
test "routes five or fewer pieces to the local slot" do
assert Router.backend(@three) == {:ok, Lichess}
assert Router.backend(@three) == {:ok, FathomBackend}
end
test "routes six and seven pieces to the remote slot" do
@@ -0,0 +1,44 @@
defmodule Chesstrainer.Endgames.Tablebase.SanTest do
use ExUnit.Case, async: true
alias Chesstrainer.Endgames.Tablebase.San
@start "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"
test "pawn and piece moves" do
assert San.for_uci(@start, "e2e4") == "e4"
assert San.for_uci(@start, "g1f3") == "Nf3"
end
test "captures" do
after_e4_d5 = "rnbqkbnr/ppp1pppp/8/3p4/4P3/8/PPPP1PPP/RNBQKBNR w KQkq d6 0 2"
assert San.for_uci(after_e4_d5, "e4d5") == "exd5"
end
test "castling" do
fen = "r3k2r/8/8/8/8/8/8/R3K2R w KQkq - 0 1"
assert San.for_uci(fen, "e1g1") == "O-O"
assert San.for_uci(fen, "e1c1") == "O-O-O"
end
test "promotion and check" do
assert San.for_uci("8/P7/8/8/8/4k3/8/4K3 w - - 0 1", "a7a8q") == "a8=Q"
assert San.for_uci("4k3/P7/8/8/8/8/8/4K3 w - - 0 1", "a7a8q") == "a8=Q+"
end
test "disambiguates by file when two pieces can reach the square" do
fen = "4k3/8/8/8/8/8/8/1N1NK3 w - - 0 1"
assert San.for_uci(fen, "b1c3") == "Nbc3"
assert San.for_uci(fen, "d1c3") == "Ndc3"
end
test "checkmate suffix" do
fen = "6k1/5ppp/8/8/8/8/8/4R1K1 w - - 0 1"
assert San.for_uci(fen, "e1e8") == "Re8#"
end
test "returns nil for an invalid move" do
assert San.for_uci(@start, "e2e5") == nil
assert San.for_uci(@start, "zzzz") == nil
end
end
@@ -33,6 +33,17 @@ defmodule Chesstrainer.Endgames.TablebaseTest do
end
end
describe "normalize_wdl/1" do
test "maps Fathom WDL atoms, treating 50-move-rule draws as draws" do
assert Tablebase.normalize_wdl(:win) == :win
assert Tablebase.normalize_wdl(:loss) == :loss
assert Tablebase.normalize_wdl(:draw) == :draw
assert Tablebase.normalize_wdl(:cursed_win) == :draw
assert Tablebase.normalize_wdl(:blessed_loss) == :draw
assert Tablebase.normalize_wdl(:unknown) == :unknown
end
end
describe "struct shape" do
test "Tablebase carries only the fields the trainer reads" do
assert %Tablebase{} |> Map.from_struct() |> Map.keys() |> Enum.sort() ==
@@ -177,7 +177,9 @@ defmodule ChesstrainerWeb.EndgameLive.PlayTest do
describe "tablebase availability" do
test "blocks the attempt when the tablebase is cooling down", %{conn: conn} do
endgame = endgame_fixture()
# Six pieces routes to Lichess, so the cooldown applies (five or fewer
# would be answered locally).
endgame = endgame_fixture(fen: "8/8/5p2/1P1K1k2/8/2r5/8/7R w - - 0 1")
TablebaseCache.delete(FEN.position_key(endgame.fen))
Ratelimiters.lichess_add_cooldown()
on_exit(fn -> Ratelimiters.lichess_reset_cooldown() end)