mirror of
https://github.com/TheRealOwenRees/chess-endgame-trainer.git
synced 2026-11-18 03:32:06 +00:00
* Document the remote shell for production user management * add sitemap to footer links * remove border under theme selection * Add admin theme and user management (#144) Add /admin/themes CRUD and /admin/users list/detail with role and plan editing, guarded by a shared Policy module. Include last-super-admin and self-role protections plus structured id-only admin logs. Also add admin invitations, confirmation resends and password-reset emails, which work in development through the Swoosh mailbox before SES is configured. * Rate-limit password logins and emit auth events (#142) Add a two-bucket login throttle (5 failures per login, 10 per IP in a 15-minute window) and a shared client-address helper for controllers and LiveViews. Blocked attempts skip the database and bcrypt. Log auth.login.failed and auth.login.throttled warnings with boot-scoped HMAC addresses only, and stop repeated confirmation emails on unconfirmed login attempts. * Harden the production login edge for Cloudflare Document the layered login defence: the app throttle is primary, Caddy must pass Cloudflare's CF-Connecting-IP to the app, and Cloudflare edge rate limiting covers request floods. Update the reference Caddyfile to accept only Cloudflare ranges and reject direct-to-origin traffic, and document the exact Cloudflare dashboard rules. * Document 10-second Cloudflare rate-limit window Cloudflare may only offer a 10-second period. Keep the edge rule at 10 login POSTs per 10 seconds per IP and rely on the app's 15-minute login/address throttle for credential guessing. * Document Cloudflare limits for LiveView registration Registration submits over the shared LiveView socket, so a Cloudflare rule on POST /users/register will not count account creations. Note the app-level throttle/Turnstile options instead. * Document current Cloudflare free-plan edge controls Record the production setup: Cloudflare-only Caddy origins, Bot Fight Mode, and the 10-second login flood rule (10 POSTs per 10 seconds, 10-second block). Clarify that the application's 15-minute throttle remains the real credential lockout. * Auto-refresh Cloudflare CIDRs for Caddy Move the origin allowlist into a generated cloudflare-ranges.caddy imported by Caddy. Add an hourly systemd timer and script that fetch Cloudflare's v4/v6 lists, reject malformed responses, validate the candidate Caddyfile, and reload Caddy only when the ranges change. Restore the previous file on validation or reload failure. * Document Cloudflare CIDR refresher setup and operations Record the installed paths, permissions, timer checks, manual run, update procedure, and the direct-on-VPS setup fallback when the deploy directory is absent.
224 lines
8.0 KiB
Markdown
224 lines
8.0 KiB
Markdown
# Logging
|
|
|
|
Production writes one JSON object per line to standard output, which systemd
|
|
captures in journald. Development and test keep the readable
|
|
`[level] message` formatter.
|
|
|
|
## Configuration
|
|
|
|
`config/prod.exs` swaps the default handler's formatter for
|
|
`Chesstrainer.JSONLogFormatter`; nothing else changes per environment. The
|
|
formatter adds no runtime dependency beyond `Jason`, which the app already uses
|
|
for Phoenix JSON.
|
|
|
|
## Line shape
|
|
|
|
```json
|
|
{
|
|
"timestamp": "2026-11-10T21:36:28.489000Z",
|
|
"level": "error",
|
|
"message": "GET /endgames 500 in 12.5ms",
|
|
"request_id": "F7x...",
|
|
"method": "GET",
|
|
"path": "/endgames",
|
|
"status": 500,
|
|
"duration_ms": 12.5,
|
|
"error": {
|
|
"kind": "RuntimeError",
|
|
"reason": "bad thing",
|
|
"stacktrace": ["lib/foo.ex:10: Foo.bar/1"]
|
|
}
|
|
}
|
|
```
|
|
|
|
`timestamp`, `level` and `message` are always present. Metadata keys are
|
|
flattened into the object; reserved internal keys (`time`, `gl`, `report_cb`,
|
|
`error_logger`, `exception`, `stacktrace`, `crash_reason`, `reason`, `kind`)
|
|
are either unpacked into the line or omitted. Fixed fields win over metadata
|
|
with the same name.
|
|
|
|
| Field | When | Meaning |
|
|
|---|---|---|
|
|
| `timestamp` | always | ISO-8601 UTC, microsecond precision |
|
|
| `level` | always | `debug`, `info`, `notice`, `warning`, `error`, … |
|
|
| `message` | always | Rendered log message, truncated to 8 KiB |
|
|
| `request_id` | request-scoped logs | `Plug.RequestId` value, including LiveView processes |
|
|
| `method`, `path`, `status`, `duration_ms` | access lines | One HTTP request; no query string or body |
|
|
| `error.kind` | exceptions/crashes | Exception module, or the crash reason's kind |
|
|
| `error.reason` | exceptions/crashes | `Exception.message/1` or an inspected reason |
|
|
| `error.stacktrace` | exceptions/crashes | Array of formatted frames, capped at 50 |
|
|
| everything else | as supplied | Flattened atom metadata, sanitised and redacted |
|
|
|
|
Exceptions (`Logger.error(..., exception: ..., stacktrace: ...)`) and OTP crash
|
|
reports (`meta.crash_reason`, including Bandit request crashes and GenServer
|
|
terminations) become the `error` object. Stacktrace frames are an array of
|
|
strings so `jq -r '.error.stacktrace[]'` prints them readably.
|
|
|
|
## Redaction
|
|
|
|
The formatter redacts:
|
|
|
|
- metadata keys named `email`, `*_email`, `email_address`, `contact_email` or
|
|
`recipient`;
|
|
- any email-shaped string anywhere in metadata, nested `params`, or the rendered
|
|
message;
|
|
- credentials in `password`, `secret`, `token`, `authorization`, `cookie`,
|
|
`api_key`, `private_key` and `credential` keys;
|
|
- `token=...`/`password=...` assignments in messages.
|
|
|
|
Path segments of 20 or more URL-safe characters (reset/confirmation tokens,
|
|
UUIDs) are replaced with `[REDACTED]` in access lines. Query parameters and
|
|
request bodies are never logged.
|
|
|
|
## Access lines
|
|
|
|
`ChesstrainerWeb.RequestLogger` listens to `[:phoenix, :endpoint, :stop]` and
|
|
emits one `:info` line per HTTP request that reaches the endpoint, including a
|
|
LiveView's initial request. Phoenix's own endpoint start/stop pair is disabled
|
|
with `log: false` in `ChesstrainerWeb.Endpoint` so there is exactly one line.
|
|
|
|
`ChesstrainerWeb.Plugs.PutRequestId` copies `Plug.RequestId`'s id into the
|
|
session, and `UserAuth.mount_current_scope/2` puts it into `Logger.metadata/1`,
|
|
so logs from the LiveView process keep the id after the HTTP request ends.
|
|
|
|
## Writing logs
|
|
|
|
Normal `Logger` calls need no changes. Atom metadata keys are flattened into the
|
|
JSON object:
|
|
|
|
```elixir
|
|
require Logger
|
|
|
|
Logger.info("tablebase cache miss", table: :tablebase, key: key)
|
|
Logger.warning("tablebase fetch failed", status: 503, reason: reason)
|
|
```
|
|
|
|
For a caught exception, pass the exception and stacktrace so the formatter can
|
|
build the `error` object:
|
|
|
|
```elixir
|
|
Logger.error("cache backup failed", exception: exception, stacktrace: __STACKTRACE__)
|
|
```
|
|
|
|
For background work, wrap the body so `{:error, reason}` returns, throws, exits
|
|
and crashes all produce a log line:
|
|
|
|
```elixir
|
|
Task.Supervisor.start_child(Chesstrainer.Cache.BackupSupervisor, fn ->
|
|
Chesstrainer.Background.run("send digest", [user_id: user.id], fn ->
|
|
send_digest(user)
|
|
end)
|
|
end)
|
|
```
|
|
|
|
Prefer identifiers such as `user_id` or `table` over emails, and never log
|
|
params or request bodies. The formatter is a safety net, not a licence to log
|
|
PII.
|
|
|
|
## Authentication events
|
|
|
|
`ChesstrainerWeb.UserSessionController` emits warning lines consumed by
|
|
host-level tooling such as fail2ban:
|
|
|
|
| Message | Metadata | Meaning |
|
|
|---|---|---|
|
|
| `auth.login.failed` | `reason` (`invalid_credentials` / `unconfirmed`), `ip` | A password login failed. |
|
|
| `auth.login.throttled` | `reason` (`ip` / `login`), `ip` | The address or login bucket blocked the attempt before bcrypt. |
|
|
|
|
`ip` is a boot-scoped HMAC of the client address, not the address itself. The
|
|
raw login identifier and password are never included. The login throttle is
|
|
configured in `config/config.exs` and defaults to five failures per login and
|
|
ten per address in a fifteen-minute window.
|
|
|
|
## Querying logs
|
|
|
|
`journalctl` prints the raw line only with `-o cat`; the default output wraps it
|
|
in journald fields. Install `jq` once on the VPS with
|
|
`sudo apt-get install -y jq`, then:
|
|
|
|
```sh
|
|
# Pretty-print recent lines
|
|
sudo journalctl -u chesstrainer -o cat -n 50 --no-pager | jq .
|
|
|
|
# Errors only
|
|
sudo journalctl -u chesstrainer -o cat --since "1 hour ago" \
|
|
| jq 'select(.level == "error")'
|
|
|
|
# Everything from one request id
|
|
sudo journalctl -u chesstrainer -o cat \
|
|
| jq 'select(.request_id == "REQUEST_ID")'
|
|
|
|
# Crash reason and stacktrace
|
|
sudo journalctl -u chesstrainer -o cat \
|
|
| jq -r 'select(.error) | "\(.timestamp) \(.error.kind): \(.error.reason)\n\((.error.stacktrace // []) | join("\n"))"'
|
|
|
|
# Slow requests
|
|
sudo journalctl -u chesstrainer -o cat \
|
|
| jq 'select(.duration_ms > 1000) | {timestamp, method, path, status, duration_ms}'
|
|
|
|
# Most common error kinds in the current boot
|
|
sudo journalctl -u chesstrainer -o cat -b \
|
|
| jq -r 'select(.error.kind) | .error.kind' | sort | uniq -c | sort -rn
|
|
```
|
|
|
|
`Plug.RequestId` also returns the id in the `x-request-id` response header, so a
|
|
user can report the id from a failed request and you can filter straight to it.
|
|
|
|
## Local checks
|
|
|
|
The formatter tests run without a production release:
|
|
|
|
```sh
|
|
mix test test/chesstrainer/json_log_formatter_test.exs \
|
|
test/chesstrainer/background_test.exs \
|
|
test/chesstrainer_web/request_logger_test.exs
|
|
```
|
|
|
|
To render one event without touching the Logger configuration:
|
|
|
|
```sh
|
|
mix run --no-start -e '
|
|
event = %{
|
|
level: :error,
|
|
msg: {:string, "boom"},
|
|
meta: %{
|
|
time: System.system_time(:microsecond),
|
|
request_id: "req-1",
|
|
email: "user@example.com",
|
|
exception: %RuntimeError{message: "bad"},
|
|
stacktrace: [{Foo, :bar, 1, [file: ~c"lib/foo.ex", line: 10]}]
|
|
}
|
|
}
|
|
IO.puts(Chesstrainer.JSONLogFormatter.format(event, %{}))
|
|
'
|
|
```
|
|
|
|
That prints one JSON line with the email redacted and the exception unpacked
|
|
into `error`.
|
|
|
|
## Background work audit (#107)
|
|
|
|
Checked:
|
|
|
|
- `Chesstrainer.Cache` scheduled backup: `Task.Supervisor.start_child` ignored
|
|
the task's return value and `Persist.backup/1` returns `{:error, reason}`
|
|
without logging. Now wrapped in `Chesstrainer.Background.run/3`, which logs
|
|
`{:error, reason}` returns and crashes (with exception, reason and
|
|
stacktrace). A failure to start the task is logged too.
|
|
- `Chesstrainer.Cache.Persist.restore/1`: an unreadable backup was silently
|
|
skipped. It now logs a warning with the reason and starts with an empty cache.
|
|
- `Chesstrainer.Endgames.Tablebase.Parser`: Lichess 5xx and network failures
|
|
were returned to the caller (which shows a flash) but never logged. They now
|
|
log a warning with the status or reason. 429 cooldowns stay quiet because they
|
|
are expected.
|
|
- `Mix.Tasks.Tablebase.Fetch`: already aborts loudly with `Mix.raise` on HTTP or
|
|
network failure; no change.
|
|
- Email delivery: sends are synchronous and their error is returned to the
|
|
controller, so nothing is silent today. Future background sends should use
|
|
`Chesstrainer.Background.run/3`.
|
|
|
|
## Reading logs on the VPS
|
|
|
|
See the "Reading production logs" and "Keep the journal" sections in
|
|
[`deploy/README.md`](../deploy/README.md).
|