diff --git a/README.md b/README.md index 6ee22f9..b551423 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,14 @@ restart the service; the value is read at boot. See [docs/tablebases.md](docs/tablebases.md) for the full reference — dev vs VPS paths, `mix tablebase.check`, and boot logging. +## Logging + +Production logs are one JSON object per line so they can be filtered with +`journalctl` and `jq`; development and test keep the readable formatter. See +[docs/logging.md](docs/logging.md) for the line shape, redaction rules and the +background-work audit, and [`deploy/README.md`](deploy/README.md) for the VPS +runbook. + ## Learn more * Official website: https://www.phoenixframework.org/ diff --git a/config/prod.exs b/config/prod.exs index fa7a5a9..a1e7fdc 100644 --- a/config/prod.exs +++ b/config/prod.exs @@ -33,5 +33,10 @@ config :swoosh, local: false # Do not print debug messages in production config :logger, level: :info +# Emit one JSON object per line in production so journald output can be +# filtered and shipped without parsing prose. Dev and test keep the readable +# default formatter configured in config.exs. +config :logger, :default_handler, formatter: {Chesstrainer.JSONLogFormatter, %{}} + # Runtime production configuration, including reading # of environment variables, is done on config/runtime.exs. diff --git a/deploy/README.md b/deploy/README.md index 52eebd4..8772c5b 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -310,6 +310,121 @@ sudo docker compose -f pull chesstrainer-db sudo docker compose -f up -d chesstrainer-db ``` +## Read-only database access + +The app connects as `chesstrainer`, which is a superuser. Use a separate +read-only role for inspection, debugging, and dashboards. + +Create it once per environment: + +```sh +sudo docker compose -f exec -T chesstrainer-db \ + psql -U chesstrainer -d chesstrainer_prod <<'SQL' +CREATE ROLE chesstrainer_ro LOGIN PASSWORD ''; +GRANT CONNECT ON DATABASE chesstrainer_prod TO chesstrainer_ro; +GRANT USAGE ON SCHEMA public TO chesstrainer_ro; +GRANT SELECT ON ALL TABLES IN SCHEMA public TO chesstrainer_ro; + +-- Cover tables created by future migrations. Migrations run as chesstrainer. +ALTER DEFAULT PRIVILEGES FOR ROLE chesstrainer IN SCHEMA public + GRANT SELECT ON TABLES TO chesstrainer_ro; + +-- Optional: read other sessions and server settings in pg_stat_* views. +GRANT pg_read_all_stats TO chesstrainer_ro; +SQL +``` + +Generate the password with `openssl rand -hex 32` and keep it in a password +manager. The only access it has is `SELECT`; there are no `INSERT`, `UPDATE`, +`DELETE`, or object-creation grants. If the role already exists, skip the +`CREATE ROLE` line and re-run the grants. + +Optional defence in depth: + +```sql +ALTER ROLE chesstrainer_ro SET default_transaction_read_only = on; +``` + +This is not a hard boundary (the role can override it per session), but it +turns an accidental write into an error. + +### Connect from the VPS + +`docker exec` connects over the container's Unix socket, which the Postgres +image trusts locally, so no password is needed: + +```sh +sudo docker exec -it chesstrainer-db \ + psql -U chesstrainer_ro -d chesstrainer_prod +``` + +### Connect from a workstation over SSH + +Postgres is published on `127.0.0.1:5432` only. OpenSSH also disables +forwarding by default in `/etc/ssh/sshd_config` (`AllowTcpForwarding no`). +Allow local forwarding for the deploy user only, restricted to Postgres, by +appending this to the **end** of that file: + +```text +Match User + AllowTcpForwarding local + PermitOpen 127.0.0.1:5432 +``` + +Then validate and reload: + +```sh +sudo sshd -t +sudo systemctl reload ssh +sudo sshd -T -C user=,host=localhost,addr=127.0.0.1 | grep -iE 'allowtcpforwarding|permitopen' +``` + +Open the tunnel (local port `5433` because the dev database already binds +`5432`): + +```sh +ssh -N -L 5433:127.0.0.1:5432 @ +``` + +Connect from a second terminal: + +```sh +psql "postgres://chesstrainer_ro@127.0.0.1:5433/chesstrainer_prod" +``` + +Install `postgresql-client` if `psql` is not available on the workstation. + +To avoid the password prompt, add this to `~/.pgpass` (mode `0600`): + +```text +127.0.0.1:5433:chesstrainer_prod:chesstrainer_ro: +``` + +DBeaver uses the same tunnel. Create a PostgreSQL connection with host +`127.0.0.1`, port `5432`, database `chesstrainer_prod`, user +`chesstrainer_ro`, then on the **SSH** tab enable the tunnel with host +``, port `22`, user ``, and a public key. The +`AllowTcpForwarding local` rule above is required for any tunnelled client. + +Reset the role password: + +```sh +sudo docker compose -f exec -T chesstrainer-db \ + psql -U chesstrainer -d chesstrainer_prod \ + -c "ALTER ROLE chesstrainer_ro PASSWORD '';" +``` + +Verify the role is read-only: + +```sh +psql "postgres://chesstrainer_ro@127.0.0.1:5433/chesstrainer_prod" \ + -c 'select current_user, current_database();' +psql "postgres://chesstrainer_ro@127.0.0.1:5433/chesstrainer_prod" \ + -c 'create table should_fail (i int);' +``` + +The second command must fail with `permission denied for schema public`. + ## systemd unit From the deploy artifact copy: @@ -438,7 +553,7 @@ sudo -E /opt/chesstrainer/src/deploy/activate /tmp/chesstrainer-release.tar.gz ## GitHub Actions CD Merging to `main` (or running the workflow manually) tests, builds, uploads, -and deploys the release. The deploy job uses SSH as `owen`. +and deploys the release. The deploy job uses SSH as ``. ### Deploy key @@ -447,7 +562,7 @@ Generate a dedicated key, not your personal one: ```sh ssh-keygen -t ed25519 -C "github-actions-chesstrainer" \ -f ~/.ssh/chesstrainer_deploy -N "" -ssh-copy-id -i ~/.ssh/chesstrainer_deploy.pub owen@ +ssh-copy-id -i ~/.ssh/chesstrainer_deploy.pub @ ``` `VPS_HOST` must be the public IP or DNS name reachable from GitHub. Do not use @@ -460,7 +575,7 @@ sudo visudo -f /etc/sudoers.d/chesstrainer-deploy ``` ```text -owen ALL=(root) NOPASSWD: /opt/chesstrainer/src/deploy/activate, /opt/chesstrainer/src/deploy/rollback + ALL=(root) NOPASSWD: /opt/chesstrainer/src/deploy/activate, /opt/chesstrainer/src/deploy/rollback ``` Validate: @@ -481,7 +596,7 @@ add environment secrets: | Secret | Value | |---|---| | `VPS_HOST` | public VPS IP or DNS name | -| `VPS_USER` | `owen` | +| `VPS_USER` | `` | | `VPS_SSH_KEY` | private deploy key | | `VPS_KNOWN_HOSTS` | `ssh-keyscan -H ` output | @@ -533,24 +648,100 @@ curl -i http://127.0.0.1:4100/health It returns 200 for `ok`/`degraded` and 503 for `down`. A missing local tablebase reports `degraded` but still 200 because Lichess fallback is available. -## Backups +### Reading production logs -Take a logical backup of the production database: +Production emits one JSON object per line. Use `-o cat` so journald prints the +raw line instead of its own field wrapper, then pipe to `jq`: ```sh -sudo docker compose -f \ - exec -T chesstrainer-db pg_dump -U chesstrainer chesstrainer_prod \ +# Recent logs, pretty-printed +sudo journalctl -u chesstrainer -o cat -n 50 --no-pager | jq . + +# Errors only +sudo journalctl -u chesstrainer -o cat --since "1 hour ago" | jq 'select(.level == "error")' + +# Everything from one request +sudo journalctl -u chesstrainer -o cat | jq 'select(.request_id == "REQUEST_ID")' + +# Crash reasons and stacktraces +sudo journalctl -u chesstrainer -o cat | jq -r 'select(.error) | .error.reason, (.error.stacktrace[]?)' +``` + +Each line has `timestamp`, `level` and `message`, plus `request_id` when one +exists. Access lines add `method`, `path`, `status` and `duration_ms`; crashes +add an `error` object with `kind`, `reason` and `stacktrace`. Query parameters +and request bodies are never logged, token-shaped path segments are redacted, +and email addresses or credentials in metadata render as `[REDACTED]`. + +The full field table, redaction rules and examples for adding background logs +are in [`docs/logging.md`](../docs/logging.md). + +Dev and test still use the readable `[level] message` formatter. + +### Keep the journal + +By default some distributions keep journald logs in memory only. On the VPS, +make sure they survive a reboot and have a retention budget: + +```sh +sudo mkdir -p /var/log/journal +sudo systemctl restart systemd-journald +journalctl --disk-usage +``` + +Set `Storage=persistent` and a `SystemMaxUse=` value in +`/etc/systemd/journald.conf` if the distribution does not already. journald +rate-limits bursts; if a crash storm drops lines, raise `RateLimitBurst` or +`RateLimitIntervalSec` there. + +## Backups + +`docker exec` connects over the container's Unix socket, which the Postgres +image trusts locally, so `pg_dump` needs no password. Run it as the owner +(`chesstrainer`), not `chesstrainer_ro`: a full dump also needs sequence access +and ownership information. + +Take a compressed logical backup (custom format, restorable with `pg_restore`): + +```sh +sudo docker compose -f exec -T chesstrainer-db \ + pg_dump -U chesstrainer -d chesstrainer_prod -Fc \ + > /var/backups/chesstrainer-$(date -u +%Y%m%d%H%M%S).dump +``` + +For a plain SQL file instead (human-readable, restorable with `psql`): + +```sh +sudo docker compose -f exec -T chesstrainer-db \ + pg_dump -U chesstrainer -d chesstrainer_prod \ | gzip > /var/backups/chesstrainer-$(date -u +%Y%m%d%H%M%S).sql.gz ``` +Check that a custom-format dump is readable before trusting it: + +```sh +sudo docker exec -i chesstrainer-db pg_restore -l \ + < /var/backups/.dump > /dev/null +``` + +Restore into a scratch database (do not restore over the live database without +stopping `chesstrainer` first): + +```sh +sudo docker compose -f exec -T chesstrainer-db \ + createdb -U chesstrainer chesstrainer_restore +sudo docker exec -i chesstrainer-db pg_restore -U chesstrainer \ + -d chesstrainer_restore < /var/backups/.dump +``` + Example nightly entry in root's crontab (`sudo crontab -e`): ```cron -15 3 * * * docker compose -f /home/owen/docker-compose.yml exec -T chesstrainer-db pg_dump -U chesstrainer chesstrainer_prod | gzip > /var/backups/chesstrainer-$(date +\%Y\%m\%d).sql.gz +15 3 * * * docker compose -f /home//docker-compose.yml exec -T chesstrainer-db pg_dump -U chesstrainer -d chesstrainer_prod -Fc > /var/backups/chesstrainer-$(date +\%Y\%m\%d).dump ``` Keep `/var/backups` outside `/opt/chesstrainer/releases` and copy backups off -the VPS regularly. +the VPS regularly. A backup that is only on the VPS is not a backup. ## New VPS checklist @@ -564,4 +755,4 @@ the VPS regularly. 7. `/health` and `/` verified on `127.0.0.1:4100`. 8. Caddy block added, validated, reloaded. 9. Public `/`, `/privacy`, `/terms-conditions`, and `/health` verified. -10. Backups scheduled and tested. +10. Read-only role created, tunnel verified, backups scheduled and tested. diff --git a/docs/logging.md b/docs/logging.md new file mode 100644 index 0000000..22a9fc5 --- /dev/null +++ b/docs/logging.md @@ -0,0 +1,208 @@ +# Logging + +Production writes one JSON object per line to standard output, which systemd +captures in journald. Development and test keep the readable +`[level] message` formatter. + +## Configuration + +`config/prod.exs` swaps the default handler's formatter for +`Chesstrainer.JSONLogFormatter`; nothing else changes per environment. The +formatter adds no runtime dependency beyond `Jason`, which the app already uses +for Phoenix JSON. + +## Line shape + +```json +{ + "timestamp": "2026-11-10T21:36:28.489000Z", + "level": "error", + "message": "GET /endgames 500 in 12.5ms", + "request_id": "F7x...", + "method": "GET", + "path": "/endgames", + "status": 500, + "duration_ms": 12.5, + "error": { + "kind": "RuntimeError", + "reason": "bad thing", + "stacktrace": ["lib/foo.ex:10: Foo.bar/1"] + } +} +``` + +`timestamp`, `level` and `message` are always present. Metadata keys are +flattened into the object; reserved internal keys (`time`, `gl`, `report_cb`, +`error_logger`, `exception`, `stacktrace`, `crash_reason`, `reason`, `kind`) +are either unpacked into the line or omitted. Fixed fields win over metadata +with the same name. + +| Field | When | Meaning | +|---|---|---| +| `timestamp` | always | ISO-8601 UTC, microsecond precision | +| `level` | always | `debug`, `info`, `notice`, `warning`, `error`, … | +| `message` | always | Rendered log message, truncated to 8 KiB | +| `request_id` | request-scoped logs | `Plug.RequestId` value, including LiveView processes | +| `method`, `path`, `status`, `duration_ms` | access lines | One HTTP request; no query string or body | +| `error.kind` | exceptions/crashes | Exception module, or the crash reason's kind | +| `error.reason` | exceptions/crashes | `Exception.message/1` or an inspected reason | +| `error.stacktrace` | exceptions/crashes | Array of formatted frames, capped at 50 | +| everything else | as supplied | Flattened atom metadata, sanitised and redacted | + +Exceptions (`Logger.error(..., exception: ..., stacktrace: ...)`) and OTP crash +reports (`meta.crash_reason`, including Bandit request crashes and GenServer +terminations) become the `error` object. Stacktrace frames are an array of +strings so `jq -r '.error.stacktrace[]'` prints them readably. + +## Redaction + +The formatter redacts: + +- metadata keys named `email`, `*_email`, `email_address`, `contact_email` or + `recipient`; +- any email-shaped string anywhere in metadata, nested `params`, or the rendered + message; +- credentials in `password`, `secret`, `token`, `authorization`, `cookie`, + `api_key`, `private_key` and `credential` keys; +- `token=...`/`password=...` assignments in messages. + +Path segments of 20 or more URL-safe characters (reset/confirmation tokens, +UUIDs) are replaced with `[REDACTED]` in access lines. Query parameters and +request bodies are never logged. + +## Access lines + +`ChesstrainerWeb.RequestLogger` listens to `[:phoenix, :endpoint, :stop]` and +emits one `:info` line per HTTP request that reaches the endpoint, including a +LiveView's initial request. Phoenix's own endpoint start/stop pair is disabled +with `log: false` in `ChesstrainerWeb.Endpoint` so there is exactly one line. + +`ChesstrainerWeb.Plugs.PutRequestId` copies `Plug.RequestId`'s id into the +session, and `UserAuth.mount_current_scope/2` puts it into `Logger.metadata/1`, +so logs from the LiveView process keep the id after the HTTP request ends. + +## Writing logs + +Normal `Logger` calls need no changes. Atom metadata keys are flattened into the +JSON object: + +```elixir +require Logger + +Logger.info("tablebase cache miss", table: :tablebase, key: key) +Logger.warning("tablebase fetch failed", status: 503, reason: reason) +``` + +For a caught exception, pass the exception and stacktrace so the formatter can +build the `error` object: + +```elixir +Logger.error("cache backup failed", exception: exception, stacktrace: __STACKTRACE__) +``` + +For background work, wrap the body so `{:error, reason}` returns, throws, exits +and crashes all produce a log line: + +```elixir +Task.Supervisor.start_child(Chesstrainer.Cache.BackupSupervisor, fn -> + Chesstrainer.Background.run("send digest", [user_id: user.id], fn -> + send_digest(user) + end) +end) +``` + +Prefer identifiers such as `user_id` or `table` over emails, and never log +params or request bodies. The formatter is a safety net, not a licence to log +PII. + +## Querying logs + +`journalctl` prints the raw line only with `-o cat`; the default output wraps it +in journald fields. Install `jq` once on the VPS with +`sudo apt-get install -y jq`, then: + +```sh +# Pretty-print recent lines +sudo journalctl -u chesstrainer -o cat -n 50 --no-pager | jq . + +# Errors only +sudo journalctl -u chesstrainer -o cat --since "1 hour ago" \ + | jq 'select(.level == "error")' + +# Everything from one request id +sudo journalctl -u chesstrainer -o cat \ + | jq 'select(.request_id == "REQUEST_ID")' + +# Crash reason and stacktrace +sudo journalctl -u chesstrainer -o cat \ + | jq -r 'select(.error) | "\(.timestamp) \(.error.kind): \(.error.reason)\n\((.error.stacktrace // []) | join("\n"))"' + +# Slow requests +sudo journalctl -u chesstrainer -o cat \ + | jq 'select(.duration_ms > 1000) | {timestamp, method, path, status, duration_ms}' + +# Most common error kinds in the current boot +sudo journalctl -u chesstrainer -o cat -b \ + | jq -r 'select(.error.kind) | .error.kind' | sort | uniq -c | sort -rn +``` + +`Plug.RequestId` also returns the id in the `x-request-id` response header, so a +user can report the id from a failed request and you can filter straight to it. + +## Local checks + +The formatter tests run without a production release: + +```sh +mix test test/chesstrainer/json_log_formatter_test.exs \ + test/chesstrainer/background_test.exs \ + test/chesstrainer_web/request_logger_test.exs +``` + +To render one event without touching the Logger configuration: + +```sh +mix run --no-start -e ' +event = %{ + level: :error, + msg: {:string, "boom"}, + meta: %{ + time: System.system_time(:microsecond), + request_id: "req-1", + email: "user@example.com", + exception: %RuntimeError{message: "bad"}, + stacktrace: [{Foo, :bar, 1, [file: ~c"lib/foo.ex", line: 10]}] + } +} +IO.puts(Chesstrainer.JSONLogFormatter.format(event, %{})) +' +``` + +That prints one JSON line with the email redacted and the exception unpacked +into `error`. + +## Background work audit (#107) + +Checked: + +- `Chesstrainer.Cache` scheduled backup: `Task.Supervisor.start_child` ignored + the task's return value and `Persist.backup/1` returns `{:error, reason}` + without logging. Now wrapped in `Chesstrainer.Background.run/3`, which logs + `{:error, reason}` returns and crashes (with exception, reason and + stacktrace). A failure to start the task is logged too. +- `Chesstrainer.Cache.Persist.restore/1`: an unreadable backup was silently + skipped. It now logs a warning with the reason and starts with an empty cache. +- `Chesstrainer.Endgames.Tablebase.Parser`: Lichess 5xx and network failures + were returned to the caller (which shows a flash) but never logged. They now + log a warning with the status or reason. 429 cooldowns stay quiet because they + are expected. +- `Mix.Tasks.Tablebase.Fetch`: already aborts loudly with `Mix.raise` on HTTP or + network failure; no change. +- Email delivery: sends are synchronous and their error is returned to the + controller, so nothing is silent today. Future background sends should use + `Chesstrainer.Background.run/3`. + +## Reading logs on the VPS + +See the "Reading production logs" and "Keep the journal" sections in +[`deploy/README.md`](../deploy/README.md). diff --git a/lib/chesstrainer/background.ex b/lib/chesstrainer/background.ex new file mode 100644 index 0000000..128b6c8 --- /dev/null +++ b/lib/chesstrainer/background.ex @@ -0,0 +1,63 @@ +defmodule Chesstrainer.Background do + @moduledoc """ + Runs background work and logs any failure. + + Background failures otherwise disappear: a task exits, the supervisor keeps + the rest of the system running, and nobody sees why the work did not happen. + This wrapper turns crashes, throws, exits and `{:error, reason}` returns into + one structured error line with the reason and stacktrace. + + The label is a short description of the work; metadata is added to the log + line unchanged (minus the formatter's own redaction). + """ + + require Logger + + @type result :: term() + + @doc """ + Runs `fun` and logs a failure. Returns the function's value, or `:error` when + it crashed or returned an error tuple. + """ + @spec run(String.t(), (-> result)) :: result() | :error + def run(label, fun), do: run(label, [], fun) + + @doc """ + Like `run/2`, with extra metadata for the error line. + """ + @spec run(String.t(), keyword(), (-> result)) :: result() | :error + def run(label, metadata, fun) + when is_binary(label) and is_list(metadata) and is_function(fun, 0) do + case fun.() do + {:error, reason} -> + Logger.error( + "#{label} failed: #{format_term(reason)}", + Keyword.merge(metadata, reason: reason) + ) + + :error + + result -> + result + end + rescue + exception -> + Logger.error( + "#{label} crashed: #{Exception.message(exception)}", + Keyword.merge(metadata, exception: exception, stacktrace: __STACKTRACE__) + ) + + :error + catch + kind, reason -> + Logger.error( + "#{label} crashed: #{format_term(reason)}", + Keyword.merge(metadata, kind: kind, reason: reason, stacktrace: __STACKTRACE__) + ) + + :error + end + + defp format_term(term) when is_binary(term), do: term + defp format_term(term), do: inspect(term, limit: 10, printable_limit: 500) +end diff --git a/lib/chesstrainer/cache.ex b/lib/chesstrainer/cache.ex index ab4f4ec..acd4416 100644 --- a/lib/chesstrainer/cache.ex +++ b/lib/chesstrainer/cache.ex @@ -11,7 +11,8 @@ defmodule Chesstrainer.Cache do that were used heavily once but have not been used since eventually age out. A daily backup runs off the cache process at a configurable UTC time, so reads - and writes stay responsive while the snapshot is written. + and writes stay responsive while the snapshot is written. Failures are logged + rather than lost. ## Examples @@ -19,7 +20,9 @@ defmodule Chesstrainer.Cache do """ use GenServer + require Logger + alias Chesstrainer.Background alias Chesstrainer.Cache.BackupSchedule alias Chesstrainer.Cache.Persist @@ -121,7 +124,18 @@ defmodule Chesstrainer.Cache do table = state.table backup_fun = state.backup_fun - Task.Supervisor.start_child(Chesstrainer.Cache.BackupSupervisor, fn -> backup_fun.(table) end) + task = fn -> + Background.run("cache backup", [table: table], fn -> backup_fun.(table) end) + end + + case Task.Supervisor.start_child(Chesstrainer.Cache.BackupSupervisor, task) do + {:ok, _pid} -> + :ok + + {:error, reason} -> + Logger.error("cache backup could not start", table: table, reason: reason) + :ok + end end defp schedule_backup(state) do diff --git a/lib/chesstrainer/cache/persist.ex b/lib/chesstrainer/cache/persist.ex index 8ffa6f9..c0d16e4 100644 --- a/lib/chesstrainer/cache/persist.ex +++ b/lib/chesstrainer/cache/persist.ex @@ -5,8 +5,13 @@ defmodule Chesstrainer.Cache.Persist do A backup is written to a temporary DETS file in the same directory and then atomically renamed into place. A crash during the write therefore leaves the previous backup untouched instead of a truncated file that fails to restore. + + A backup that cannot be opened at boot is logged and skipped, so a damaged + file does not stop the application from starting with an empty cache. """ + require Logger + @backup_path "priv/static/backups" @format_version "v2" @@ -54,7 +59,8 @@ defmodule Chesstrainer.Cache.Persist do :dets.close(table) - {:error, _} -> + {:error, reason} -> + Logger.warning("cache backup restore failed", table: table, reason: reason) :ok end end diff --git a/lib/chesstrainer/endgames/tablebase/parser.ex b/lib/chesstrainer/endgames/tablebase/parser.ex index fcace1a..b760c58 100644 --- a/lib/chesstrainer/endgames/tablebase/parser.ex +++ b/lib/chesstrainer/endgames/tablebase/parser.ex @@ -11,6 +11,8 @@ defmodule Chesstrainer.Endgames.Tablebase.Parser do import Chesstrainer.Ratelimiters alias Chesstrainer.Endgames.Tablebase + require Logger + @lichess_tablebase_url "https://tablebase.lichess.ovh/standard?fen=" @spec from_fen(String.t()) :: @@ -74,6 +76,7 @@ defmodule Chesstrainer.Endgames.Tablebase.Parser do # Server errors {:ok, %Req.Response{status: status, body: body}} when status in 500..599 -> + Logger.warning("tablebase fetch failed with HTTP #{status}", status: status) {:error, {:server_error, status, body}} # Catch‑all for other status codes @@ -82,6 +85,7 @@ defmodule Chesstrainer.Endgames.Tablebase.Parser do # Network / client‑side failure {:error, reason} -> + Logger.warning("tablebase fetch failed: #{inspect(reason)}", reason: reason) {:error, reason} end end diff --git a/lib/chesstrainer/json_log_formatter.ex b/lib/chesstrainer/json_log_formatter.ex new file mode 100644 index 0000000..a02b777 --- /dev/null +++ b/lib/chesstrainer/json_log_formatter.ex @@ -0,0 +1,307 @@ +defmodule Chesstrainer.JSONLogFormatter do + @moduledoc """ + An OTP logger formatter that emits one JSON object per log line. + + Production uses this formatter (see `config/prod.exs`), while development and + test keep `Logger.Formatter`'s readable output. Every line carries a + timestamp, level and message, plus a request id when one is in the logger + metadata. Access lines add `method`, `path`, `status` and `duration_ms`; + exceptions and OTP crash reports add an `error` object with `kind`, `reason` + and `stacktrace`. + + The formatter is defensive. Metadata is sanitised into JSON-safe values, and + an event that cannot be encoded still produces a single JSON line instead of + taking the logger down. Email addresses and credentials found in metadata, + params or the rendered message are replaced with `[REDACTED]`. + """ + + @max_message_bytes 8_096 + @max_stacktrace_entries 50 + @redacted "[REDACTED]" + + @email_regex ~r/[A-Za-z0-9.!#$%&'*+\/=?^_`{|}~-]+@[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?(?:\.[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?)+/ + @secret_assignment_regex ~r/\b(token|password|secret|api[_-]?key|authorization)=([^&\s"']+)/i + + @sensitive_key_fragments ~w(password secret token authorization cookie api_key private_key credential) + @sensitive_email_keys ~w(email email_address user_email contact_email recipient) + + # These are unpacked into the line itself rather than copied as metadata. + @internal_keys [ + :time, + :gl, + :report_cb, + :error_logger, + :exception, + :stacktrace, + :crash_reason, + :reason, + :kind, + :conn, + :socket + ] + + # Fixed fields always win over a metadata key with the same name. + @reserved_keys [ + :timestamp, + :level, + :message, + :error, + :request_id, + :method, + :path, + :status, + :duration_ms + ] + + @doc """ + Formats a `:logger` event as a newline-terminated JSON object. + """ + @spec format(:logger.log_event(), :logger.formatter_config()) :: iodata() + def format(event, _config) do + event + |> build() + |> Jason.encode!() + |> then(&[&1, ?\n]) + rescue + exception -> fallback(exception) + catch + kind, reason -> fallback({kind, reason}) + end + + defp build(event) do + meta = event.meta || %{} + + %{ + "timestamp" => timestamp(meta), + "level" => to_string(event.level), + "message" => message(event) + } + |> put_access(meta) + |> put_metadata(meta) + |> put_error(meta) + end + + defp timestamp(%{time: time}) when is_integer(time) do + time + |> DateTime.from_unix!(:microsecond) + |> DateTime.to_iso8601() + end + + defp timestamp(_meta), do: DateTime.utc_now() |> DateTime.to_iso8601() + + defp message(event) do + event + |> Logger.Formatter.format_event(@max_message_bytes) + |> Logger.Formatter.prune() + |> IO.iodata_to_binary() + |> redact_text() + rescue + _exception -> + event.msg + |> inspect(limit: 20, printable_limit: @max_message_bytes) + |> redact_text() + catch + _kind, reason -> + {event.msg, reason} + |> inspect(limit: 20, printable_limit: @max_message_bytes) + |> redact_text() + end + + defp put_access(entry, meta) do + entry + |> put_if_present("request_id", meta[:request_id]) + |> put_if_present("method", meta[:method]) + |> put_if_present("path", meta[:path]) + |> put_if_present("status", meta[:status]) + |> put_if_present("duration_ms", meta[:duration_ms]) + end + + defp put_if_present(entry, _key, nil), do: entry + + defp put_if_present(entry, key, value) do + Map.put(entry, key, sanitize(value, key)) + end + + defp put_metadata(entry, meta) do + Enum.reduce(meta, entry, fn + {key, _value}, acc when key in @internal_keys -> + acc + + {key, _value}, acc when key in @reserved_keys -> + acc + + {key, value}, acc -> + Map.put(acc, key |> key_string() |> redact_text(), metadata_value(key, value)) + end) + end + + defp metadata_value(:mfa, {module, function, arity}) + when is_atom(module) and is_atom(function) and is_integer(arity) do + Exception.format_mfa(module, function, arity) + end + + defp metadata_value(key, value), do: sanitize(value, key) + + # Only the first exception/reason source is rendered. OTP crash reports carry + # `crash_reason`, while Logger.error(..., exception: ..., stacktrace: ...) + # carries the two keys directly. + defp put_error(entry, meta) do + case error(meta) do + nil -> entry + error when map_size(error) == 0 -> entry + error -> Map.put(entry, "error", error) + end + end + + defp error(%{exception: %{__exception__: true} = exception} = meta) do + %{ + "kind" => inspect(exception.__struct__), + "reason" => Exception.message(exception), + "stacktrace" => stacktrace(meta[:stacktrace]) + } + |> drop_nil() + end + + defp error(%{crash_reason: crash_reason} = meta) do + {reason, trace} = split_crash_reason(crash_reason, meta[:stacktrace]) + + %{ + "kind" => error_kind(reason), + "reason" => error_reason(reason), + "stacktrace" => stacktrace(trace) + } + |> drop_nil() + end + + defp error(meta) do + if meta[:reason] || meta[:stacktrace] do + %{ + "kind" => meta[:kind] && to_string(meta[:kind]), + "reason" => error_reason(meta[:reason]), + "stacktrace" => stacktrace(meta[:stacktrace]) + } + |> drop_nil() + end + end + + defp split_crash_reason({reason, trace}, _fallback) when is_list(trace), do: {reason, trace} + defp split_crash_reason(reason, fallback), do: {reason, fallback} + + defp error_kind(%{__exception__: true} = exception), do: inspect(exception.__struct__) + defp error_kind(reason) when is_atom(reason), do: Atom.to_string(reason) + defp error_kind(reason) when is_tuple(reason), do: reason |> elem(0) |> error_kind() + defp error_kind(_reason), do: nil + + defp error_reason(nil), do: nil + defp error_reason(%{__exception__: true} = exception), do: Exception.message(exception) + defp error_reason(reason) when is_binary(reason), do: redact_text(reason) + + defp error_reason(reason) do + reason + |> inspect(limit: 30, printable_limit: 2_000) + |> redact_text() + end + + defp stacktrace(stacktrace) when is_list(stacktrace) and stacktrace != [] do + stacktrace + |> Enum.take(@max_stacktrace_entries) + |> Enum.map(&format_stacktrace_entry/1) + |> Enum.map(&redact_text/1) + end + + defp stacktrace(_stacktrace), do: nil + + defp format_stacktrace_entry(entry) do + Exception.format_stacktrace_entry(entry) + rescue + _exception -> inspect(entry, limit: 10, printable_limit: 1_000) + end + + defp drop_nil(map), do: Map.reject(map, fn {_key, value} -> is_nil(value) end) + + defp key_string(key) when is_atom(key), do: Atom.to_string(key) + defp key_string(key) when is_binary(key), do: key + defp key_string(key), do: inspect(key) + + defp sanitize(nil, _key), do: nil + defp sanitize(value, _key) when is_boolean(value), do: value + defp sanitize(value, _key) when is_integer(value) or is_float(value), do: value + + defp sanitize(value, key) when is_binary(value) do + cond do + sensitive_key?(key) -> @redacted + String.valid?(value) -> redact_text(value) + true -> inspect(value) + end + end + + defp sanitize(value, _key) when is_atom(value), do: Atom.to_string(value) + + defp sanitize(value, _key) when is_struct(value) do + value + |> inspect(limit: 20, printable_limit: 2_000) + |> redact_text() + end + + defp sanitize(value, key) when is_list(value) do + cond do + sensitive_key?(key) -> @redacted + value != [] and List.ascii_printable?(value) -> value |> List.to_string() |> redact_text() + true -> Enum.map(value, &sanitize(&1, nil)) + end + end + + defp sanitize(value, _key) when is_tuple(value) do + value + |> Tuple.to_list() + |> Enum.map(&sanitize(&1, nil)) + end + + defp sanitize(value, _key) when is_map(value) do + Enum.into(value, %{}, fn {key, nested} -> + {key |> key_string() |> redact_text(), sanitize(nested, key)} + end) + end + + defp sanitize(value, _key) do + value + |> inspect(limit: 20, printable_limit: 2_000) + |> redact_text() + end + + defp sensitive_key?(nil), do: false + + defp sensitive_key?(key) do + key = key |> key_string() |> String.downcase() + + key in @sensitive_email_keys or String.ends_with?(key, "_email") or + Enum.any?(@sensitive_key_fragments, &String.contains?(key, &1)) + end + + defp redact_text(text) when is_binary(text) do + text = if String.valid?(text), do: text, else: inspect(text) + + text + |> then(&Regex.replace(@email_regex, &1, @redacted)) + |> then(&Regex.replace(@secret_assignment_regex, &1, "\\1=#{@redacted}")) + end + + defp fallback(reason) do + entry = %{ + "timestamp" => DateTime.utc_now() |> DateTime.to_iso8601(), + "level" => "error", + "message" => "could not format log event", + "error" => %{ + "reason" => reason |> inspect(limit: 10, printable_limit: 1_000) |> redact_text() + } + } + + json = + case Jason.encode(entry) do + {:ok, json} -> json + {:error, _reason} -> ~s({"level":"error","message":"could not format log event"}) + end + + [json, ?\n] + end +end diff --git a/lib/chesstrainer_web/endpoint.ex b/lib/chesstrainer_web/endpoint.ex index d94b5c8..c0927fd 100644 --- a/lib/chesstrainer_web/endpoint.ex +++ b/lib/chesstrainer_web/endpoint.ex @@ -41,7 +41,10 @@ defmodule ChesstrainerWeb.Endpoint do cookie_key: "request_logger" plug Plug.RequestId - plug Plug.Telemetry, event_prefix: [:phoenix, :endpoint] + + # Access lines come from ChesstrainerWeb.RequestLogger (one structured line + # per request); disable Phoenix.Logger's endpoint start/stop pair. + plug Plug.Telemetry, event_prefix: [:phoenix, :endpoint], log: false plug Plug.Parsers, parsers: [:urlencoded, :multipart, :json], diff --git a/lib/chesstrainer_web/plugs/put_request_id.ex b/lib/chesstrainer_web/plugs/put_request_id.ex new file mode 100644 index 0000000..08e5a11 --- /dev/null +++ b/lib/chesstrainer_web/plugs/put_request_id.ex @@ -0,0 +1,25 @@ +defmodule ChesstrainerWeb.Plugs.PutRequestId do + @moduledoc """ + Copies `Plug.RequestId`'s request id into the session. + + LiveView processes handle later events after the original HTTP request has + finished. Keeping the id in the session lets `UserAuth.mount_current_scope/2` + put it into `Logger.metadata/1`, so LiveView logs remain correlatable with the + request that mounted them. + """ + + @behaviour Plug + + import Plug.Conn + + @impl true + def init(opts), do: opts + + @impl true + def call(conn, _opts) do + case conn.assigns[:request_id] do + nil -> conn + request_id -> put_session(conn, :request_id, request_id) + end + end +end diff --git a/lib/chesstrainer_web/request_logger.ex b/lib/chesstrainer_web/request_logger.ex new file mode 100644 index 0000000..528523b --- /dev/null +++ b/lib/chesstrainer_web/request_logger.ex @@ -0,0 +1,66 @@ +defmodule ChesstrainerWeb.RequestLogger do + @moduledoc """ + Emits one structured access line per HTTP request. + + The line carries the method, path, status, duration in milliseconds and the + request id when one is available. Query parameters and bodies are never + logged. Path segments that look like tokens or identifiers (20 or more + URL-safe characters) are replaced with `[REDACTED]`. + + `Phoenix.Logger`'s own endpoint start/stop lines are disabled in + `ChesstrainerWeb.Endpoint` so each request produces exactly one line. + """ + + require Logger + + @event [:phoenix, :endpoint, :stop] + @handler_id {__MODULE__, :access_log} + @token_segment ~r/\A[A-Za-z0-9_.-]{20,}\z/ + + @doc false + def install do + case :telemetry.attach(@handler_id, @event, &__MODULE__.handle_event/4, :ok) do + :ok -> :ok + {:error, :already_exists} -> :ok + end + end + + @doc false + def handle_event(_event, measurements, metadata, _config) do + conn = metadata.conn + duration_ms = duration_ms(measurements.duration) + path = redact_path(conn.request_path) + status = conn.status || 0 + + metadata = + [ + method: conn.method, + path: path, + status: status, + duration_ms: duration_ms + ] + |> put_request_id(conn.assigns[:request_id]) + + Logger.info("#{conn.method} #{path} #{status} in #{duration_ms}ms", metadata) + end + + defp duration_ms(native) do + native + |> System.convert_time_unit(:native, :microsecond) + |> Kernel./(1000) + |> Float.round(1) + end + + defp put_request_id(metadata, nil), do: metadata + defp put_request_id(metadata, request_id), do: Keyword.put(metadata, :request_id, request_id) + + defp redact_path(path) do + path + |> String.split("/") + |> Enum.map_join("/", &redact_segment/1) + end + + defp redact_segment(segment) do + if Regex.match?(@token_segment, segment), do: "[REDACTED]", else: segment + end +end diff --git a/lib/chesstrainer_web/router.ex b/lib/chesstrainer_web/router.ex index 55bcfb9..2783f5b 100644 --- a/lib/chesstrainer_web/router.ex +++ b/lib/chesstrainer_web/router.ex @@ -6,6 +6,7 @@ defmodule ChesstrainerWeb.Router do pipeline :browser do plug :accepts, ["html"] plug :fetch_session + plug ChesstrainerWeb.Plugs.PutRequestId plug :fetch_live_flash plug :put_root_layout, html: {ChesstrainerWeb.Layouts, :root} plug :protect_from_forgery diff --git a/lib/chesstrainer_web/telemetry.ex b/lib/chesstrainer_web/telemetry.ex index 471bc97..0afad86 100644 --- a/lib/chesstrainer_web/telemetry.ex +++ b/lib/chesstrainer_web/telemetry.ex @@ -8,6 +8,8 @@ defmodule ChesstrainerWeb.Telemetry do @impl true def init(_arg) do + ChesstrainerWeb.RequestLogger.install() + children = [ # Telemetry poller will execute the given period measurements # every 10_000ms. Learn more here: https://telemetry-metrics.hexdocs.pm diff --git a/lib/chesstrainer_web/user_auth.ex b/lib/chesstrainer_web/user_auth.ex index 7e104c9..25f5d18 100644 --- a/lib/chesstrainer_web/user_auth.ex +++ b/lib/chesstrainer_web/user_auth.ex @@ -290,6 +290,10 @@ defmodule ChesstrainerWeb.UserAuth do end defp mount_current_scope(socket, session) do + if request_id = session["request_id"] do + Logger.metadata(request_id: request_id) + end + Phoenix.Component.assign_new(socket, :current_scope, fn -> {user, _} = if user_token = session["user_token"] do diff --git a/test/chesstrainer/background_test.exs b/test/chesstrainer/background_test.exs new file mode 100644 index 0000000..f68908c --- /dev/null +++ b/test/chesstrainer/background_test.exs @@ -0,0 +1,115 @@ +defmodule Chesstrainer.BackgroundTest do + use ExUnit.Case, async: false + + import ExUnit.CaptureLog + + alias Chesstrainer.Background + alias Chesstrainer.JSONLogFormatter + + setup do + handler_id = :"background_test_capture_#{System.unique_integer([:positive])}" + + :ok = + :logger.add_handler(handler_id, :logger_std_h, %{ + formatter: {Chesstrainer.TestLogHandler, %{pid: self()}}, + level: :all + }) + + on_exit(fn -> :logger.remove_handler(handler_id) end) + + :ok + end + + describe "run/3" do + test "returns the function's value on success" do + assert :ok = Background.run("test task", fn -> :ok end) + assert {:ok, 42} = Background.run("test task", fn -> {:ok, 42} end) + end + + test "logs an error tuple with its reason" do + capture_log(fn -> + assert :error = + Background.run("test task", [table: :demo], fn -> {:error, :disk_full} end) + end) + + assert_receive {:log_event, event} + + assert event.level == :error + assert event.meta[:table] == :demo + assert event.meta[:reason] == :disk_full + assert message(event) == "test task failed: :disk_full" + end + + test "logs a crash with the exception, reason and stacktrace" do + capture_log(fn -> + assert :error = Background.run("test task", fn -> raise "boom" end) + end) + + assert_receive {:log_event, event} + + assert event.level == :error + assert %RuntimeError{message: "boom"} = event.meta[:exception] + assert is_list(event.meta[:stacktrace]) + assert event.meta[:stacktrace] != [] + assert message(event) =~ "test task crashed: boom" + + decoded = event |> JSONLogFormatter.format(%{}) |> IO.iodata_to_binary() |> Jason.decode!() + + assert decoded["error"]["kind"] == "RuntimeError" + assert decoded["error"]["reason"] == "boom" + assert decoded["error"]["stacktrace"] |> is_list() + assert decoded["error"]["stacktrace"] != [] + end + + test "logs a throw with its reason and stacktrace" do + capture_log(fn -> + assert :error = Background.run("test task", fn -> throw(:nope) end) + end) + + assert_receive {:log_event, event} + + assert event.level == :error + assert event.meta[:kind] == :throw + assert event.meta[:reason] == :nope + assert message(event) == "test task crashed: :nope" + end + end + + describe "Chesstrainer.Cache scheduled backups" do + test "log a failed backup" do + table = :"test_failed_backup_log_#{System.unique_integer([:positive])}" + parent = self() + + backup_fun = fn _table -> + send(parent, {:backup_started, self()}) + {:error, :disk_full} + end + + log = + capture_log([level: :info], fn -> + pid = + start_supervised!( + Supervisor.child_spec( + {Chesstrainer.Cache, {table, [backup_fun: backup_fun]}}, + id: table + ) + ) + + send(pid, :backup) + + assert_receive {:backup_started, task_pid}, 1_000 + ref = Process.monitor(task_pid) + assert_receive {:DOWN, ^ref, :process, ^task_pid, :normal}, 1_000 + end) + + assert log =~ "cache backup failed" + assert log =~ ":disk_full" + end + end + + defp message(event) do + event + |> Logger.Formatter.format_event(8_096) + |> IO.iodata_to_binary() + end +end diff --git a/test/chesstrainer/json_log_formatter_test.exs b/test/chesstrainer/json_log_formatter_test.exs new file mode 100644 index 0000000..8324c70 --- /dev/null +++ b/test/chesstrainer/json_log_formatter_test.exs @@ -0,0 +1,190 @@ +defmodule Chesstrainer.JSONLogFormatterTest do + use ExUnit.Case, async: true + + alias Chesstrainer.JSONLogFormatter + + @time 1_760_000_000_123_456 + + describe "format/2" do + test "emits one JSON line with timestamp, level, message and request id" do + event = %{ + level: :info, + msg: {:string, "hello"}, + meta: %{time: @time, request_id: "request-1"} + } + + decoded = format_and_decode(event) + + assert decoded["timestamp"] == + @time |> DateTime.from_unix!(:microsecond) |> DateTime.to_iso8601() + + assert decoded["level"] == "info" + assert decoded["message"] == "hello" + assert decoded["request_id"] == "request-1" + end + + test "includes access-line fields" do + event = %{ + level: :info, + msg: {:string, "GET /health 200 in 1.2ms"}, + meta: %{ + time: @time, + request_id: "request-1", + method: "GET", + path: "/health", + status: 200, + duration_ms: 1.2 + } + } + + decoded = format_and_decode(event) + + assert decoded["method"] == "GET" + assert decoded["path"] == "/health" + assert decoded["status"] == 200 + assert decoded["duration_ms"] == 1.2 + assert decoded["request_id"] == "request-1" + end + + test "renders an exception's reason and stacktrace" do + event = %{ + level: :error, + msg: {:string, "boom"}, + meta: %{ + time: @time, + exception: %RuntimeError{message: "bad thing"}, + stacktrace: stacktrace() + } + } + + decoded = format_and_decode(event) + + assert decoded["error"] == %{ + "kind" => "RuntimeError", + "reason" => "bad thing", + "stacktrace" => ["lib/foo.ex:10: Foo.bar/1"] + } + end + + test "renders an OTP crash report's reason and stacktrace" do + event = %{ + level: :error, + msg: {:report, %{label: :gen_server_terminate}}, + meta: %{ + time: @time, + crash_reason: {%RuntimeError{message: "call boom"}, stacktrace()} + } + } + + decoded = format_and_decode(event) + + assert decoded["error"]["kind"] == "RuntimeError" + assert decoded["error"]["reason"] == "call boom" + assert decoded["error"]["stacktrace"] == ["lib/foo.ex:10: Foo.bar/1"] + end + + test "redacts email addresses and credentials in metadata, params and the message" do + event = %{ + level: :info, + msg: {:string, "contact user@example.com"}, + meta: %{ + time: @time, + email: "user@example.com", + token: "secret-token", + params: %{ + "user" => %{ + "email" => "user@example.com", + "name" => "Ada" + }, + "user@example.com" => %{"name" => "Keyed by email"} + } + } + } + + decoded = format_and_decode(event) + + assert decoded["email"] == "[REDACTED]" + assert decoded["token"] == "[REDACTED]" + assert decoded["params"]["user"]["email"] == "[REDACTED]" + assert decoded["params"]["user"]["name"] == "Ada" + assert decoded["params"]["[REDACTED]"]["name"] == "Keyed by email" + assert decoded["message"] == "contact [REDACTED]" + end + + test "redacts credential assignments in the message" do + event = %{ + level: :info, + msg: {:string, "GET /reset token=abc123&next=/home"}, + meta: %{time: @time} + } + + decoded = format_and_decode(event) + + assert decoded["message"] == "GET /reset token=[REDACTED]&next=/home" + end + + test "sanitises non-JSON metadata instead of crashing" do + event = %{ + level: :info, + msg: {:string, "weird"}, + meta: %{ + time: @time, + pid: self(), + reference: make_ref(), + tuple: {:ok, 1}, + charlist: ~c"hello", + domain: [:bandit], + invalid: <<0, 255>> + } + } + + decoded = format_and_decode(event) + + assert decoded["pid"] =~ "#PID" + assert is_binary(decoded["reference"]) + assert decoded["tuple"] == ["ok", 1] + assert decoded["charlist"] == "hello" + assert decoded["domain"] == ["bandit"] + assert decoded["invalid"] == "<<0, 255>>" + end + + test "falls back to a JSON line when the message cannot be formatted" do + event = %{ + level: :error, + msg: {:report, %{label: :broken}}, + meta: %{ + time: @time, + report_cb: fn _report -> raise "report blew up" end + } + } + + decoded = format_and_decode(event) + + assert decoded["level"] == "error" + assert is_binary(decoded["message"]) + assert decoded["message"] =~ "broken" + end + end + + describe "production configuration" do + test "uses the JSON formatter only in production" do + config = Config.Reader.read!("config/prod.exs", env: :prod) + + assert get_in(config, [:logger, :default_handler, :formatter]) == + {Chesstrainer.JSONLogFormatter, %{}} + end + end + + defp format_and_decode(event) do + line = event |> JSONLogFormatter.format(%{}) |> IO.iodata_to_binary() + + assert String.ends_with?(line, "\n") + assert length(String.split(line, "\n", trim: true)) == 1 + + Jason.decode!(line) + end + + defp stacktrace do + [{Foo, :bar, 1, [file: ~c"lib/foo.ex", line: 10]}] + end +end diff --git a/test/chesstrainer_web/request_id_test.exs b/test/chesstrainer_web/request_id_test.exs new file mode 100644 index 0000000..4bde6de --- /dev/null +++ b/test/chesstrainer_web/request_id_test.exs @@ -0,0 +1,47 @@ +defmodule ChesstrainerWeb.RequestIdTest do + use ExUnit.Case, async: false + + import Plug.Conn + + alias ChesstrainerWeb.Plugs.PutRequestId + alias ChesstrainerWeb.UserAuth + + setup do + on_exit(fn -> Logger.metadata(request_id: nil) end) + end + + test "PutRequestId copies the request id into the session" do + conn = + :get + |> Phoenix.ConnTest.build_conn("/") + |> Phoenix.ConnTest.init_test_session(%{}) + |> assign(:request_id, "request-1") + |> PutRequestId.call([]) + + assert get_session(conn, :request_id) == "request-1" + end + + test "PutRequestId leaves the session alone without a request id" do + conn = + :get + |> Phoenix.ConnTest.build_conn("/") + |> Phoenix.ConnTest.init_test_session(%{}) + |> PutRequestId.call([]) + + assert get_session(conn, :request_id) == nil + end + + test "mounting a LiveView puts the session's request id into logger metadata" do + socket = %Phoenix.LiveView.Socket{} + + assert {:cont, _socket} = + UserAuth.on_mount( + :mount_current_scope, + %{}, + %{"request_id" => "request-2", "user_token" => nil}, + socket + ) + + assert Logger.metadata()[:request_id] == "request-2" + end +end diff --git a/test/chesstrainer_web/request_logger_test.exs b/test/chesstrainer_web/request_logger_test.exs new file mode 100644 index 0000000..2c3a86c --- /dev/null +++ b/test/chesstrainer_web/request_logger_test.exs @@ -0,0 +1,97 @@ +defmodule ChesstrainerWeb.RequestLoggerTest do + use ChesstrainerWeb.ConnCase, async: false + + import ExUnit.CaptureLog + import Plug.Conn + + alias ChesstrainerWeb.RequestLogger + + setup do + level = Logger.level() + Logger.configure(level: :info) + on_exit(fn -> Logger.configure(level: level) end) + :ok + end + + describe "handle_event/4" do + test "logs one access line with method, path, status, duration and request id" do + conn = access_conn("GET", "/endgames", 200, "request-1") + + log = + capture_log([level: :info], fn -> + RequestLogger.handle_event( + [:phoenix, :endpoint, :stop], + %{duration: 12_500_000}, + %{conn: conn}, + :ok + ) + end) + + assert log =~ "GET /endgames 200 in 12.5ms" + assert log =~ "request_id=request-1" + end + + test "redacts token-like path segments" do + token = "SFMyNTY." <> String.duplicate("a", 43) + conn = access_conn("GET", "/users/reset-password/#{token}", 200, "request-1") + + log = + capture_log([level: :info], fn -> + RequestLogger.handle_event( + [:phoenix, :endpoint, :stop], + %{duration: 1_000_000}, + %{conn: conn}, + :ok + ) + end) + + assert log =~ "GET /users/reset-password/[REDACTED] 200 in 1.0ms" + refute log =~ token + end + + test "never logs the query string" do + conn = + :get + |> build_conn("/endgames?email=user@example.com&secret=abc") + |> assign(:request_id, "request-1") + |> put_status(200) + + log = + capture_log([level: :info], fn -> + RequestLogger.handle_event( + [:phoenix, :endpoint, :stop], + %{duration: 1_000_000}, + %{conn: conn}, + :ok + ) + end) + + assert log =~ "GET /endgames 200 in 1.0ms" + refute log =~ "user@example.com" + refute log =~ "secret=abc" + end + end + + describe "endpoint integration" do + test "emits exactly one access line for a request", %{conn: conn} do + log = capture_log([level: :info], fn -> get(conn, ~p"/health") end) + + assert length(Regex.scan(~r/GET \/health 200 in [\d.]+ms/, log)) == 1 + assert log =~ "request_id=" + end + + test "emits an access line for a LiveView initial request", %{conn: conn} do + log = capture_log([level: :info], fn -> get(conn, ~p"/users/log-in") end) + + assert length(Regex.scan(~r/GET \/users\/log-in 200 in [\d.]+ms/, log)) == 1 + assert log =~ "request_id=" + end + end + + defp access_conn(method, path, status, request_id) do + method + |> build_conn(path) + |> assign(:request_id, request_id) + |> put_status(status) + end +end diff --git a/test/support/test_log_handler.ex b/test/support/test_log_handler.ex new file mode 100644 index 0000000..ccd1d5b --- /dev/null +++ b/test/support/test_log_handler.ex @@ -0,0 +1,9 @@ +defmodule Chesstrainer.TestLogHandler do + @moduledoc false + + @doc false + def format(event, %{pid: pid}) do + send(pid, {:log_event, event}) + "" + end +end