Tests by change

Focused tests shorten feedback; complete gates catch shared contracts. A change ships with the prose and generated artifacts it makes stale.

Change class	Focused checks	Complete relevant gate
Python builder or publisher	matching pipeline/tests/builders/test_*.py, test_cli.py, publication/history tests	uv run --project pipeline pytest pipeline/tests
Scenario definition or generator	scenario and calibration/ensemble tests	scenario generate, check, Python suite, package scenario tests
Contract/schema	package contract tests and schema generation	package test + typecheck; Python suite when publisher shape changes
Pure derivation	matching package test	package test + typecheck
Scene/SVG/key/token	matching scene or serializer tests and golden diff review	package test + typecheck + site check/build
Documentation/content	site check/build, link and fragment audit	Playwright boundary, accessibility, interaction, search, and responsive visual QA
Provider declaration	focused builder/catalogue parity tests	Python suite and affected package/site gates

Scenario checks are two different operations

uv run --project pipeline windgram scenarios generate writes committed outputs. uv run --project pipeline windgram scenarios check is read-only and fails when definitions, generated profiles, hashes, or registry drift. Never regenerate merely to make a diff disappear; inspect whether the lesson and assertions still hold.

Golden SVG changes require review

Golden files under toolkit/test/golden/ protect the reference renderer and selected scenario output. A mismatch is a review request asking whether geometry, labels, units, IDs, accessibility, or tokens changed intentionally — not an instruction to accept new pixels.

1. Change the authoritative scene, tokens, renderer, or scenario input.

2. Run the package tests and inspect the failed SVG diff.

3. Confirm geometry, labels, units, ordering, accessibility, and stable IDs.

4. Only then update snapshots deliberately:

   pnpm --dir toolkit exec vitest run --update

5. Review every changed SVG in the diff and rerun the package tests, site check, and site build.

Never update a golden solely to obtain a green test, and avoid broad golden churn: if a supposedly local change rewrites unrelated fixtures, find the unstable input or unintended shared change first. When chart visibility changes, test buildKeySpec against that exact scene; key snapshots are not a substitute for scene/key parity assertions.

Site tests

Run the site’s browser tests with:

pnpm --dir site test

It builds the static site, serves that output locally, and checks two things: every research entry in the content source produced a route in the build, and the rendered windgram on the site matches the package’s own presentation defaults — chart and key drawn from the same tokens every consumer receives. test:built reuses an existing static build, which is what CI runs after its own build step.

Deterministic CI inputs

• committed provider fixtures;
• fixed clocks and explicit timezones;
• seeded synthetic scenarios;
• blocked external browser network access; and
• no downstream credentials.

Provider verification and network smoke runs are explicit review activities. Fixtures and synthetic scenarios make the automated suite deterministic.