-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Night Watch Component Auditor
Assigned to: Cindy's Navi (cixzhang)
Goal: Ensure all components follow Astryx conventions — theming compatibility, API consistency, accessibility contracts, and export hygiene.
Source of truth documents:
- API Conventions — API rules (naming, structure, composition, a11y, exports)
- Theming Infrastructure — Theming rules (tokens, themeProps, variant maps, sub-element targeting)
- Design Conventions — Visual/interaction rules. Audit only the objectively checkable conventions (the 🟢 rows in its "Making Conventions Evaluatable" table); leave subjective visual judgment to Component Hardening Protocol Layer 3.
⚠️ Keep in sync. When updating conventions in those pages, update the corresponding check in this auditor. When the auditor finds a pattern not covered by those pages, add it there first, then update the check here.
Component quality regresses silently. A hardcoded color, a missing themeProps, a boolean prop without is prefix, a missing displayName — these don't break tests or CI but they break the developer and theming contracts. This role catches the drift across all dimensions in a single pass per component.
This role runs the Component Audit Rubric in mode N — its auto and semi items only. manual items are skipped, not failed, and §5b (rendered design judgment) is reported not measured because a nightly pass captures no screenshots.
That means the checks themselves are not restated here. The rubric is the single source for what is checked, how it is verified, and how it is scored; this page owns only the nightly mechanics below — selection, batching, the PR conventions, the refinement loop, and the state file.
| Rubric section | What the nightly pass covers |
|---|---|
| Component Audit Rubric §1 Accessibility | the semi items — ARIA wiring, busy-vs-disabled, hover-only paths, target-size greps |
| §2 Theming & token integrity | tokens, themeProps placement and coverage, component var naming, primitive reuse, 'use client'
|
| §3 Public API contract | prop and type naming, BaseProps and ...rest, input field surface, export hygiene |
| §5a Design conventions — objective | the 🟢 rows: spacing grid, concentric radius, control heights, type hierarchy, overlay tokens, motion safety, stacking, the reorder contract |
| §6 / §8 Testing and docs | the auto items — story and doc presence, doc↔source parity, the theming guards |
Everything a nightly pass cannot judge — the manual rows, and all of §5b — goes to the PR's Needs Review section rather than being scored.
Reporting. Publish the objective subscore as objective: x/y checks. Do not synthesize a letter grade from an automated pass; only modes P, O and R produce a grade.
This role runs once per night and produces one PR covering all findings.
Check memory/xds-night-watch-state.json for componentAuditor.lastRunAt. If it's from today (PST), skip and NO_REPLY.
The auditor runs two passes per night with separate PRs to keep hardening-issue work distinct from regular queue progress.
- List open hardening issues:
gh issue list --repo facebook/astryx --state open --label hardening \ --json number,title,body --limit 50
- Extract component names from single-component issues (e.g. "Hardening: Toolbar" →
Toolbar) -
Already audited? If the component is in
completedComponents(state file), close the issue immediately — no re-audit needed:gh issue close <number> --repo facebook/astryx --reason completed \ --comment "Component previously audited by Night Watch. See audit PRs in component history."
- For remaining components: audit them (Step 3) and create a separate PR (branch:
navi/hardening-issues/YYYY-MM-DD) - Add all audited components to
completedComponentsin state. - Do NOT close hardening issues that have a new PR — the PR itself is the resolution. The issue closes when the PR merges (via GitHub's linked issues or the PM's gardening pass).
- For batch hardening issues (e.g. "Hardening: Navigation Components"): read the issue body, extract all component names listed. If ALL are now in
completedComponents, close with a comment listing which audit PRs covered them. Otherwise skip — it'll close once the last component is done.
- Check
memory/xds-night-watch-state.jsonforcomponentAuditor.auditQueue - If the queue is empty, scan
packages/core/src/for all component directories and build a fresh queue - Pick the next 5 components from the queue
- Dedupe: skip any that were already audited in Pass 1 this run
- Audit them (Step 3) and create a separate PR (branch:
navi/component-audit/YYYY-MM-DD) - Update state: move completed to
completedComponents, advance queue
For each component, run all checks (sections 1-10 above) in a single pass:
- Read the source files — component, sub-components, types, styles, index.ts
- Check theming: CSS variables, themeProps, component reuse
- Check API: prop naming, type naming, structure, composition, input consistency, a11y, exports
For each issue found, log to memory/xds-night-watch/{date}.md under a ## Component Audit section:
- Categorize as:
hardcoded-value,missing-classname,classname-wrong-element,naming-violation,missing-type-prefix,structure-issue,composition-violation,input-inconsistency,a11y-gap,export-gap - Include file path, line number, current code, and what the fix should be
Create separate PRs for each pass:
- Pass 1 branch:
navi/hardening-issues/YYYY-MM-DD - Pass 2 branch:
navi/component-audit/YYYY-MM-DD
For each PR:
-
Run
pnpm testandpnpm buildbefore creating the PR. Fix any failures. - Publish PRs ready for review (not draft) — label with
hardening - When a finding needs human judgment, include it under a Needs Review section in the PR description
- If no issues found for a pass, do not create a PR — just update state and close issues
After creating the PR, don't wait for the daytime PR Review job. Run the refinement loop immediately:
- Record the HEAD sha you just pushed
- Schedule a one-shot recheck 5 minutes out (to let CI start)
- On recheck:
a. SHA collision check:
git fetch origin <branch> && git log -1 --format=%H origin/<branch>. If HEAD sha differs from what you recorded → STOP (someone else pushed, don't clobber their work) b. Check CI:gh pr checks <number>. If all green → done, log success c. If failing: Read the CI failure logs (gh run view <run_id> --job <job_id> --log-failed). Fix the failures in the worktree, runpnpm test && pnpm buildlocally, commit, push. Record the new HEAD sha. d. Schedule another recheck 5 minutes out - Iteration limit: max 3 fix attempts per PR. If still failing after 3 → log it and move on. The daytime PR Review job will pick it up.
Key rules:
- Always check SHA before acting — if it changed, someone else is working on it
- Only fix CI failures (test, lint, build) — don't re-audit or expand scope during refinement
- Each recheck is a one-shot scheduled job, not a busy-wait
- Log each iteration to
memory/xds-night-watch/{date}.md
Update memory/xds-night-watch-state.json:
{
"componentAuditor": {
"lastAuditedComponent": "Button",
"auditQueue": ["Card", "Dialog", ...],
"prsFiled": ["#650", "#800"],
"lastRunAt": "2026年03月22日T04:00:00Z",
"completedComponents": ["Button", "TextInput"]
}
}When the queue is empty, clear completedComponents and rebuild from scratch.
- Subjective design decisions — only objective checks (the objectively-checkable Design Conventions in dimension 11 are in scope; visual judgment — proportions, state treatment, density, composition — stays in Hardening L3)
- Review PRs (that's Reviewer)
- Fix CI beyond the refinement loop (that's QA)
- General issue triage (that's PM) — only closes hardening-labeled issues after auditing
{
"componentAuditor": {
"lastAuditedComponent": "string",
"auditQueue": ["string[]"],
"prsFiled": ["string[]"],
"lastRunAt": "ISO timestamp",
"completedComponents": ["string[]"]
}
}Start here Astryx Philosophy Contributing with AI Assistants Contributing
Architecture System Architecture Architecture Cheat Sheet Theming Infrastructure Distribution
Building a component Component Lifecycle Component Authoring Guide API Conventions Design Conventions
Quality Component Audit Rubric Accessibility Checklist AST-009 assistive-technology verification
Operations Release Process Night Watch Overview