-
Notifications
You must be signed in to change notification settings - Fork 0
dump: make app.dump timestamps deterministic (precondition for daily refresh) #18
Open
Milestone
Description
Why
Moving the full refresh/dump to a daily cadence is blocked by timestamp churn.
app.dump is a stateless rebuild: every run stamps created_at/updated_at with the build time, so even when the underlying data/ is unchanged it emits a ~1400-file diff. On a daily schedule this means a giant no-op PR to TechAPI every day, drowning real changes and defeating review. (This is the same churn that was hit and reverted during a manual dump.)
What
Make dump output deterministic so a re-run with unchanged data produces no diff:
- preserve
created_at(never rewrite once set), and - only bump
updated_atwhen the record's substantive fields actually change, or - drop build-time timestamps from the static dump entirely / pin them (e.g.
SOURCE_DATE_EPOCH).
Unblocks
Once dump diffs reflect only real data changes, weekly-refresh (or a split dump-only job) can safely move to daily. Until then it stays weekly.
Related
Scheduling split landed in b53bcab (coverage-report daily; scrape + ingest weekly).