ca-tariff-parse

Turn a published California electricity rate schedule into machine-readable structured data, with a citation for every single value.

Utility rate schedules are unstructured PDFs. They describe tiered prices, time-of-use windows, seasonal changes and eligibility conditions in prose and hand-set tables. Nothing public turns them into data you can compute with. This does, and it shows its work: every number it emits carries the document, page, sheet, section and line it was read from.

This is not rate advice and it is not a bill estimate. A parsed schedule is a representation of a document, not a calculation of what anyone owes. Real bills depend on surcharges, prorations, discounts and other schedules this tool does not combine.

Not affiliated with, endorsed by, or approved by SMUD or any other utility. The publishers listed here have no involvement in this project.

Install a signed release without cloning (needs uv):

uv tool install "git+https://github.com/ChelseaKR/ca-tariff-parse@v0.2.0"
ca-tariff-parse --version

Releases are signed annotated tags. CI verifies each one against .github/allowed_signers before it publishes anything, and git verify-tag v0.2.0 checks the same signature locally once that file is set as your gpg.ssh.allowedSignersFile. The package is not on PyPI yet; a publish-pypi workflow exists and runs only when the maintainer dispatches it.

Or work from a clone:

git clone https://github.com/ChelseaKR/ca-tariff-parse
cd ca-tariff-parse
make install
# Download the published schedules named in sources/sources.toml.
# fetch and watch are the only commands that touch the network.
make fetch
# Parse one, with every value cited.
uv run ca-tariff-parse parse sources/1-R-TOD.pdf --id smud-r-tod -o parsed.json
# See exactly how much of the document was accounted for.
uv run ca-tariff-parse coverage sources/1-R-TOD.pdf --id smud-r-tod

No account, no key, no telemetry. Once the source document is on disk, everything runs offline.

{
 "label": { "value": "Peak $/kWh" },
 "kind": "energy_usage",
 "price": {
 "amount": { "value": "0.1724" },
 "currency": "USD",
 "unit": { "value": "$/kWh" }
 },
 "effective_from": { "value": "May 1, 2025" },
 "season": { "value": "Non-Summer Season (October - May)" },
 "tou_period": { "value": "Peak" }
}

Each of those objects is abbreviated above. In the real output every one of them also carries a provenance block:

"provenance": {
 "document_id": "smud-r-tod",
 "document_sha256": "9be2f188d4bb39b4bb6436a9a37035457d0e0ab51f769fe995f58ca35422accd",
 "page": 2, "sheet": "R-TOD-2", "section": "II.A", "line": 11,
 "snippet": "Peak $/kWh 0ドル.1724 0ドル.1776 0ドル.1829",
 "locator": "smud-r-tod p.2 sheet R-TOD-2 II.A L11"
}

The full shape is published as a JSON Schema at schemas/parsed-schedule-v1.schema.json, so a downstream consumer can validate against the shape rather than against this document's prose description of it. tests/test_schema.py validates every committed golden file and every synthetic fixture's output against it, and, when the real source documents are present locally, all seven of those too, so the schema and the code that emits parse's output cannot drift apart unnoticed.

Never invent a rate, a rate structure, a time window, or a citation.

A tariff parser that emits a plausible looking price nobody published would be actively harmful, because someone might rely on it. So the design is built around refusing rather than guessing.

A value with no citation cannot exist. Cited refuses to construct without a fully populated Provenance, and a separate audit walks the finished result and fails if it can reach any value not wrapped in one. The two mechanisms are independent on purpose: the audit still catches a recognizer that bypasses the model. Nothing is written to disk until the audit passes.

Where the parser is unsure, it emits nothing. Concretely, it refuses to emit:

• a priced row whose unit it cannot read from the label, rather than assuming one;
• any row where an amount does not sit clearly under exactly one effective-date column, because a price under the wrong date is worse than no price;
• a start and end time for a window defined by exclusion ("All other hours"), or one carrying an exception ("between noon and midnight except during the Peak hours"), because a bare range would misstate the rule. The verbatim definition is carried instead;
• a time-of-use window whose definition holds a currency amount, because that is a price sitting in the same column, not a statement of when a period runs;
• a price for a cell the publisher marked n/a;
• a holiday row with a missing cell, or a whole holiday table whose header does not divide into three headings;
• a dated block pricing several categories at once whose amounts do not line up one for one with the headings above them.

Coverage is a published output, not an implicit claim. Every parse reports how many content lines it accounted for, and everything it did not understand appears in unparsed with its location and reason. Nothing is ever silently dropped: unrecognised text is still carried verbatim in notes. A document containing a section the parser does not understand cannot produce the same output as one it fully understands, and there is a test that proves it.

Seven published schedules from two publishers are in the manifest. None of them parses completely, and the figure for each is an output of the tool rather than a claim made here.

make coverage-real reproduces the table from the fetched documents, and tests/test_published_figures.py asserts that every figure in it is the one the parser reports, so the table cannot drift away from the tool while the documents are present. It skips where they are not, like the other real document tests.

The three PG&E figures were 0% until a document profile was added. A profile is selected per manifest entry and carries only what a document cannot state about itself. There are four fields, and a document naming no profile gets a default in which all four are the refusing value.

Nothing else is in it, and in particular no coordinate. ADR 0005 expected the profile to state the width of the keyword column; across three schedules that column starts anywhere from 72 to 101 points and the body beside it anywhere from 133 to 172, so a single number cannot separate them and the column is read from the page instead.

What that bought, and what it did not, is in ADR 0006. The short version is that the outline is worth most of the coverage, the bracket notation is worth the prices that would otherwise have been refused or reversed, and the new prices are dated sheet by sheet, because these sheets are filed one at a time and the sheets of one schedule take effect on different days.

Coverage of the four SMUD schedules did not move and their golden output is byte for byte unchanged. That is the test that this is a seam rather than a second branch: the first publisher takes the default for all four fields.

Three of the four SMUD schedules carry a small table pairing a billing circumstance ("Bill period is shorter than 27 days") with the basis on which a charge is prorated under it. Reading it in line order does not work: the "Basis for Proration" cell is sometimes one row tall and sometimes drawn to span two or three circumstances at once, and text extracted in reading order cannot tell those apart. Two circumstances that share one basis print as one paragraph starting a line late, which looks exactly like an unrelated basis misattributed to the wrong circumstance.

The two cases are told apart by going around the text entirely: pdfplumber reports the ruled lines a table's own cells are drawn with, and a cell whose border spans two rows is a merge, not a guess about one. Reading that border directly, rather than inferring row breaks from spacing, is what lets Bill period is shorter than 27 days and Bill period is longer than 34 days share one basis while a third circumstance keeps its own — exactly what the published page shows, and unrelated to how any of the three sentences happens to wrap. A circumstance whose cell does not sit inside exactly one basis cell's border is left unparsed rather than paired at a guess.

Only a table with a real ruled border is read this way. A schedule with no such table, or one under a different header, is untouched by this and keeps being read line by line as before. The full account, including the case that originally looked like it needed a spacing threshold and did not, is in ADR 0007.

Most of it, and each refusal is a case where a value could otherwise be wrong.

• A row that does not fill every column its table names. A row carrying one amount where the table has two columns has to say which column it sits in, and it does not: the price may be that column's or the whole row's. This is what keeps the "all usage" rows of the unbundling sheets, which set one amount between two columns, and the PDP tables, which set one under the first of two, unread. A page that names no columns at all is still refused whole, exactly as before.

• A block whose heading states no unit, and no heading over it states one either. A unit the publisher broke across a line ending is read, because the bracket it is written in opens on one line and closes on the next and the publisher's own punctuation says where it ends. A bracket that never closes, or that takes more than one line ending to close, states nothing that can be read without reconstructing it, and refuses. See ADR 0014. Otherwise: a table can state its unit once and then name each component of it on a line of its own, and the unit reaches over the table the page sets under it. How far under is read off the heading's own first line: nothing set further left than that line is in the table, and a heading whose first line is set left of it heads nothing. A component line set level with the rows beneath a heading is a sibling of those rows, passed over rather than made their label; a block with no component line of its own is priced under the heading's own name, exactly as a row directly beneath the heading is. See ADR 0013 and ADR 0017.

• A row set further left than the table's first line. Below the components, the commercial sheet sets its remaining component rows level with the unit heading itself, left of the indented line the table began with, and nothing on the page says what those rows are priced per. The residential sheet sets its first line, Generation: 0ドル.12855, level with the heading, so its later rows at that level are in the table and are read. See ADR 0017.

• A row whose label may go on to the next line. A label the publisher broke at a line ending is joined where its own brackets say it continues, the rule ADR 0014 reads a wrapped unit by, and cited to both lines with the label's own words as the quote. Where nothing on the page says the next line finishes the label, an unpriced line set as a row, stating no unit and heading nothing, the row is refused rather than published with half its name: Bundled Power Charge Indifference over Adjustment (all usage)**** on the residential sheet is that case. A label still opening a bracket is refused for the same reason. See ADR 0017.

• The cross-reference wording and the credit form. Each is a statement about how one publisher writes, not a thing a document cannot state about itself, so neither belongs in a profile. Closing them means finding the shape, not adding a field.

  The identity fields were the third item here, and the shape turned out to be that a running head runs: the line naming the schedule is the one that appears on every sheet, wherever the publisher sets it, and a sentence ending in the word "schedule" appears once. Both publishers' codes are read now, and so is a title where the page settles which neighbouring line it is. Where both neighbours repeat, no title is read. See ADR 0015. The remaining three identity fields are null for this publisher because the page states no value for them, not because one went unread: Resolution is printed as a label with nothing beside it on every sheet, nothing is printed as adopted, and each sheet states its own effective date, which is carried on every charge rather than borrowed for the schedule. See ADR 0018.

A filing change marker -- a bracketed capital such as (R) beside a revised line, or a change bar in the right margin -- is no longer one of these, when the marker is the only thing on the line: it is now read as furniture, the same category as a running header, rather than reported as unrecognised content. A marker attached to real text is untouched, exactly as printed inside whatever citation quotes that line. See ADR 0010. On a row read across named columns the same markers fall between the cells as well as after them, and are skipped there for the same reason.

A page that names its columns is no longer one of these either. Sheet 3 of the commercial schedule sets its two rate options over its amounts:

Total Bundled Time-of-Use Rates B-1 Rates B1-ST Rates
Total TOU Energy Rates ($ per kWh)
 Peak Summer 0ドル.47087 0ドル.49377
 Partial-Peak Winter (for B1-ST only) --- 0ドル.36632

Where the page names them, each price is attributed to the column it sits under and carries that column's own name in applies_to, cited to the line that names it. Where it names nothing, the page is still refused whole. A cell the publisher marked with dashes is read as that column carrying no price for that row rather than as a reason to refuse the row: the winter partial-peak rate above is published for one rate option and not the other, and emitting it for both, or refusing it entirely, would each misstate the sheet. See ADR 0012.

Two of the three PG&E schedules once emitted no value at all, and the third emitted two time-of-use windows under a season the publisher never wrote. The full account of that first pass is in ADR 0005. What turned out to be general was the machinery that made the failure legible rather than dangerous: the positional layout model, the citation and audit rules, the coverage accounting, and above all the refusals. A rate table whose shape is not recognised produces nothing rather than something plausible.

What is left unaccounted for on the four SMUD schedules is largely genuine narrative: critical peak pricing terms, service voltage definitions, metering conditions. One specific thing is structured and stays refused, deliberately rather than for lack of a rule:

• A price stated inside a sentence. SSR gives its export compensation rate as "The Export Compensation Rate effective June 1, 2026 will be 0ドル.0960 per kWh (subject to updates as described in the paragraph below)." Reading a price out of prose means deciding by guesswork what the price is for, and this one is additionally stated as provisional on a formula defined elsewhere, which an ordinary Charge has no way to say. SSR emits no charges at all rather than one, and stays that way; see ADR 0011 for why this is a closed decision rather than an open gap.

Two other shapes that were structured and refused when this section was first written are read now. smud-ci-tod1's commercial transition table states its unit in a column of its own and dates its prices to a bare year carrying a footnote; see ADR 0008. An enumerated condition list outside any Applicability or Eligibility part, such as the Standby Service option's own "Standby Service applies when all of the following conditions are met: 1. ... 2. ... 3. ..." on three SMUD schedules, is read as a new Condition record that carries no disposition; see ADR 0009 for why Applicability's scale does not fit one item of a conjunction.

$ uv run ca-tariff-parse coverage sources/CI-TOD1.pdf --id smud-ci-tod1
content lines 142/201 recognized (70.6%)
sections 15/29 fully recognized (51.7%)
fully recognized False
emitted 85 charge(s), 5 time-of-use window(s), 11 holiday(s), 5 cross reference(s), 3 proration rule(s), 3 condition(s)
unparsed:
 VIII p.7 L13 to p.8 L1 (3) 3 of 13 lines in a recognized section matched no rule
 | *Subject to future rate increases.
 | **Time-of-Day periods apply as described in Section VII.
 | THIS PAGE INTENTIONALLY LEFT BLANK

The published PDFs are not redistributed from this repository. What is committed is sources/sources.toml: the publisher, URL, retrieval date and SHA-256 of the exact bytes that were read. make fetch downloads them and make verify-source confirms you hold the same bytes the parser was run against. If a publisher revises a schedule at the same URL the digest stops matching, which is a signal to review the change deliberately rather than to relax the check.

sources/sources.toml records where a document came from. It is not a claim of permission, endorsement, or any relationship with the publisher.

Retrieval honours robots.txt and is a handful of requests, never a crawl. robots.txt for a host is read before anything is fetched from it, and a publisher that disallows the path is not fetched from at all.

Requests identify themselves as ca-tariff-parse/<version> with a link back to this repository, so a publisher can recognise them in a log and refuse them by name:

User-agent: ca-tariff-parse
Disallow: /

That group is honoured. Until 2026年09月06日 it was not: the tool sent a spoofed desktop Chrome header, which urllib.robotparser reduces to the token mozilla, so no named group was ever selected and only a User-agent: * group could refuse a fetch.

A publisher can revise a schedule at the same URL. ca-tariff-parse watch downloads each pinned document and, where the bytes are not the pinned bytes, parses the revision and writes a value-level diff against the last reviewed parse: every price, window, holiday, rule and identity field that was added, removed or changed, each with the citation it was read from before and after. Values are matched by what they are, not where they sit, so a value that only moved on the page is not a change. A download that fails is an error, never "unchanged": the watch has to be able to say it looked.

The last reviewed parse of each document is its baseline, committed under data/parsed/ as a projection of parse's output without notes and the samples under unparsed, the two places most of an unread document's prose would otherwise travel. Every cited value is there untouched, and the file says what was left out and why. make watch-baseline regenerates them from the pinned documents, and a test fails when a committed baseline is not what the current parser writes. See ADR 0016.

.github/workflows/tariff-watch.yml runs the watch weekly and opens one pull request per revised document, carrying the diff report, the new baseline and the manifest entry's new digest, retrieval date, page count and size. The watch merges nothing: accepting a revision is the deliberate review the manifest's digest check asks for, and a person does it. PDFs are never committed, by the watch or by anyone.

ca-tariff-parse diff old.json new.json runs the same comparison on any two parses of one document, as Markdown or, with --jsonl, one object per change.

sources reports each document as not fetched, present, or mismatched. mismatched means a file exists at the manifest's filename but its bytes do not match the pinned SHA-256 — a truncated download, a publisher revision saved under the old filename, or a hand-edited file. sources only reports the state; it exits 0 either way. verify-source raises on a mismatch, so use it (or parse --id / coverage --id, which call it first) where a non-zero exit code matters, such as in a script or CI step.

parse and coverage accept --id <manifest-id>, which verifies the file against the pinned digest before parsing, records the publisher and retrieval date in the output, and reads the document with the profile its manifest entry names. They also accept --min-coverage, which exits non-zero when too little of the document was understood, and --profile, which names a document profile for a file that is not in the manifest.

coverage --json writes the same figures as JSON instead of the text report, for a CI step gating one document, a script tracking coverage over time, or anything else that wants the numbers rather than the prose. Every value in it is selected out of parse's own report rather than computed a second time, so the two cannot come to disagree, and --min-coverage gates identically either way.

parse says what it read. It does not say whether what it read hangs together, and a consumer who works that out downstream is one step from this project's defining defect: a missing thing read as a value. A charge whose period no window defines is not a charge that applies all day. A residual window is not a window with no hours.

ca-tariff-parse check parsed.json --manifest sources/sources.toml answers five named properties, each in one of three states:

There is deliberately no state meaning "no problems found". A document with no charges satisfies "every charge has a window" vacuously, and reporting that as a pass is the same error as printing a suppressed cell as zero — so smud-ssr, which prices nothing, reports all five properties as cannot be established rather than five passes.

The properties are period-window-closure, season-vocabulary, window-enumerability, unit-uniformity and cross-reference-pinned. --require <property> exits 4 unless that property holds; cannot be established counts as unmet, because a caller who says a value depends on a property is not helped by "we could not tell". --json writes the same report as JSON. check infers nothing, fills nothing in, fetches nothing, and is byte-for-byte deterministic over the same payload. It reads parse output and watch baselines alike: the baseline projection drops only notes and the unparsed samples, which no property reads.

Two of the answers over today's corpus are worth stating, because they are facts about the documents rather than about the tool. season-vocabulary reports cannot be established on every real document, because each names its seasons twice — Summer Season (June - September) on the charge tables and Summer (Jun 1 - Sept 30) on the time-of-use table — and nothing in the parse says those denote the same season. Matching them would be inference. And period-window-closure does not hold on smud-r-tod or on the complete synthetic fixture, where one credit line carries the period midnight to 6:00 a.m. daily, a phrase read from prose that no window defines.

ca-tariff-parse history --id smud-r-tod \
 --match 'kind=energy_usage label="Generation" season=Summer'

walks the committed change reports under data/changes/ and the reviewed baseline under data/parsed/, and prints every state a matching record has held, each with the retrieval date and the citation of the revision that set it, ending with the baseline. --all does every record the reports mention; --jsonl writes one object per timeline.

Everything is built from what is committed. Nothing is fetched and nothing is interpolated. Three refusals are the substance of it:

• A gap is reported, not joined. Each report states the digest of the bytes on both sides. When one report's "before" digest is not the previous report's "after" digest, a revision is missing between them, and the timeline says so at that point. The values on either side are real; the line between them is not. The same check runs against the reviewed baseline, so a baseline written from bytes no report produced is a gap too.
• The order comes from the reports, not the filenames. A retrieval date is a fact the report states, so it is read from the report. Two reports whose dates run backwards are refused with both dates named, because the order to read them in is exactly what is in doubt.
• No leg claims one parser read both sides. Each leg carries diff's three-state parser_comparison, which has no state meaning "the same parser": two equal release stamps cannot establish one, and every build between two releases stamps the same string.

A record no report mentions is listed with one state from the baseline, rather than left out — omitting it would make "no result" mean both "no such record" and "a record the publisher has not moved". A --match term naming a field no record kind is identified by is an error, not a term that quietly matches nothing; history exits 5 when a well-formed match selects no record, which is a different exit from "the reports could not be read". --all asks for nothing by name, so an empty result there is a statement about what has been committed rather than a failed request, and it exits 0 saying which directory holds no reports.

To make this possible, diff --jsonl now repeats both sides' retrieval date, digest, parser version and parser_comparison on every change line. It is still one object per change; the stamps travel on each line so a line lifted out of the file still says which retrieval it came from.

ca-tariff-parse calendar tests/golden/smud-r-tod.json --dir dist/calendar

writes smud-r-tod.ics and smud-r-tod.refused.json. Both, always. A missing refusal file reads as "no refusals"; an empty list reads as "nothing was refused", and those are different statements.

This is the first consumer-facing derivation the project ships, so the fence matters more than the feature. A rule is written only where it re-expresses text the parser already committed to:

• a residual window ("All other hours, including weekends and holidays") is refused, because the parser has already refused to give it hours and deriving them here would undo that refusal — and it is refused on the residual flag itself, not on the absence of times, so a residual window that did carry hours is still refused;
• a window defined by exception ("Weekdays between noon and midnight except during the Peak hours") is refused, because the exception is prose this module cannot subtract;
• a window with hours but no stated day type is refused rather than assumed daily;
• a window whose stated end is at or before its start crosses midnight, and splitting it is a decision the document did not make;
• a holiday day_rule outside a closed grammar — a fixed day (25), an ordinal weekday (Third Monday), or a last weekday (Last Monday) — is refused. Day after Thanksgiving is a real rule with a defensible date and it is not in the grammar, so it is refused rather than rendered as the fourth Friday. A guess in a calendar entry has a calendar entry's authority.

On smud-r-tod that is 2 of 5 windows and 11 of 11 holidays rendered, with three refusals listed. On smud-ci-tod1, which states no bare times, it is 0 of 5 windows: the calendar is empty and says so.

Season bounds only where the season states whole months. BYMONTH selects whole months and nothing else, so Summer (Jun 1 - Sept 30) becomes BYMONTH=6,7,8,9, while Summer (Jun 15 - Sept 30) would have to be widened or narrowed to fit and is left unbounded and marked X-CA-SEASON-BOUNDS:partial. The season string still travels in the summary.

No time zone is inferred. Times are floating local times, as the document states them, and the file says so. The recurrences are anchored to 1970 because a published rule states no year; only the month, day and time of a DTSTART mean anything, and the file says that too. Every DTSTART is a real instance of its own rule, so a reader cannot treat it as an extra occurrence.

Nothing reads the clock. DTSTAMP is the document's own retrieval date, or the epoch when it states none — a generation time would make two renderings of one parse differ. The same parse renders identical bytes.

A watch baseline is refused: it omits the verbatim prose, and a refusal that cannot quote the text it refused is not much of a refusal.

parse emits a tree. An analyst comparing thirty schedules in a spreadsheet wants one row per charge with the citation beside it.

ca-tariff-parse export data/parsed/pge-e-1.json --table charges > charges.csv
ca-tariff-parse export data/parsed/pge-e-1.json --all dist/tables --format jsonl

Tables are charges, tou_windows, holidays, proration, conditions, cross_references and applicability. Every row carries document_id, document_sha256 and parser_version, so a row lifted out of its file still names the bytes it came from, and every cited field is immediately followed by a <field>.locator column. --snippets adds <field>.snippet too; it is off by default because a snippet carries the document's own text (ADR 0003).

The column order is derived from schemas/parsed-schedule-v1.schema.json, not listed in the export, so a field added to the model and the schema appears without anyone remembering to add a column, and a field added to only one of them fails loudly.

Four rules keep a table from saying more than the parse did.

• A null is an empty cell. Never 0, never n/a. A charge that states no season states no season, and a spreadsheet will not total it.
• Nothing is dropped in the reshape. Each table's row count is checked against the record count of the parse, and a mismatch raises rather than writing a short table.
• Nothing is computed. There is no annualised price and no hours-per-window column. A derived number sitting in a table of cited ones is indistinguishable from them.
• An empty table is a file with a header. smud-ssr prices nothing, so its charges table is one header line. A missing file would read as "not exported"; a header with no rows reads as what the parse says.

Rows are sorted by their first citation — on the citation's own page, sheet, section and line rather than its rendered text, so page 10 follows page 9 — with the whole row as the final tiebreak. Two exports of one parse are byte identical. CSV cells that a spreadsheet would evaluate as a formula are prefixed with an apostrophe; a leading minus is left alone when the cell is a number, because a credit is printed as -0.05 and neutralising it would change what a reader sees.

export reads a full parse and a watch baseline alike. The projection removes only the verbatim prose, which no table carries, so both produce identical tables. notes and unparsed are deliberately not tables: notes are the document's prose rather than records with fields, and an unparsed section reports where the parser stopped rather than a value it read, so it has no citation to flatten. Excluding unparsed is checked against the schema — if that record ever gains a cited field, the export refuses to run rather than hiding it.

OpenEI's Utility Rate Database is the dataset most tools reach for when they need a California tariff, and its records carry no citation to a page. reconcile audits a record the user downloaded themselves against a parse of the document it claims to describe.

ca-tariff-parse reconcile data/parsed/pge-e-1.json my-urdb-record.json

Nothing is fetched. The record is a file you supply; reconcile reads it, compares it, prints a report and writes nothing back. It never emits a URDB record and never fills a field it did not read.

Each field of the record comes back in one of four states:

There is deliberately no state meaning "checked out fine". smud-ssr prices nothing (ADR 0011), so every priced field of any record reconciled against it comes back no statement — never confirmed, and never contradicted. Reporting "nothing to disagree with" as agreement is the same error as printing a suppressed cell as zero.

Every key in the record is reported. A field the mapping does not cover is listed as not comparable with its reason rather than dropped, so a reader can see the whole record was looked at and exactly how much of it this model can speak to.

Deliberately not compared, each named in the report with its reason:

• The schedule matrices (energyweekdayschedule and the other three). URDB names its rate periods by index; the document names them in the words it prints. Neither record states the correspondence, so aligning them would be a guess rather than a reading.
• mincharge. This model records the priced line items a page prints. Nothing in a charge states the role "the minimum a bill must reach", and picking one by reading its label text would be a guess.
• A tier carrying a non-zero adj. What a customer pays is rate + adj; what this model records is the price the page prints. Comparing the bare rate against a printed price would confirm a number nobody is billed.
• A tier's max and sell. A tier boundary and an export rate; this model records a price.
• URDB's own metadata — label, utility, eiaid, sector, description, source, uri, approved and the rest. Facts about the database, not statements the document makes.

• Values are compared by membership, not by position. Because the period correspondence is unstated, reconcile asks the only question both records can answer: does the document state this amount, in a comparable unit, on the effective date the record names? A contradiction therefore means the two records disagree about what the schedule prices. That is either a URDB entry error or a gap in this parser, and reconcile diagnoses neither.
• A date the parse does not carry widens the comparison rather than emptying it. When the record's startdate matches an effective date in the parse, priced fields are compared only against charges effective on that date. When it matches none, they are compared against every charge, and the report says so. Narrowing to nothing would have reported silence where there is a disagreement.
• A credit is not a rate. A credit is a reduction the document prints as a negative amount, and URDB carries that idea elsewhere. Matching one against a rate would confirm a price nobody is billed, so credits are excluded from every comparison.
• Numbers are read exactly. The record's JSON is decoded with Decimal, not float, so a rate printed 0.1724 is compared as 0.1724 rather than as the nearest binary approximation of it.

--json writes the same findings as a ca-tariff-parse/reconcile/v1 payload. reconcile exits 0 when nothing contradicts, 3 when something does, and 2 when the record cannot be read at all — so a script can tell "the record disagrees" from "the record is not readable" without parsing stderr.

Everything the command line does with a parse, a caller can do with the same records. load reads a file parse wrote, or a watch baseline committed under data/parsed/, back into typed objects:

import ca_tariff_parse as ctp
schedule = ctp.load("data/parsed/pge-e-1.json")
for charge in schedule.charges.where(kind="energy_usage", season=None):
 print(charge.label.value, charge.price.amount.value, charge.price.unit.value)
 print(" ", schedule.cite(charge, "label").locator)
 print(" ", charge.price.amount.provenance.locator)

Three properties are worth stating, because they are the reason this exists rather than a json.load in the caller.

Loading is not trusting. Every record goes back through the constructors a fresh parse uses. A citation missing a field, a digest that is not 64 hex characters, a page number that is not positive — each raises ProvenanceError and no object is produced. There is no partially cited result to accidentally publish. assert_fully_cited then walks the reconstruction independently, the same walk that runs before parse writes anything.

A derived value is recomputed, never read. A locator, an unparsed span, line_ratio, section_ratio and fully_recognized are all derived from fields beside them. load recomputes each and compares. A payload asserting fully_recognized: true over counters that say otherwise is refused, rather than believed — which is the shape ADR 0002 exists to prevent, arriving from the outside instead of from the parser.

An omission is not an empty answer. A watch baseline drops the document's verbatim prose on purpose (ADR 0003, ADR 0016). A schedule loaded from one carries schedule.withheld == ("notes", "unparsed[].sample"), its notes refuse to be queried at all rather than answering "none", and re-serialising it writes a baseline again — never a full parse with "notes": [], which would state that the document has no prose. It has prose; the projection dropped it.

where(field=value) compares a cited field on its value and a structural field directly. where(season=None) selects the charges that state no season, which is a different question from where(season="Summer"). A field name the record does not have raises rather than returning nothing, because an empty result reads as "the schedule states none of these" and a typo is not that.

schedule.cite(record, field) returns the Provenance behind one field. It raises when the document did not state the field, and when the field is structural metadata such as kind: a caller asking where a value came from is told there is no value, rather than handed a blank to render.

The names in ca_tariff_parse.__all__ are the supported surface, versioned with the package under SemVer. Within a major version, a name is not removed and its meaning does not change; new names, new optional record fields and new subcommands are minor changes. The JSON payload has its own version, printed in its schema key and published as schemas/parsed-schedule-v1.schema.json; a breaking change to the payload is a new schema id, not a silent edit to this one.

Reading a parse needs only the standard library. import ca_tariff_parse does not import pdfplumber, so a consumer of the committed baselines can install the package without the PDF stack present and load still works; only parse_path and parse_document, which read a document, need it.

1. Extract. pdfplumber gives the position of every word. Positions are kept, because which price belongs to which effective date is carried entirely by horizontal alignment. Where a table has a ruled border, pdfplumber's own line-drawing detection is read too, so a cell a publisher drew to span several rows is captured as the single merged cell it is rather than guessed at from spacing.
2. Segment. Lines are grouped into the document's own outline, so every value can cite a part and an unrecognised part can be named rather than lost. Two outlines are known: statute-style numbering (roman parts, lettered subsections) and a keyword set in a column with the body beside it. Which one a document uses comes from its profile, because the page does not say.
3. Recognize. Small independent recognizers each claim a section shape and report exactly which lines they consumed.
4. Account. Any line no recognizer consumed becomes an unparsed entry and a verbatim note.
5. Audit. The provenance walk runs before anything is written.

Tests run against clearly labelled synthetic fixtures, so the suite works offline and without redistributing a publisher's document. One fixture is written in a keyword outline with accounting-bracket negatives and a supersession header, so the profile is exercised in CI too, and parsing it without a profile has to refuse all three.

The golden output of the four SMUD schedules is committed under tests/golden/, so a parser change that would alter a published price shows up as a reviewable diff. No golden file is committed for the three PG&E schedules: most of each document is still carried verbatim in notes, and committing that would republish it. Those three are covered instead by a spot check of six prices quoted from the sheets with their unit, effective date and heading.

make verify # install, lint, typecheck, test with the coverage floor
make fmt # apply formatting and safe fixes
make golden # regenerate golden output (review every changed price)
make coverage-real # report parse coverage of every fetched document

docs/ROADMAP.md holds the plan: what is being read next, what each phase would have to prove before it lands, and what has already been decided against.

Apache-2.0. See LICENSE.

Licensing the code says nothing about the source documents. Rate schedules remain the work of their publishers, which is one reason they are fetched rather than vendored.