Release 11/11: design the data-correction intake (deferred; gates the API announcement) #52

New Issue

2026-08-08T18:45:14-04:00

jared commented

2026-08-08 18:45:14 -04:00

Deferred by decision during the 0.3.0 design — recorded here because it gates the API announcement, not because it blocks this release.

What it costs to defer

The release cannot invite data-error reports, and cannot make the "traceable and correctable" claim — which is the thing that most distinguishes this corpus from Census's own files. That claim is currently on the shelf.

The design question, when you come back to it

A report like "Spokane's FY2019 police figure looks wrong" is a claim about the corpus, produced by cog_pipeline, which stays private. It is not a bug in this package. So BugReports in DESCRIPTION correctly points at package issues on the GitHub mirror, and data corrections need a different door.

There are already three doors a user could knock on — Hugging Face Discussions (live), the API, and this package — and no named intake.

The proposal that was on the table: a public, code-free us-cog-finance repo holding the data dictionary, the series-breaks catalog, corpus release notes, methodology, and a corrections tracker with structured issue forms. Every other surface points at it. Not empty on day one, so it does not read as abandoned.

The part that makes it more than a ticket queue

cog_pipeline already publishes lineage_events.parquet and a series_breaks table, and uscogdata already surfaces breaks in provenance (series_break_refs, corpus_break_refs) on every result.

So a verified correction should not terminate in a closed ticket — it should terminate as a lineage_event or series_break row. Then it is machine-readable, it flows through cog_explain(), and it appears unprompted in every future query touching the affected cells, including through the API, which passes provenance verbatim. Someone reports a bad figure; six months later a different person querying the same government is told about it without asking.

That terminus argues for a structured intake: a freeform "this looks wrong" cannot become a series_break row without a round trip. An issue form asking for govid, fiscal year, item code, published value, source value, and a citation to the Census document can.

Not blocking

0.3.0 ships without this.

**Deferred by decision** during the 0.3.0 design — recorded here because it gates the API announcement, not because it blocks this release. ## What it costs to defer The release cannot invite data-error reports, and cannot make the **"traceable and correctable"** claim — which is the thing that most distinguishes this corpus from Census's own files. That claim is currently on the shelf. ## The design question, when you come back to it A report like *"Spokane's FY2019 police figure looks wrong"* is a claim about the **corpus**, produced by `cog_pipeline`, which stays private. It is not a bug in this package. So `BugReports` in `DESCRIPTION` correctly points at package issues on the GitHub mirror, and data corrections need a different door. There are already three doors a user could knock on — Hugging Face Discussions (live), the API, and this package — and no named intake. **The proposal that was on the table:** a public, code-free `us-cog-finance` repo holding the data dictionary, the series-breaks catalog, corpus release notes, methodology, and a corrections tracker with structured issue forms. Every other surface points at it. Not empty on day one, so it does not read as abandoned. ## The part that makes it more than a ticket queue `cog_pipeline` already publishes `lineage_events.parquet` and a `series_breaks` table, and `uscogdata` already surfaces breaks in provenance (`series_break_refs`, `corpus_break_refs`) on every result. So a verified correction should not terminate in a closed ticket — it should terminate as a **`lineage_event` or `series_break` row**. Then it is machine-readable, it flows through `cog_explain()`, and it appears unprompted in every future query touching the affected cells, including through the API, which passes provenance verbatim. Someone reports a bad figure; six months later a different person querying the same government is told about it without asking. That terminus argues for a structured intake: a freeform "this looks wrong" cannot become a `series_break` row without a round trip. An issue form asking for govid, fiscal year, item code, published value, source value, and a citation to the Census document can. ## Not blocking 0.3.0 ships without this.

jared referenced this issue

2026-08-08 18:45:14 -04:00

Release 9/11: short release post on jaredknowles.com #50

jared referenced this issue

2026-08-10 19:55:18 -04:00

Release 9/11: short release post on jaredknowles.com #50

jared referenced this issue

2026-08-10 19:55:46 -04:00

Release 7/11: tag v0.4.0 and register with r-universe #47

jared referenced this issue

2026-08-11 09:42:34 -04:00

Release 7/11: tag v0.4.0 and register with r-universe #47