fix: regenerate the bundled fixture against the sparsified corpus (#18)
The fixture predated three shipped corpus changes at once: no J rows in summary_categories (it was built before the crosswalk completion), no representation.parquet or code_set.parquet, and a still-dense wide era. Every test in this package and in cog-api runs against it, so both suites were green against a corpus that no longer exists. This is #18's stated prerequisite; it proves nothing about production until it lands. Regenerated from the publish tree at pipeline_commit 83f9715 (schema v6, built 2026-07-29). FY2011 goes from 2,864,212 rows to 496,004 -- 82.7% of the old partition was explicit zeros -- and the fixture now ships all ten publish-tree metadata tables rather than six. The generator's file list is a single constant now, so the copy step and the manifest step cannot drift. Three test repairs, each a real consequence of sparsification rather than a number to bump: test-categories.R "assistance" joined the spending subtype vocabulary with the J-prefix codes. test-spending.R The harmonization block counts rows that exist. Broward's E21/F21/G21 were zero-pads and are gone, so the anchor moves to FL state, whose three NA-mapped rows carry $2.83B -- the amount accounting was previously asserted only against 0 and could not have caught a bug. Broward keeps a test of its own, now asserting the zero-pads are absent. test-expenditure-concept.R Coverage-gap suggestions are presence-based. AL state's only FY2011 B47 cell was an explicit zero, so ig_federal_b47_wide stopped being a candidate there; FL state carries a real amount, so the counterpart guard is exercised against a suggestion that fires. test-fixture-vintage.R pins the structural facts that separate this vintage from its predecessor -- the ten metadata tables, the dense/sparse representation contract, zero explicit zeros in FY2011, code_set coverage, and J19's category. Checked against the old fixture: FY2011 carried 2,368,208 explicit zeros, so the assertion discriminates rather than merely passing. Suites: uscogdata 594 pass / 0 fail / 6 skip (was 576/0/6). cog-api 357 pass / 0 fail / 8 skip against the regenerated fixture, unchanged from its baseline.
This commit is contained in:
@@ -1,5 +1,31 @@
|
||||
# uscogdata 0.1.0 (development)
|
||||
|
||||
## Bundled fixture regenerated against the sparsified corpus
|
||||
|
||||
* `inst/extdata/fixture_corpus/` now tracks the corpus published on
|
||||
2026-07-29 (`pipeline_commit 83f9715`, schema v6). The wide era no longer
|
||||
stores explicit zeros: FY2011 fell from 2,864,212 rows to 496,004, of
|
||||
which none are `$0`. **Absence now means two different things** — in a
|
||||
`dense_source` year (≤ FY2011) an absent cell means Census published `$0`;
|
||||
in a `sparse_source` year (≥ FY2012) it means not reported. The corpus
|
||||
carries that rule in two new tables the fixture now ships,
|
||||
`representation.parquet` and `code_set.parquet`, alongside
|
||||
`census_collection_coverage.parquet` and `lineage_events.parquet`
|
||||
(all ten publish-tree metadata tables, up from six). Catalogued upstream
|
||||
as series break `SB194`.
|
||||
* `cog_categories()` gains an `assistance` spending subtype: the J-prefix
|
||||
aid/benefit codes (`J19`, `J67`, `J68`, `J85`) are categorised now that
|
||||
the upstream crosswalk covers every flow code carrying dollars.
|
||||
* Two consequences worth knowing about, both visible in provenance rather
|
||||
than in returned dollars. The harmonization block's `na_rows_excluded`
|
||||
counts only rows that exist, so wide-era codes that were zero-padded no
|
||||
longer appear there. Coverage-gap `suggestions` are presence-based for the
|
||||
same reason, so a recipe whose component codes were all `$0` for a given
|
||||
government-year is no longer suggested for it.
|
||||
* `tests/testthat/test-fixture-vintage.R` pins these structural facts, so a
|
||||
fixture left behind by a future publish fails loudly instead of letting the
|
||||
suite pass against a corpus that no longer exists.
|
||||
|
||||
## Breaking: corpus schema_version 4 (Phase P canonical ids)
|
||||
|
||||
* The package now requires corpus `schema_version = 4` (`MinCorpusSchema` /
|
||||
|
||||
Reference in New Issue
Block a user