jared
c375c55da7
The fixture predated three shipped corpus changes at once: no J rows in summary_categories (it was built before the crosswalk completion), no representation.parquet or code_set.parquet, and a still-dense wide era. Every test in this package and in cog-api runs against it, so both suites were green against a corpus that no longer exists. This is #18's stated prerequisite; it proves nothing about production until it lands. Regenerated from the publish tree at pipeline_commit 83f9715 (schema v6, built 2026-07-29). FY2011 goes from 2,864,212 rows to 496,004 -- 82.7% of the old partition was explicit zeros -- and the fixture now ships all ten publish-tree metadata tables rather than six. The generator's file list is a single constant now, so the copy step and the manifest step cannot drift. Three test repairs, each a real consequence of sparsification rather than a number to bump: test-categories.R "assistance" joined the spending subtype vocabulary with the J-prefix codes. test-spending.R The harmonization block counts rows that exist. Broward's E21/F21/G21 were zero-pads and are gone, so the anchor moves to FL state, whose three NA-mapped rows carry $2.83B -- the amount accounting was previously asserted only against 0 and could not have caught a bug. Broward keeps a test of its own, now asserting the zero-pads are absent. test-expenditure-concept.R Coverage-gap suggestions are presence-based. AL state's only FY2011 B47 cell was an explicit zero, so ig_federal_b47_wide stopped being a candidate there; FL state carries a real amount, so the counterpart guard is exercised against a suggestion that fires. test-fixture-vintage.R pins the structural facts that separate this vintage from its predecessor -- the ten metadata tables, the dense/sparse representation contract, zero explicit zeros in FY2011, code_set coverage, and J19's category. Checked against the old fixture: FY2011 carried 2,368,208 explicit zeros, so the assertion discriminates rather than merely passing. Suites: uscogdata 594 pass / 0 fail / 6 skip (was 576/0/6). cog-api 357 pass / 0 fail / 8 skip against the regenerated fixture, unchanged from its baseline.
121 lines
5.2 KiB
JSON
121 lines
5.2 KiB
JSON
{
|
|
"schema_version": 6,
|
|
"built_at": "2026-07-30T14:07:36Z",
|
|
"pipeline_commit": "83f9715",
|
|
"fixture_note": "Four-year (2011, 2012, 2019, 2020) fixture for uscogdata tests. Full corpus available via USCOGDATA_URL. Regenerated from the sparsified schema-v6 corpus: the wide era (<= FY2011) no longer stores explicit zeros, so FY2011 absence means Census published $0 while FY2012+ absence means not reported. representation.parquet and code_set.parquet carry that rule and ship in full, as do every other metadata table in the publish tree. 2011/2012 straddle both the wide-aggregate -> modern-leaf format boundary (exercised by basis=\"harmonized\" and recipe= queries) and the dense -> sparse representation boundary (SB194); 2019/2020 retain the prior per-capita/CPI regression anchors. Regenerated via data-raw/regenerate_fixture_corpus.R.",
|
|
"data_vintage": {
|
|
"source_vintages": {
|
|
"2012": "10162019",
|
|
"2013": "10162019",
|
|
"2014": "10162019",
|
|
"2015": "10162019",
|
|
"2016": "10162019",
|
|
"2017": "06102021",
|
|
"2018": "06102021",
|
|
"2019": "06102021",
|
|
"2020": "06122023",
|
|
"2021": "06122023",
|
|
"2022": "06052025",
|
|
"2023": "06052025"
|
|
},
|
|
"registry_rows": 148,
|
|
"acs_vintage": "ACS 2018-2022 5-year"
|
|
},
|
|
"scope": {
|
|
"gov_types_included": [0, 1, 2, 3],
|
|
"gov_types_excluded": [4, 5],
|
|
"scope_note": "v0.1 covers state, county, city/municipality, and township governments. Special districts (type 4) and school districts (type 5) are excluded pending validation in a future cycle."
|
|
},
|
|
"schema": {
|
|
"long_column_count": 28,
|
|
"long_columns": ["fips_state", "type", "fips_county", "govid", "gov_blank", "gov_name", "county_name", "fips_state_asof", "fips_county_asof", "cog_legacy_state", "cog_legacy_county", "fips_place_code", "population", "popyear", "enrollment", "enrollyear", "function_code", "sch_level_code", "fiscal_year_end", "srvy_year", "item_code", "amt", "srv_data", "impute_flag", "is_aggregate", "canonical_govid", "harmonized_code", "survey_weight"],
|
|
"data_dictionary": "docs/data_dictionary.md"
|
|
},
|
|
"files": {
|
|
"long_partitions": [
|
|
{
|
|
"year": 2011,
|
|
"path": "data/long/year=2011/part-0.parquet",
|
|
"sha256": "7848e18497080c8980a4f89c5b386205b2c5bc90db6773827ea01ab3943d16b1",
|
|
"row_count": 496004,
|
|
"size_bytes": 2202455
|
|
},
|
|
{
|
|
"year": 2012,
|
|
"path": "data/long/year=2012/part-0.parquet",
|
|
"sha256": "b82ac82d5e35f844b26c887445601f3748438c52c998ba4e403b025941a6f170",
|
|
"row_count": 1163338,
|
|
"size_bytes": 5929917
|
|
},
|
|
{
|
|
"year": 2019,
|
|
"path": "data/long/year=2019/part-0.parquet",
|
|
"sha256": "5cbd4726dcc7d0dab5c2a05a64702e979533ae119ed0587073cd31c089e0d737",
|
|
"row_count": 318139,
|
|
"size_bytes": 1719548
|
|
},
|
|
{
|
|
"year": 2020,
|
|
"path": "data/long/year=2020/part-0.parquet",
|
|
"sha256": "ee548fec80bf1beda844fe03916ac145f10dd34c45968407cc330ec260935f00",
|
|
"row_count": 317500,
|
|
"size_bytes": 1722918
|
|
}
|
|
],
|
|
"metadata": [
|
|
{
|
|
"path": "data/canonical_alias.parquet",
|
|
"sha256": "3f617051c23a99bea322889857f7106df0c92954564afeec181df7083ee6698e",
|
|
"description": "canonical_alias.parquet"
|
|
},
|
|
{
|
|
"path": "data/canonical_fips_xwalk.parquet",
|
|
"sha256": "f98742f941269dacf8f7de5c273aa4dd4e75017a5bb70c054da35852a95a8d46",
|
|
"description": "canonical_fips_xwalk.parquet"
|
|
},
|
|
{
|
|
"path": "data/census_collection_coverage.parquet",
|
|
"sha256": "143e025616cde684da7c4442bc00d07fbd1556fabb0ea96223931b737e5d10a4",
|
|
"description": "census_collection_coverage.parquet"
|
|
},
|
|
{
|
|
"path": "data/code_set.parquet",
|
|
"sha256": "4cffcb0198dd51e4ff2b694050bb371a5f9965cdac12f25521cb628fb8e118a9",
|
|
"description": "code_set.parquet"
|
|
},
|
|
{
|
|
"path": "data/harmonization_map.parquet",
|
|
"sha256": "4cf32d0f817079ba4f28dc0ce65450d3247ebbf08d94c0c26c0d02af597bf812",
|
|
"description": "harmonization_map.parquet"
|
|
},
|
|
{
|
|
"path": "data/harmonization_recipes.parquet",
|
|
"sha256": "1133e9a0b02f8f34f5f936e55c5ecd596bb8a55d8425dcce76767f0f3203581c",
|
|
"description": "harmonization_recipes.parquet"
|
|
},
|
|
{
|
|
"path": "data/lineage_events.parquet",
|
|
"sha256": "36c16acfbe621d61010984767f1c566993b8a5f481a2c1e134c4c0a600e4502f",
|
|
"description": "lineage_events.parquet"
|
|
},
|
|
{
|
|
"path": "data/representation.parquet",
|
|
"sha256": "31ec328a7dd505a321b45f97aafff12e53d68a1a986f63509863035b22a4360d",
|
|
"description": "representation.parquet"
|
|
},
|
|
{
|
|
"path": "data/series_breaks.parquet",
|
|
"sha256": "5ae050dd7a76c4d25e5f99e7c2e81c1896482e3504e0443b47ab5d78ba148953",
|
|
"description": "series_breaks.parquet"
|
|
},
|
|
{
|
|
"path": "data/summary_categories.parquet",
|
|
"sha256": "e71d6d70d767c26c983fe56213baf204355f879582aa94841e62d9aea1877f83",
|
|
"description": "summary_categories.parquet"
|
|
}
|
|
]
|
|
},
|
|
"series_breaks_ref": "docs/series_breaks.md",
|
|
"reader_spec_ref": "docs/reader-specification.md"
|
|
}
|