feat: accept corpus schema v6 (FIPS geography harmonization); v6 fixture
Schema v6 (cog_pipeline 2026-07-22) renamed the long table's
fips_state_code/fips_county_code to fips_state_asof/fips_county_asof and added
cog_legacy_state/cog_legacy_county (26 -> 28 cols). This package references
none of those columns and its geography always came from
canonical_fips_xwalk (already present-based), so acceptance is a version-set
bump: supported = c(4L, 5L) -> c(4L, 5L, 6L) in .validate_schema() and
cog_open(). A prominent note in .validate_schema() documents the SILENT
semantic change for raw-long readers: long fips_state/fips_county are now
PRESENT/harmonized geography (carried back per government), not as-of-year.
Fixture regenerated from the published v6 tree (schema_version 6, 28 cols).
Test updates:
* test-manifest.R: v6 accepted; boundary rejection moves to v7.
* test-spending.R: the na_rows_excluded pin (0) predated the Task 18 map
extension, which added E/F/G-prefix discontinued_na rulings (E21/F21/G21,
Education NEC local, SB184-186). Broward's 2011 partition zero-pads
exactly those codes: 3 NA-harmonized rows excluded, all amt=0, so the
excluded AMOUNT pin stays 0. Data-verified against the v6 fixture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -111,15 +111,20 @@ test_that("cog_manifest returns the active session's parsed manifest", {
|
||||
})
|
||||
})
|
||||
|
||||
test_that(".validate_schema accepts schema_version 4 and 5, rejects others", {
|
||||
test_that(".validate_schema accepts schema_version 4, 5 and 6, rejects others", {
|
||||
expect_silent(uscogdata:::.validate_schema(list(schema_version = 4L)))
|
||||
expect_silent(uscogdata:::.validate_schema(list(schema_version = 5L)))
|
||||
# v6 = FIPS geography harmonization (2026-07-22): _code -> _asof rename +
|
||||
# cog_legacy_* columns (26 -> 28 cols). This package references none of the
|
||||
# renamed columns and its geography comes from the xwalk, so v6 is accepted
|
||||
# without behavioural change -- see .validate_schema()'s note.
|
||||
expect_silent(uscogdata:::.validate_schema(list(schema_version = 6L)))
|
||||
expect_error(
|
||||
uscogdata:::.validate_schema(list(schema_version = 3L)),
|
||||
"schema_version"
|
||||
)
|
||||
expect_error(
|
||||
uscogdata:::.validate_schema(list(schema_version = 6L)),
|
||||
uscogdata:::.validate_schema(list(schema_version = 7L)),
|
||||
"schema_version"
|
||||
)
|
||||
})
|
||||
|
||||
@@ -250,12 +250,17 @@ test_that("provenance carries basis + harmonization block with na_rows_excluded"
|
||||
expect_true(prov$harmonization$applied)
|
||||
expect_true(prov$harmonization$na_rows_excluded >= 0L)
|
||||
expect_true(prov$harmonization$na_amount_excluded >= 0)
|
||||
# Data-verified for this fixture: none of the discontinued_na rulings
|
||||
# (S74, Z61, X04, X06, the debt-detail family, L24) fall inside the
|
||||
# E/F/G/K spending prefixes, so the exclusion count is exactly zero for
|
||||
# every year in the bundled window -- see
|
||||
# docs/phase_r_harmonization_review.md § 1.3/1.4.
|
||||
expect_equal(prov$harmonization$na_rows_excluded, 0L)
|
||||
# Data-verified for the v6 fixture (corpus 2026-07-22). The Task 18 map
|
||||
# extension added E/F/G-prefix discontinued_na rulings the earlier pin's
|
||||
# comment predated: E21/F21/G21 (Education NEC local, SB184-186,
|
||||
# "trivial; explicit-NA, full wide-era window"). Broward's 2011 legacy
|
||||
# partition zero-pads exactly those three codes, so this query now
|
||||
# excludes 3 NA-harmonized rows -- all with amt = 0, hence the excluded
|
||||
# AMOUNT stays exactly zero. (The other discontinued_na rulings -- S74,
|
||||
# Z61, X04, X06, the debt-detail family, L24 -- remain outside the
|
||||
# E/F/G/K prefixes.) See docs/phase_r_harmonization_review.md § 1.3/1.4
|
||||
# and cog_pipeline data/harmonization_map.csv E21/F21/G21 rows.
|
||||
expect_equal(prov$harmonization$na_rows_excluded, 3L)
|
||||
expect_equal(prov$harmonization$na_amount_excluded, 0)
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user