fix: exercise real harmonized-view SQL in tests; unambiguous recipe provenance
test-views.R's harmonized-view test previously ran a hand-rolled REPLACE
query with no WHERE clause, so a regression in any of
inst/sql/22-spending_long_harmonized.sql / 23-revenue_long_harmonized.sql's
three predicates (NOT is_aggregate, harmonized_code IS NOT NULL, the
E/F/G/K or T/A/U/B/C/D prefix filter) would go uncaught. Replaced it with a
test that reads the real SQL files off disk, substitutes {url} exactly as
.register_views() does, and executes them (plus their 10-long.sql
dependency) against a synthetic hive-partitioned parquet tree written via
DuckDB's own COPY ... TO (FORMAT PARQUET) (no arrow dependency, matching
this package's existing convention). Ten rows are crafted so each predicate
is independently falsifiable by a specific row; manually broke each
predicate in turn to confirm the test fails exactly as expected, then
restored the SQL files (see the task report for the RED-phase transcript).
Also fixes a provenance ambiguity: a recipe= query bypasses
spending_annotated(_harmonized)/revenue_annotated(_harmonized) entirely
(.run_recipe() joins `long` directly), so basis= has no effect on it, but
provenance was still reporting basis = "harmonized"/"raw" (whatever the
argument resolved to) with harmonization$applied = FALSE alongside it --
misleading, since it looks like harmonization was evaluated and found
nothing to exclude rather than "not applicable here." Recipe results now
report basis = "recipe" with an inert harmonization block carrying an
explicit note, regardless of what basis= was passed.
This commit is contained in:
+9
-2
@@ -40,14 +40,21 @@ reproduces the pre-Phase-R2 behavior (published item codes, no
|
||||
folding). On a corpus with `schema_version < 5` (no harmonization
|
||||
tables), `basis` silently resolves to `"raw"` when left at its default
|
||||
and the resolution is recorded in the provenance; explicitly passing
|
||||
`basis = "harmonized"` on such a corpus aborts.}
|
||||
`basis = "harmonized"` on such a corpus aborts. Ignored when `recipe`
|
||||
is set (see below).}
|
||||
|
||||
\item{recipe}{Optional harmonization recipe id (see [cog_recipes()]) for
|
||||
multi-code cross-vintage series that a 1:1 harmonized_code mapping
|
||||
can't express (e.g. a wide-era aggregate that only splits into leaf
|
||||
codes in the modern era). Mutually exclusive with `category`. The
|
||||
result's subtype column reads `"recipe"` and `category` reads the
|
||||
recipe's label. Requires `schema_version >= 5`.}
|
||||
recipe's label. Requires `schema_version >= 5`. A recipe query bypasses
|
||||
`basis` entirely (it joins `long` directly rather than going through
|
||||
the `*_annotated`/`*_annotated_harmonized` views), so the `basis`
|
||||
argument is ignored and the result's provenance reports
|
||||
`basis = "recipe"` with an inert `harmonization` block (`applied =
|
||||
FALSE`, pointing at the `recipe` block instead) rather than a
|
||||
possibly-misleading `"harmonized"`/`"raw"` value.}
|
||||
}
|
||||
\value{
|
||||
Tibble with columns `year`, `canonical_govid`, `gov_name`,
|
||||
|
||||
Reference in New Issue
Block a user