feat: three-concept expenditure model classified by crosswalk membership (#11)
R-CMD-check / check (push) Successful in 3m5s

Rewrites expenditure/revenue classification off item-code first-letter
prefixes and onto summary_categories membership (F-018: prefix Y spans
revenue, expenditure, and balance codes), and exposes
expenditure_concept = c("primary", "direct", "total") with primary as
the new default:

  primary = operations + capital + assistance
  direct  = primary + interest + insurance_benefits   (Census Direct)
  total   = direct + intergovernmental                (M/L/Q via ig views)

- inst/sql: flow views (20-25) select by crosswalk membership;
  summary_categories moves to 11- so it registers before them (DuckDB
  binds view sources eagerly). The IG leg gains Q11/Q12/Q18 state
  school-system payments (F-017).
- R: one subtype scope per verb call drives the verb SQL, the
  harmonization exclusion count, and the complete = TRUE grid;
  flow_prefixes survives only to scope recipe suggestions.
  cog_geographic_rollup/cog_peer_compare accept primary|direct, still
  refuse total, and now actually pass the concept through.
- Balance codes can never reach a spending or revenue result
  (uscogdata#25), asserted at both view and verb level.
- Deletes the #11 skip; per the 2026-07-30 owner ruling the F-018 Y01
  proof is asserted against the crosswalk, not the default
  cog_revenue() call (which stays General Revenue pending #12).

Suite: 696 pass / 0 fail / 1 skip (#12, expected).

Closes #11
This commit is contained in:
2026-07-30 16:56:50 -04:00
parent 7d798b9937
commit 93300ae0c1
25 changed files with 560 additions and 239 deletions
+7 -7
View File
@@ -10,7 +10,7 @@ cog_geographic_rollup(
years,
per_capita = FALSE,
adjust_to_year = NULL,
expenditure_concept = c("direct", "total"),
expenditure_concept = c("primary", "direct", "total"),
coverage = c("all", "census", "consistent")
)
}
@@ -30,12 +30,12 @@ are excluded from the result.}
\item{adjust_to_year}{Integer base year for CPI-U conversion, or `NULL`.}
\item{expenditure_concept}{`"direct"` (default) or `"total"`. Currently only
`"direct"` is accepted; the `"total"` option exists in [cog_spending()] for
single-government queries but cannot be used here because combining Total
across multiple layers of government double-counts intergovernmental
transfers (a state's payment to a school district is the same dollar the
district reports as its own Direct spending).}
\item{expenditure_concept}{`"primary"` (default), `"direct"`, or
`"total"` -- see [cog_spending()] for the three concepts. `"total"` is
refused here because combining Total across multiple layers of
government double-counts intergovernmental transfers (a state's payment
to a school district is the same dollar the district reports as its own
Direct spending); `"primary"` and `"direct"` combine safely.}
\item{coverage}{How to handle the Census of Governments survey cycle,
which is a **complete census only in years ending in 2 and 7** -- every
+6 -5
View File
@@ -11,7 +11,7 @@ cog_peer_compare(
years,
per_capita = TRUE,
adjust_to_year = NULL,
expenditure_concept = c("direct", "total"),
expenditure_concept = c("primary", "direct", "total"),
coverage = c("all", "census", "consistent")
)
}
@@ -30,10 +30,11 @@ population.}
\item{adjust_to_year}{Integer base year for CPI-U conversion or `NULL`.}
\item{expenditure_concept}{`"direct"` (default) or `"total"`. Currently only
`"direct"` is accepted; the `"total"` option exists in [cog_spending()] for
single-government queries but cannot be used here because combining Total
across peer sets counts intergovernmental transfers twice.}
\item{expenditure_concept}{`"primary"` (default), `"direct"`, or
`"total"` -- see [cog_spending()] for the three concepts. `"total"` is
refused here because combining Total across peer sets counts
intergovernmental transfers twice; `"primary"` and `"direct"` combine
safely.}
\item{coverage}{How to handle the Census of Governments survey cycle,
which is a **complete census only in years ending in 2 and 7** -- every
+29 -17
View File
@@ -12,7 +12,7 @@ cog_spending(
adjust_to_year = NULL,
basis = c("harmonized", "raw"),
recipe = NULL,
expenditure_concept = c("direct", "total"),
expenditure_concept = c("primary", "direct", "total"),
complete = FALSE
)
}
@@ -58,22 +58,34 @@ argument is ignored and the result's provenance reports
FALSE`, pointing at the `recipe` block instead) rather than a
possibly-misleading `"harmonized"`/`"raw"` value.}
\item{expenditure_concept}{`"direct"` (default) returns only the
government's own direct spending (item codes `E`/`F`/`G`), unchanged
from prior releases. `"total"` additionally UNIONs in the
intergovernmental leg -- payments to local governments (`M` codes) and
to the state government (`L` codes, excluding the `L--` family-total
rollup) -- so results gain rows with `spend_subtype ==
"intergovernmental"`. Requires the active corpus's `summary_categories`
to carry M/L rows (added by cog_pipeline PR #59); aborts with class
`uscogdata_ig_categories_unsupported` on an older corpus rather than
silently under-reporting. Mutually exclusive with `recipe` (a recipe
already defines its own component codes). **Do not sum `"total"`
results across levels of government** (e.g. state + county + city):
a state's `M12` payment to a school district is the same dollar the
district reports as its own direct `E12`, so summing both double-counts
it. This matters in particular with [cog_geographic_rollup()], which
sums across exactly that kind of multi-layer government set.
\item{expenditure_concept}{Which spending concept to return. Concepts are
defined as sets of the crosswalk's `spend_subtype` values -- never as
item-code first letters, which cannot classify correctly (prefix `Y`
alone spans revenue, expenditure, and balance codes):
* `"primary"` (default) -- the government's own service provision:
`operations` + `capital` + `assistance` subtypes.
* `"direct"` -- Census's published Direct Expenditure: `primary` plus
`interest` (interest on debt) and `insurance_benefits` (insurance
trust benefit payments, e.g. pensions -- Census manual section
5.2.2.1 includes payments to retirees in Direct).
* `"total"` -- `direct` plus the intergovernmental leg: payments to
local governments (`M` codes), to the state government (`L` codes,
excluding the `L--` family-total rollup), and state payments to
school systems (`Q11`/`Q12`/`Q18`), so results gain rows with
`spend_subtype == "intergovernmental"`. Requires the active corpus's
`summary_categories` to carry M/L rows (added by cog_pipeline PR
#59); aborts with class `uscogdata_ig_categories_unsupported` on an
older corpus rather than silently under-reporting. Mutually
exclusive with `recipe` (a recipe already defines its own component
codes).
**Do not sum `"total"` results across levels of government** (e.g.
state + county + city): a state's `M12` payment to a school district is
the same dollar the district reports as its own direct `E12`, so
summing both double-counts it. This matters in particular with
[cog_geographic_rollup()], which sums across exactly that kind of
multi-layer government set.
In the legacy wide era (<= FY2011), some functions are published ONLY
as an aggregate-flagged family total (e.g. Corrections' `E04`/`E05`