feat(coverage): n_units_collected separates sampling from real zeros (#36)
R-CMD-check / check (push) Successful in 3m39s
R-CMD-check / check (pull_request) Successful in 3m42s

provenance$coverage's n_units_reporting is category-conditional: it
counts governments with rows for the SPECIFIC requested category, which
conflates two different things -- a government never collected that
year (sampling), and one collected but genuinely spending nothing in
that category (a real zero). FY2012 Georgia Police is the motivating
case from the issue: a complete census year reads as a 69% "response
rate" because most of the gap is cities that contract policing to the
county sheriff, not non-response.

Adds a second counter, n_units_collected: how many of the caller's
expected cohort appear in the corpus that year for ANY category.
n_units_collected / n_units_expected is the true collection rate;
n_units_reporting / n_units_collected is category participation among
collected units. cog_geographic_rollup() and cog_peer_compare() both
carry it; cog_explain() prints it alongside n_units_reporting.

Two real bugs caught and fixed while finishing this (both against the
already-written, previously-uncommitted draft):

- .coverage_table()'s candidate list for the collection query was
  derived from the category-filtered result rows, not the caller's
  full expected cohort. A government with zero rows in the requested
  category across every requested year never appears in that result,
  so it was silently excluded from n_units_collected too -- collapsing
  the new counter back to the old, broken one for exactly the
  governments it exists to count. Fixed by threading an explicit
  `expected_ids` (all_govids / peer_govids) through instead.
- The collection query hardcoded long_view = "spending_long_harmonized",
  which does not exist on a corpus with schema_version < 5 (R/basis.R
  resolves basis = "raw" there; R/views.R only registers the
  harmonized views on v5+). cog_geographic_rollup()/cog_peer_compare()
  would hard-error on a corpus vintage the package otherwise explicitly
  supports. Fixed by deriving long_view from the basis cog_spending()
  actually resolved (prov$basis) via the existing .select_long_view()
  helper, matching how every other basis-aware query in the package
  already does this.

Also: cog_explain()'s general "complete census only in years ending in
2 or 7" footnote was gated on the OLD counter's absence, making it
permanently unreachable now that both callers always supply the new
one -- ungated it, since the explanation is orthogonal to which
counter set is present. Dropped a dead conditional branch, fixed two
stale roxygen blocks in R/peers.R/R/rollup.R still describing the old
two-counter model, fixed the same staleness in README.md, and switched
two `uscogdata:::` self-references to the package's own convention of
calling internal helpers unqualified.

1101 tests pass (2 skipped live-corpus), including new direct
regression tests for both bugs above (one exercising a government
collected-but-absent from a category result, one running the full
rollup/peer-compare path against a doctored schema_version 4 corpus).

Reviewed by an independent code-reviewer pass (1 HIGH, 1 MEDIUM, 3 LOW
-- all addressed above).

Closes #36.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-09 11:46:17 -04:00
co-authored by Claude Sonnet 5
parent d0d724c4c8
commit b41d5ee2aa
9 changed files with 348 additions and 65 deletions
+23 -12
View File
@@ -55,11 +55,13 @@ Direct spending); `"primary"` and `"direct"` combine safely.}
a balanced panel.
Regardless of mode, `provenance$coverage` always carries per-year
`n_units_reporting`, `n_units_expected` and `is_census_year`, and
`provenance$coverage_mode` records the mode. `is_census_year` is a
statement about the **survey calendar**, never a claim of completeness:
FY1967 is a census year in which only 97 of Wisconsin's 608 cities
report. `n_units_reporting` is the number that tells the truth.}
`n_units_expected`, `n_units_collected`, `n_units_reporting` and
`is_census_year`, and `provenance$coverage_mode` records the mode.
`is_census_year` is a statement about the **survey calendar**, never a
claim of completeness: FY1967 is a census year in which only 97 of
Wisconsin's 608 cities report. `n_units_reporting` is
category-conditional and is not a response rate on its own -- see
"Reading `coverage`" below for what each counter answers.}
}
\value{
Tibble with columns `year`, `layer`, `canonical_govid`, `gov_name`,
@@ -86,14 +88,23 @@ by design — see `vignette('population-denominators')`.
}
\section{Reading `coverage`}{
`provenance$coverage` reports `n_units_reporting` against
`n_units_expected` per year. **`n_units_reporting` is category-conditional:
it counts governments with rows for the category you asked for, not
governments collected that year.** A government that was surveyed and
genuinely spends nothing in that category is indistinguishable here from one
that was never surveyed.
`provenance$coverage` carries three per-year counters:
* `n_units_expected` -- how many governments you asked about.
* `n_units_collected` -- how many of those appear in the corpus at all
that year (in ANY category), separating sampling from real zeros.
* `n_units_reporting` -- how many have rows for the SPECIFIC category you
requested. This is always <= n_units_collected: a government can be
collected but have no rows for "Police" because it contracts policing
to the county sheriff, not because it wasn't surveyed.
**`n_units_reporting` is category-conditional** and therefore **not a
response rate**: `n_units_reporting / n_units_expected` conflates sampling
(never collected) with real zeros (collected but spends nothing in your
category). Use `n_units_collected / n_units_expected` for the true
collection rate, and `n_units_reporting / n_units_collected` for category
participation among collected units.
The ratio is therefore **not a response rate** and must not be used as one.
In FY2022 — a complete census year — Georgia reports 393 of 567 cities for
`category = "Police"`; the 174-city gap is overwhelmingly cities that
contract policing to the county sheriff, not non-response.
+23 -12
View File
@@ -50,11 +50,13 @@ safely.}
a balanced panel.
Regardless of mode, `provenance$coverage` always carries per-year
`n_units_reporting`, `n_units_expected` and `is_census_year`, and
`provenance$coverage_mode` records the mode. `is_census_year` is a
statement about the **survey calendar**, never a claim of completeness:
FY1967 is a census year in which only 97 of Wisconsin's 608 cities
report. `n_units_reporting` is the number that tells the truth.
`n_units_expected`, `n_units_collected`, `n_units_reporting` and
`is_census_year`, and `provenance$coverage_mode` records the mode.
`is_census_year` is a statement about the **survey calendar**, never a
claim of completeness: FY1967 is a census year in which only 97 of
Wisconsin's 608 cities report. `n_units_reporting` is
category-conditional and is not a response rate on its own -- see
"Reading `coverage`" below for what each counter answers.
The comparison target is exempt from `"consistent"` balancing -- it is the
subject of the comparison, not a member of the cohort -- and the
@@ -109,14 +111,23 @@ them.
}
\section{Reading `coverage`}{
`provenance$coverage` reports `n_units_reporting` against
`n_units_expected` per year. **`n_units_reporting` is category-conditional:
it counts cohort members with rows for the category you asked for, not
cohort members collected that year.** A government that was surveyed and
genuinely spends nothing in that category is indistinguishable here from one
that was never surveyed.
`provenance$coverage` carries three per-year counters:
* `n_units_expected` -- how many governments you asked about.
* `n_units_collected` -- how many of those appear in the corpus at all
that year (in ANY category), separating sampling from real zeros.
* `n_units_reporting` -- how many have rows for the SPECIFIC category you
requested. This is always <= n_units_collected: a government can be
collected but have no rows for "Police" because it contracts policing
to the county sheriff, not because it wasn't surveyed.
**`n_units_reporting` is category-conditional** and therefore **not a
response rate**: `n_units_reporting / n_units_expected` conflates sampling
(never collected) with real zeros (collected but spends nothing in your
category). Use `n_units_collected / n_units_expected` for the true
collection rate, and `n_units_reporting / n_units_collected` for category
participation among collected units.
The ratio is therefore **not a response rate** and must not be used as one.
In FY2022 — a complete census year — Georgia reports 393 of 567 cities for
`category = "Police"`; the 174-city gap is overwhelmingly cities that
contract policing to the county sheriff, not non-response.