jared
c1c6b5a6ba
.build_suggestions()'s candidate query picks recipes by component_code matching the requested category's summary_categories rows, with no flow-prefix filter. Task 1's M04/M05 category rows share the "Corrections" category with the Direct-flavored E04/E05, so corrections_ig_local_combined (entirely M-prefixed) became a raw top-level candidate for a plain (Direct) cog_spending() call. Following that hint would silently return intergovernmental dollars under provenance$expenditure_concept = "direct". Task 6's flow-family gate in .attach_ig_counterparts() already protects the *counterpart* lookup (deciding whether a firing suggestion gets an ig_recipe_id attached) but never touched the candidate list itself. Exclude any recipe with an M/L-prefixed component from candidates unconditionally -- an M/L recipe should never be a coverage-gap filler for either verb, which is a stronger guarantee than the counterpart gate's flow_prefixes check. Confirmed via the full suite: before this fix, a Direct cog_spending() call for category = "Corrections" printed "corrections_ig_local_combined ... re-run with recipe = 'corrections_ig_local_combined'" as its own suggestion; after, it appears only as the "intergovernmental counterpart" annotation on corrections_combined and its capital-outlay siblings. The pre-existing "IG Federal" mis-scoped test (revenue-side B-prefixed recipes) is unaffected -- those aren't M/L, so they remain valid candidates with ig_recipe_id still gated to NULL.
uscogdata
Curated R reader for the Civilytics US Census of Governments finance corpus.
Provides unit-level financial profiles, geographic rollups, and peer comparisons with auditable provenance and built-in cross-vintage correctness. Reads the published corpus (Hive-partitioned parquet + manifest.json) directly from Nextcloud via DuckDB httpfs — no local bulk downloads required.
Status
Under active development (Phase 2 of the cog_pipeline project). See
../cog_pipeline/docs/reader-specification.md for the reader contract this
package implements.
Installation
# pak::pkg_install("gitea.civilytics.org/Civilytics/uscogdata")
Configuration
USCOGDATA_URL— corpus root URL (public Nextcloud share, trailing slash)USCOGDATA_CACHE_DIR— optional override for the manifest cache directoryUSCOGDATA_MANIFEST_TTL_SECS— optional manifest re-fetch TTL (default 3600)
Direct vs Total spending
cog_spending(..., expenditure_concept = c("direct", "total")) controls
whose spending a result counts. "direct" (the default) is a government's
own current operations, capital outlay, and other direct spending. "total"
additionally adds in the intergovernmental legs — money it hands to other
governments to spend on its behalf — which is meaningful for describing one
government's own budget over time, but double-counts when summed across
governments (a state's payment to a county is the same dollar the county
reports as its own direct spending).
Rule of thumb: any figure that spans more than one government uses
direct. cog_geographic_rollup() and cog_peer_compare() enforce this
by refusing expenditure_concept = "total". See
vignette("total-spending", package = "uscogdata") for the full
explanation with worked examples.
Developer notes
Testing
The package ships a bundled fixture corpus at inst/extdata/fixture_corpus/ —
a 3.6 MB two-year slice (2019 + 2020) of the full corpus covering all 50
states. tests/testthat/setup.R automatically points USCOGDATA_URL at this
fixture, so the full test suite runs offline with no network dependency:
devtools::test() # uses bundled fixture, no credentials required
Releasing against the live corpus
Before cutting a release, run the test suite against the published corpus to catch any drift between the fixture and the real data:
Sys.setenv(USCOGDATA_URL = "<published-corpus-url-with-trailing-slash>")
devtools::test()
When the live-corpus run is clean, strip the fixture from the built package by
adding this line to .Rbuildignore:
^inst/extdata/fixture_corpus$
The test suite is URL-agnostic — setup.R falls back to USCOGDATA_URL when
the bundled fixture is absent, so no test code changes are needed for the
release run or after stripping the fixture.