From b03f095e496e9c7169e60a1ffb11b1a77ae04f27 Mon Sep 17 00:00:00 2001 From: Jared Knowles Date: Mon, 3 Aug 2026 11:02:16 -0400 Subject: [PATCH] docs: document cog_balances() and correct stale CLAUDE.md claims (#25) Adds the NEWS entry, a Financial data pkgdown reference section (none existed for cog_spending/cog_revenue), and corrects CLAUDE.md's SQL-layer claim, view count, test count and fixture-year description against measured values. Also documents balance_caveats in inst/schemas/provenance-v1.json (test-first: added a schema-documentation test to test-balances.R, confirmed it failed, then fixed the schema) and fleshes out cog_balances()'s @return roxygen to enumerate its conditional columns, regenerating man/cog_balances.Rd. --- CLAUDE.md | 38 +++++++++++++++++++++++++-------- NEWS.md | 13 +++++++++++ R/balances.R | 15 +++++++++++-- _pkgdown.yml | 6 ++++++ inst/schemas/provenance-v1.json | 10 +++++++++ man/cog_balances.Rd | 15 +++++++++++-- tests/testthat/test-balances.R | 8 +++++++ 7 files changed, 92 insertions(+), 13 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index d739dd8..51b0ee7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,8 +28,23 @@ USCOGDATA_URL (local path or https://) - `R/session.R` — `cog_open()`, `cog_close()`, `.ensure_session()`, `.coerce_govid_input()` - `R/manifest.R` — `.fetch_or_cache_manifest()`, `.is_local_path()` (local paths bypass HTTP/cache) - `R/views.R` — `.register_views()` (substitutes `{url}` into SQL files at `inst/sql/`) -- `inst/sql/` — 7 SQL view definitions: `long`, `spending_long`, `revenue_long`, `canonical_fips_xwalk`, `summary_categories`, `spending_annotated`, `revenue_annotated` +- `inst/sql/` — **23** SQL view definitions (measured), numbered by load order + (`10-` through `46-`): the `*_long` layer (`long`, `spending_long`, + `revenue_long`, `ig_long`, `balance_long`, plus `_harmonized` variants of + `spending_long`/`revenue_long`/`ig_long`), the `*_annotated` layer + (`spending_annotated`, `revenue_annotated`, `ig_annotated`, + `balance_annotated`, plus `_harmonized` variants of `spending_annotated`/ + `revenue_annotated`/`ig_annotated`), and metadata views + (`canonical_fips_xwalk`, `summary_categories`, `gov_population_yearly`, + `harmonization_map`, `harmonization_recipes`, `series_breaks_pq`, + `representation`, `code_set`) - `R/spending.R` / `R/revenue.R` — `cog_spending()` / `cog_revenue()` via shared `.verb_spendrev()` +- `R/balances.R` — `cog_balances()`. A third money-adjacent verb, but returns a + **stock** (a balance at a point in time) rather than a **flow** (activity + over a fiscal year), so it does NOT route through `.verb_spendrev()` and has + no `expenditure_concept`/`revenue_concept`/`complete`/`subtype` arguments. + `R/balance_caveats.R` attaches `provenance$balance_caveats` (GAAP-vs-gross + disclosure + measured per-subtype coverage windows). - `R/rollup.R` — `cog_geographic_rollup()` (accepts named list of govids by layer) - `R/peers.R` — `cog_find_peers()` + `cog_peer_compare()` - `R/search.R` — `cog_gov_search()` (name pattern, state, type filters) @@ -49,18 +64,19 @@ Any value without `://` is treated as a local path by `.is_local_path()` and rea **Version:** 0.1.0 (pre-release) **Branch:** `main`, commit `d65e9fe` -**Tests:** 181 PASS / 0 FAIL / 0 SKIP +**Tests:** 763 PASS / 0 FAIL / 0 SKIP (measured `testthat::test_local()`, 2026-08-03) **CI:** Gitea Actions green (`.gitea/workflows/ci.yml`) ### Completed (Tasks 2.1–2.7) -All 8 exported verbs implemented and tested: -`cog_spending`, `cog_revenue`, `cog_explain`, `cog_geographic_rollup`, -`cog_find_peers`, `cog_peer_compare`, `cog_gov_search`, `cog_mirror`, -plus `cog_categories`. +All 10 exported verbs implemented and tested: +`cog_spending`, `cog_revenue`, `cog_balances`, `cog_explain`, +`cog_geographic_rollup`, `cog_find_peers`, `cog_peer_compare`, +`cog_gov_search`, `cog_mirror`, plus `cog_categories`. -Bundled fixture corpus at `inst/extdata/fixture_corpus/` (3.6 MB, years -2019+2020, all 50 states). Tests run fully offline — no credentials needed. +Bundled fixture corpus at `inst/extdata/fixture_corpus/` (years +2011, 2012, 2019, 2020 — measured via DuckDB `read_parquet(hive_partitioning=1)`, +2026-08-03; all 50 states). Tests run fully offline — no credentials needed. ### Remaining to v0.1 release @@ -99,6 +115,10 @@ devtools::test() - All verbs call `.ensure_session()` first, then query via `DBI::dbGetQuery()` - Return value is always a `tbl_df` with a `provenance` attribute - govid inputs always go through `.coerce_govid_input()` (accepts character or data frame) -- SQL lives in `inst/sql/` — never inline SQL strings in R files +- SQL has two layers. **View definitions** live in `inst/sql/` and are + registered by `.register_views()`, which globs the directory in sorted order + and substitutes `{url}`. **Query construction** is inline `sprintf()` in R + (`.build_verb_sql()`, `.run_recipe()`, `.attach_per_capita()`). Add a view as + a numbered `.sql` file; build a query in R. - No arrow dependency — DuckDB reads parquet natively - `withr` is a Suggests-only dep; only used in tests diff --git a/NEWS.md b/NEWS.md index 97c7430..88e8de2 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,5 +1,18 @@ # uscogdata 0.1.0 (development) +## New: `cog_balances()` for cash-and-security holdings + +* New `cog_balances()` exposes the 14 cash-and-security holding codes + (`category_type = "balance"`): fund balances, retirement system holdings and + insurance trust balances (#25). Holdings are a stock, not a flow, so the verb + has no `expenditure_concept` / `revenue_concept` / `complete` arguments, and + no `subtype` argument either -- for holdings, `category` is a strict + coarsening of `balance_subtype`, so `category = "Fund Balances"` is exactly + the `general` family (`W01`/`W31`/`W61`). +* `cog_balances()` results carry `provenance$balance_caveats`, recording that + Census holdings are gross rather than GAAP fund balance, and the measured + coverage window of each subtype family. + ## Multi-government aggregates now disclose their reporting coverage * The Census of Governments is a **complete census only in years ending in 2 diff --git a/R/balances.R b/R/balances.R index 7e8c7f5..f521a7f 100644 --- a/R/balances.R +++ b/R/balances.R @@ -41,8 +41,19 @@ #' `"cash_securities_z77_wide"` and `"cash_securities_z78_wide"` bridge the #' wide era to the modern one. #' -#' @return A `tbl_df` with a `provenance` attribute. Amounts are full US -#' dollars. +#' @return Tibble with columns `year`, `canonical_govid`, `gov_name`, +#' `balance_subtype`, `category`, `amt_nominal`, optional +#' `amt_per_capita_nominal` and `pop_source` (when `per_capita = TRUE`), +#' optional `amt_real` and `amt_per_capita_real` (when `adjust_to_year` is +#' set), `codes_included`, `aggregate_fallback`, `notes`. Amounts are full +#' US dollars. +#' +#' Carries a `provenance` attribute matching +#' `inst/schemas/provenance-v1.json`, whose `balance_caveats` block reports +#' `not_gaap`, `not_gaap_note`, `coverage_window` (measured per-subtype year +#' extents) and `truncated` (subtypes whose coverage falls short of the +#' requested years). `expenditure_concept`/`revenue_concept` are `NA` -- +#' holdings are a stock, not a flow, so neither concept vocabulary applies. #' @export cog_balances <- function(govid, years, category = NULL, per_capita = FALSE, adjust_to_year = NULL, diff --git a/_pkgdown.yml b/_pkgdown.yml index 2022548..1541211 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -3,6 +3,12 @@ template: bootstrap: 5 reference: + - title: Financial data + desc: Spending, revenue and balance-sheet holdings for one or more governments. + contents: + - cog_spending + - cog_revenue + - cog_balances - title: Search & basket desc: Resolve place names into canonical govids. contents: diff --git a/inst/schemas/provenance-v1.json b/inst/schemas/provenance-v1.json index 779fad6..a430cd5 100644 --- a/inst/schemas/provenance-v1.json +++ b/inst/schemas/provenance-v1.json @@ -53,6 +53,16 @@ "items": { "type": "string" }, "description": "Ids of catalogued series breaks whose fin_code is the literal 'ALL' -- caveats about the corpus as a whole (dollar precision across 1976/1977, imputation exclusion from 2002, the dense -> sparse representation change at 2012, the government id scheme change at 2017) rather than about one item code. Selected on the break_year window alone, so they do not depend on which codes a result contains. Disjoint from series_break_refs by construction: an entry qualifies the whole result, not one series." }, + "balance_caveats": { + "type": ["object", "null"], + "description": "Present only on cog_balances() results (null/absent for cog_spending()/cog_revenue()). `not_gaap` is always TRUE and `not_gaap_note` explains that Census holdings are gross -- no liabilities are netted -- so they are NOT comparable to a GAAP fund balance. `coverage_window` maps each observed balance_subtype to its measured [min year, max year] in the mounted corpus (never hardcoded). `truncated` lists the subtypes whose coverage_window does not fully span the requested years.", + "properties": { + "not_gaap": { "type": "boolean" }, + "not_gaap_note": { "type": "string" }, + "coverage_window": { "type": "object" }, + "truncated": { "type": "array", "items": { "type": "string" } } + } + }, "manifest": { "type": "object" }, "sql_query": { "type": "string" } } diff --git a/man/cog_balances.Rd b/man/cog_balances.Rd index 81323a0..f3e40b9 100644 --- a/man/cog_balances.Rd +++ b/man/cog_balances.Rd @@ -46,8 +46,19 @@ and raw space are identical for holdings. Reported in wide era to the modern one.} } \value{ -A `tbl_df` with a `provenance` attribute. Amounts are full US - dollars. +Tibble with columns `year`, `canonical_govid`, `gov_name`, + `balance_subtype`, `category`, `amt_nominal`, optional + `amt_per_capita_nominal` and `pop_source` (when `per_capita = TRUE`), + optional `amt_real` and `amt_per_capita_real` (when `adjust_to_year` is + set), `codes_included`, `aggregate_fallback`, `notes`. Amounts are full + US dollars. + + Carries a `provenance` attribute matching + `inst/schemas/provenance-v1.json`, whose `balance_caveats` block reports + `not_gaap`, `not_gaap_note`, `coverage_window` (measured per-subtype year + extents) and `truncated` (subtypes whose coverage falls short of the + requested years). `expenditure_concept`/`revenue_concept` are `NA` -- + holdings are a stock, not a flow, so neither concept vocabulary applies. } \description{ Returns Census cash-and-security holdings (`category_type = "balance"`): diff --git a/tests/testthat/test-balances.R b/tests/testthat/test-balances.R index b0c850d..fe609ee 100644 --- a/tests/testthat/test-balances.R +++ b/tests/testthat/test-balances.R @@ -359,6 +359,14 @@ test_that("a request past a family's coverage window is flagged", { }) }) +test_that("the provenance schema documents balance_caveats", { + sch <- jsonlite::fromJSON( + system.file("schemas", "provenance-v1.json", package = "uscogdata"), + simplifyVector = FALSE + ) + expect_true("balance_caveats" %in% names(sch$properties)) +}) + test_that("the caveat message fires once per session", { skip_if_no_corpus() with_fixture_corpus({