docs: document cog_balances() and correct stale CLAUDE.md claims (#25)
Adds the NEWS entry, a Financial data pkgdown reference section (none existed for cog_spending/cog_revenue), and corrects CLAUDE.md's SQL-layer claim, view count, test count and fixture-year description against measured values. Also documents balance_caveats in inst/schemas/provenance-v1.json (test-first: added a schema-documentation test to test-balances.R, confirmed it failed, then fixed the schema) and fleshes out cog_balances()'s @return roxygen to enumerate its conditional columns, regenerating man/cog_balances.Rd.
This commit is contained in:
@@ -28,8 +28,23 @@ USCOGDATA_URL (local path or https://)
|
|||||||
- `R/session.R` — `cog_open()`, `cog_close()`, `.ensure_session()`, `.coerce_govid_input()`
|
- `R/session.R` — `cog_open()`, `cog_close()`, `.ensure_session()`, `.coerce_govid_input()`
|
||||||
- `R/manifest.R` — `.fetch_or_cache_manifest()`, `.is_local_path()` (local paths bypass HTTP/cache)
|
- `R/manifest.R` — `.fetch_or_cache_manifest()`, `.is_local_path()` (local paths bypass HTTP/cache)
|
||||||
- `R/views.R` — `.register_views()` (substitutes `{url}` into SQL files at `inst/sql/`)
|
- `R/views.R` — `.register_views()` (substitutes `{url}` into SQL files at `inst/sql/`)
|
||||||
- `inst/sql/` — 7 SQL view definitions: `long`, `spending_long`, `revenue_long`, `canonical_fips_xwalk`, `summary_categories`, `spending_annotated`, `revenue_annotated`
|
- `inst/sql/` — **23** SQL view definitions (measured), numbered by load order
|
||||||
|
(`10-` through `46-`): the `*_long` layer (`long`, `spending_long`,
|
||||||
|
`revenue_long`, `ig_long`, `balance_long`, plus `_harmonized` variants of
|
||||||
|
`spending_long`/`revenue_long`/`ig_long`), the `*_annotated` layer
|
||||||
|
(`spending_annotated`, `revenue_annotated`, `ig_annotated`,
|
||||||
|
`balance_annotated`, plus `_harmonized` variants of `spending_annotated`/
|
||||||
|
`revenue_annotated`/`ig_annotated`), and metadata views
|
||||||
|
(`canonical_fips_xwalk`, `summary_categories`, `gov_population_yearly`,
|
||||||
|
`harmonization_map`, `harmonization_recipes`, `series_breaks_pq`,
|
||||||
|
`representation`, `code_set`)
|
||||||
- `R/spending.R` / `R/revenue.R` — `cog_spending()` / `cog_revenue()` via shared `.verb_spendrev()`
|
- `R/spending.R` / `R/revenue.R` — `cog_spending()` / `cog_revenue()` via shared `.verb_spendrev()`
|
||||||
|
- `R/balances.R` — `cog_balances()`. A third money-adjacent verb, but returns a
|
||||||
|
**stock** (a balance at a point in time) rather than a **flow** (activity
|
||||||
|
over a fiscal year), so it does NOT route through `.verb_spendrev()` and has
|
||||||
|
no `expenditure_concept`/`revenue_concept`/`complete`/`subtype` arguments.
|
||||||
|
`R/balance_caveats.R` attaches `provenance$balance_caveats` (GAAP-vs-gross
|
||||||
|
disclosure + measured per-subtype coverage windows).
|
||||||
- `R/rollup.R` — `cog_geographic_rollup()` (accepts named list of govids by layer)
|
- `R/rollup.R` — `cog_geographic_rollup()` (accepts named list of govids by layer)
|
||||||
- `R/peers.R` — `cog_find_peers()` + `cog_peer_compare()`
|
- `R/peers.R` — `cog_find_peers()` + `cog_peer_compare()`
|
||||||
- `R/search.R` — `cog_gov_search()` (name pattern, state, type filters)
|
- `R/search.R` — `cog_gov_search()` (name pattern, state, type filters)
|
||||||
@@ -49,18 +64,19 @@ Any value without `://` is treated as a local path by `.is_local_path()` and rea
|
|||||||
|
|
||||||
**Version:** 0.1.0 (pre-release)
|
**Version:** 0.1.0 (pre-release)
|
||||||
**Branch:** `main`, commit `d65e9fe`
|
**Branch:** `main`, commit `d65e9fe`
|
||||||
**Tests:** 181 PASS / 0 FAIL / 0 SKIP
|
**Tests:** 763 PASS / 0 FAIL / 0 SKIP (measured `testthat::test_local()`, 2026-08-03)
|
||||||
**CI:** Gitea Actions green (`.gitea/workflows/ci.yml`)
|
**CI:** Gitea Actions green (`.gitea/workflows/ci.yml`)
|
||||||
|
|
||||||
### Completed (Tasks 2.1–2.7)
|
### Completed (Tasks 2.1–2.7)
|
||||||
|
|
||||||
All 8 exported verbs implemented and tested:
|
All 10 exported verbs implemented and tested:
|
||||||
`cog_spending`, `cog_revenue`, `cog_explain`, `cog_geographic_rollup`,
|
`cog_spending`, `cog_revenue`, `cog_balances`, `cog_explain`,
|
||||||
`cog_find_peers`, `cog_peer_compare`, `cog_gov_search`, `cog_mirror`,
|
`cog_geographic_rollup`, `cog_find_peers`, `cog_peer_compare`,
|
||||||
plus `cog_categories`.
|
`cog_gov_search`, `cog_mirror`, plus `cog_categories`.
|
||||||
|
|
||||||
Bundled fixture corpus at `inst/extdata/fixture_corpus/` (3.6 MB, years
|
Bundled fixture corpus at `inst/extdata/fixture_corpus/` (years
|
||||||
2019+2020, all 50 states). Tests run fully offline — no credentials needed.
|
2011, 2012, 2019, 2020 — measured via DuckDB `read_parquet(hive_partitioning=1)`,
|
||||||
|
2026-08-03; all 50 states). Tests run fully offline — no credentials needed.
|
||||||
|
|
||||||
### Remaining to v0.1 release
|
### Remaining to v0.1 release
|
||||||
|
|
||||||
@@ -99,6 +115,10 @@ devtools::test()
|
|||||||
- All verbs call `.ensure_session()` first, then query via `DBI::dbGetQuery()`
|
- All verbs call `.ensure_session()` first, then query via `DBI::dbGetQuery()`
|
||||||
- Return value is always a `tbl_df` with a `provenance` attribute
|
- Return value is always a `tbl_df` with a `provenance` attribute
|
||||||
- govid inputs always go through `.coerce_govid_input()` (accepts character or data frame)
|
- govid inputs always go through `.coerce_govid_input()` (accepts character or data frame)
|
||||||
- SQL lives in `inst/sql/` — never inline SQL strings in R files
|
- SQL has two layers. **View definitions** live in `inst/sql/` and are
|
||||||
|
registered by `.register_views()`, which globs the directory in sorted order
|
||||||
|
and substitutes `{url}`. **Query construction** is inline `sprintf()` in R
|
||||||
|
(`.build_verb_sql()`, `.run_recipe()`, `.attach_per_capita()`). Add a view as
|
||||||
|
a numbered `.sql` file; build a query in R.
|
||||||
- No arrow dependency — DuckDB reads parquet natively
|
- No arrow dependency — DuckDB reads parquet natively
|
||||||
- `withr` is a Suggests-only dep; only used in tests
|
- `withr` is a Suggests-only dep; only used in tests
|
||||||
|
|||||||
@@ -1,5 +1,18 @@
|
|||||||
# uscogdata 0.1.0 (development)
|
# uscogdata 0.1.0 (development)
|
||||||
|
|
||||||
|
## New: `cog_balances()` for cash-and-security holdings
|
||||||
|
|
||||||
|
* New `cog_balances()` exposes the 14 cash-and-security holding codes
|
||||||
|
(`category_type = "balance"`): fund balances, retirement system holdings and
|
||||||
|
insurance trust balances (#25). Holdings are a stock, not a flow, so the verb
|
||||||
|
has no `expenditure_concept` / `revenue_concept` / `complete` arguments, and
|
||||||
|
no `subtype` argument either -- for holdings, `category` is a strict
|
||||||
|
coarsening of `balance_subtype`, so `category = "Fund Balances"` is exactly
|
||||||
|
the `general` family (`W01`/`W31`/`W61`).
|
||||||
|
* `cog_balances()` results carry `provenance$balance_caveats`, recording that
|
||||||
|
Census holdings are gross rather than GAAP fund balance, and the measured
|
||||||
|
coverage window of each subtype family.
|
||||||
|
|
||||||
## Multi-government aggregates now disclose their reporting coverage
|
## Multi-government aggregates now disclose their reporting coverage
|
||||||
|
|
||||||
* The Census of Governments is a **complete census only in years ending in 2
|
* The Census of Governments is a **complete census only in years ending in 2
|
||||||
|
|||||||
+13
-2
@@ -41,8 +41,19 @@
|
|||||||
#' `"cash_securities_z77_wide"` and `"cash_securities_z78_wide"` bridge the
|
#' `"cash_securities_z77_wide"` and `"cash_securities_z78_wide"` bridge the
|
||||||
#' wide era to the modern one.
|
#' wide era to the modern one.
|
||||||
#'
|
#'
|
||||||
#' @return A `tbl_df` with a `provenance` attribute. Amounts are full US
|
#' @return Tibble with columns `year`, `canonical_govid`, `gov_name`,
|
||||||
#' dollars.
|
#' `balance_subtype`, `category`, `amt_nominal`, optional
|
||||||
|
#' `amt_per_capita_nominal` and `pop_source` (when `per_capita = TRUE`),
|
||||||
|
#' optional `amt_real` and `amt_per_capita_real` (when `adjust_to_year` is
|
||||||
|
#' set), `codes_included`, `aggregate_fallback`, `notes`. Amounts are full
|
||||||
|
#' US dollars.
|
||||||
|
#'
|
||||||
|
#' Carries a `provenance` attribute matching
|
||||||
|
#' `inst/schemas/provenance-v1.json`, whose `balance_caveats` block reports
|
||||||
|
#' `not_gaap`, `not_gaap_note`, `coverage_window` (measured per-subtype year
|
||||||
|
#' extents) and `truncated` (subtypes whose coverage falls short of the
|
||||||
|
#' requested years). `expenditure_concept`/`revenue_concept` are `NA` --
|
||||||
|
#' holdings are a stock, not a flow, so neither concept vocabulary applies.
|
||||||
#' @export
|
#' @export
|
||||||
cog_balances <- function(govid, years, category = NULL,
|
cog_balances <- function(govid, years, category = NULL,
|
||||||
per_capita = FALSE, adjust_to_year = NULL,
|
per_capita = FALSE, adjust_to_year = NULL,
|
||||||
|
|||||||
@@ -3,6 +3,12 @@ template:
|
|||||||
bootstrap: 5
|
bootstrap: 5
|
||||||
|
|
||||||
reference:
|
reference:
|
||||||
|
- title: Financial data
|
||||||
|
desc: Spending, revenue and balance-sheet holdings for one or more governments.
|
||||||
|
contents:
|
||||||
|
- cog_spending
|
||||||
|
- cog_revenue
|
||||||
|
- cog_balances
|
||||||
- title: Search & basket
|
- title: Search & basket
|
||||||
desc: Resolve place names into canonical govids.
|
desc: Resolve place names into canonical govids.
|
||||||
contents:
|
contents:
|
||||||
|
|||||||
@@ -53,6 +53,16 @@
|
|||||||
"items": { "type": "string" },
|
"items": { "type": "string" },
|
||||||
"description": "Ids of catalogued series breaks whose fin_code is the literal 'ALL' -- caveats about the corpus as a whole (dollar precision across 1976/1977, imputation exclusion from 2002, the dense -> sparse representation change at 2012, the government id scheme change at 2017) rather than about one item code. Selected on the break_year window alone, so they do not depend on which codes a result contains. Disjoint from series_break_refs by construction: an entry qualifies the whole result, not one series."
|
"description": "Ids of catalogued series breaks whose fin_code is the literal 'ALL' -- caveats about the corpus as a whole (dollar precision across 1976/1977, imputation exclusion from 2002, the dense -> sparse representation change at 2012, the government id scheme change at 2017) rather than about one item code. Selected on the break_year window alone, so they do not depend on which codes a result contains. Disjoint from series_break_refs by construction: an entry qualifies the whole result, not one series."
|
||||||
},
|
},
|
||||||
|
"balance_caveats": {
|
||||||
|
"type": ["object", "null"],
|
||||||
|
"description": "Present only on cog_balances() results (null/absent for cog_spending()/cog_revenue()). `not_gaap` is always TRUE and `not_gaap_note` explains that Census holdings are gross -- no liabilities are netted -- so they are NOT comparable to a GAAP fund balance. `coverage_window` maps each observed balance_subtype to its measured [min year, max year] in the mounted corpus (never hardcoded). `truncated` lists the subtypes whose coverage_window does not fully span the requested years.",
|
||||||
|
"properties": {
|
||||||
|
"not_gaap": { "type": "boolean" },
|
||||||
|
"not_gaap_note": { "type": "string" },
|
||||||
|
"coverage_window": { "type": "object" },
|
||||||
|
"truncated": { "type": "array", "items": { "type": "string" } }
|
||||||
|
}
|
||||||
|
},
|
||||||
"manifest": { "type": "object" },
|
"manifest": { "type": "object" },
|
||||||
"sql_query": { "type": "string" }
|
"sql_query": { "type": "string" }
|
||||||
}
|
}
|
||||||
|
|||||||
+13
-2
@@ -46,8 +46,19 @@ and raw space are identical for holdings. Reported in
|
|||||||
wide era to the modern one.}
|
wide era to the modern one.}
|
||||||
}
|
}
|
||||||
\value{
|
\value{
|
||||||
A `tbl_df` with a `provenance` attribute. Amounts are full US
|
Tibble with columns `year`, `canonical_govid`, `gov_name`,
|
||||||
dollars.
|
`balance_subtype`, `category`, `amt_nominal`, optional
|
||||||
|
`amt_per_capita_nominal` and `pop_source` (when `per_capita = TRUE`),
|
||||||
|
optional `amt_real` and `amt_per_capita_real` (when `adjust_to_year` is
|
||||||
|
set), `codes_included`, `aggregate_fallback`, `notes`. Amounts are full
|
||||||
|
US dollars.
|
||||||
|
|
||||||
|
Carries a `provenance` attribute matching
|
||||||
|
`inst/schemas/provenance-v1.json`, whose `balance_caveats` block reports
|
||||||
|
`not_gaap`, `not_gaap_note`, `coverage_window` (measured per-subtype year
|
||||||
|
extents) and `truncated` (subtypes whose coverage falls short of the
|
||||||
|
requested years). `expenditure_concept`/`revenue_concept` are `NA` --
|
||||||
|
holdings are a stock, not a flow, so neither concept vocabulary applies.
|
||||||
}
|
}
|
||||||
\description{
|
\description{
|
||||||
Returns Census cash-and-security holdings (`category_type = "balance"`):
|
Returns Census cash-and-security holdings (`category_type = "balance"`):
|
||||||
|
|||||||
@@ -359,6 +359,14 @@ test_that("a request past a family's coverage window is flagged", {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
test_that("the provenance schema documents balance_caveats", {
|
||||||
|
sch <- jsonlite::fromJSON(
|
||||||
|
system.file("schemas", "provenance-v1.json", package = "uscogdata"),
|
||||||
|
simplifyVector = FALSE
|
||||||
|
)
|
||||||
|
expect_true("balance_caveats" %in% names(sch$properties))
|
||||||
|
})
|
||||||
|
|
||||||
test_that("the caveat message fires once per session", {
|
test_that("the caveat message fires once per session", {
|
||||||
skip_if_no_corpus()
|
skip_if_no_corpus()
|
||||||
with_fixture_corpus({
|
with_fixture_corpus({
|
||||||
|
|||||||
Reference in New Issue
Block a user