From b3e544ebd86321c19f3596036e721971d84e5aed Mon Sep 17 00:00:00 2001 From: Jared Knowles Date: Mon, 27 Apr 2026 13:25:41 -0400 Subject: [PATCH] docs: add CLAUDE.md agent instructions with current state + v0.1 roadmap --- CLAUDE.md | 104 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..d739dd8 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,104 @@ +# CLAUDE.md — Agent Instructions for uscogdata + +## Project Overview + +R package providing a curated reader API for the Civilytics US Census of +Governments finance corpus. Reads Hive-partitioned parquet + `manifest.json` +published by `cog_pipeline` via DuckDB (local path or remote URL). This is a +standalone Gitea repo, sibling to `cog_explorer/cog_pipeline/`. + +**Gitea remote:** `gitea.civilytics.org/Civilytics/uscogdata` +**Full implementation plan:** `../cog_pipeline/docs/plan_phase_n_tasks.md` (Tasks 2.1–2.8 + Phase 3) +**Reader contract spec:** `../cog_pipeline/docs/reader-specification.md` + +## Architecture + +### Data flow + +``` +USCOGDATA_URL (local path or https://) + → cog_open() fetches manifest.json, registers DuckDB views from inst/sql/ + → verbs query views via DBI + → results carry provenance attr +``` + +### Key files + +- `R/config.R` — `.cfg()`, `.resolve_url()`, `.uscogdata_env` (mutable state) +- `R/session.R` — `cog_open()`, `cog_close()`, `.ensure_session()`, `.coerce_govid_input()` +- `R/manifest.R` — `.fetch_or_cache_manifest()`, `.is_local_path()` (local paths bypass HTTP/cache) +- `R/views.R` — `.register_views()` (substitutes `{url}` into SQL files at `inst/sql/`) +- `inst/sql/` — 7 SQL view definitions: `long`, `spending_long`, `revenue_long`, `canonical_fips_xwalk`, `summary_categories`, `spending_annotated`, `revenue_annotated` +- `R/spending.R` / `R/revenue.R` — `cog_spending()` / `cog_revenue()` via shared `.verb_spendrev()` +- `R/rollup.R` — `cog_geographic_rollup()` (accepts named list of govids by layer) +- `R/peers.R` — `cog_find_peers()` + `cog_peer_compare()` +- `R/search.R` — `cog_gov_search()` (name pattern, state, type filters) +- `R/mirror.R` — `cog_mirror()` (local corpus copy via httr2 WebDAV) +- `R/explain.R` — `cog_explain()` (prints/returns provenance attr) +- `R/categories.R` — `cog_categories()` (queries `summary_categories` view) +- `R/adjust.R` — `.inflate()` helper using bundled CPIAUCSL in `sysdata.rda` +- `R/provenance.R` — `.build_provenance()` attached as attr to all verb results + +### Corpus URL resolution + +`USCOGDATA_URL` env var → `uscogdata.url` option → default placeholder in `config.R`. +Any value without `://` is treated as a local path by `.is_local_path()` and reads +`manifest.json` directly from disk (no HTTP, no TTL cache). + +## Current State (2026-04-27) + +**Version:** 0.1.0 (pre-release) +**Branch:** `main`, commit `d65e9fe` +**Tests:** 181 PASS / 0 FAIL / 0 SKIP +**CI:** Gitea Actions green (`.gitea/workflows/ci.yml`) + +### Completed (Tasks 2.1–2.7) + +All 8 exported verbs implemented and tested: +`cog_spending`, `cog_revenue`, `cog_explain`, `cog_geographic_rollup`, +`cog_find_peers`, `cog_peer_compare`, `cog_gov_search`, `cog_mirror`, +plus `cog_categories`. + +Bundled fixture corpus at `inst/extdata/fixture_corpus/` (3.6 MB, years +2019+2020, all 50 states). Tests run fully offline — no credentials needed. + +### Remaining to v0.1 release + +1. **Task 2.8 — Docs:** roxygen `@param`/`@return`/`@examples` on all exports; + full `README.md`; `_pkgdown.yml`; `devtools::document()` + `pkgdown::build_site()`. + Vignettes can be stubbed for v0.1. + +2. **Phase 3 — cog_explorer bridge:** create + `cog_explorer/examples/hello_world_uscogdata.Rmd` (installs from Gitea, runs + `cog_spending()` against live Nextcloud corpus, renders a chart). Gate N14. + +3. **Release gates:** run test suite against live corpus + (`USCOGDATA_URL= devtools::test()`), add + `^inst/extdata/fixture_corpus$` to `.Rbuildignore`, tag `v0.1.0`, cut + Gitea release. + +## Testing + +```r +devtools::test() # uses bundled fixture, fully offline +testthat::test_local() # same, used by CI +``` + +`tests/testthat/setup.R` sets `USCOGDATA_URL` to the bundled fixture automatically. +`helper-fixture.R` provides `skip_if_no_corpus()`, `fixture_corpus_path()`, +`with_fixture_corpus()`. + +To run against the live corpus: +```r +Sys.setenv(USCOGDATA_URL = "") +devtools::test() +``` + +## Coding Conventions + +- All verbs call `.ensure_session()` first, then query via `DBI::dbGetQuery()` +- Return value is always a `tbl_df` with a `provenance` attribute +- govid inputs always go through `.coerce_govid_input()` (accepts character or data frame) +- SQL lives in `inst/sql/` — never inline SQL strings in R files +- No arrow dependency — DuckDB reads parquet natively +- `withr` is a Suggests-only dep; only used in tests