Compare commits

...
6 Commits
Author SHA1 Message Date
jaredandClaude Opus 5.5 ba57f9cb27 docs: record session
Mirror to GitHub / mirror (push) Successful in 8s
R-CMD-check / check (push) Successful in 3m21s
Journal entries for the move out of Nextcloud and the 2026-09-09 CI fix,
and the regenerated status board.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:15:56 -04:00
jaredandClaude Opus 5.5 63b421aea2 docs: point CLAUDE.md at the pipeline clone's paths
The repo moved out of Nextcloud today, and its sibling pipeline clone is
now census_of_governments_finance_pipeline, not cog_explorer/cog_pipeline.
The phase N task plan had already moved to docs/archive/ on 2026-07-23
(pipeline acfacf2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:15:00 -04:00
jaredandClaude Sonnet 5 d1165a507e fix(ci): find the google-chrome apt source by content, not filename
Mirror to GitHub / mirror (push) Successful in 7s
R-CMD-check / check (push) Successful in 4m35s
The prior commit's `sudo rm -f /etc/apt/sources.list.d/google-chrome.list`
guessed a filename that doesn't match what's actually on GitHub's
ubuntu-latest image: the step reported success (rm -f swallows
"no such file"), but setup-r@v2's subsequent apt-get update still hit
the same dl.google.com Hash Sum mismatch. Locate the offending source
by grepping for its URL instead, and fail this step explicitly if a
match survives removal, so a future runner-image change is loud here
rather than confusing inside setup-r@v2.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 13:53:40 -04:00
jaredandClaude Sonnet 5 d83301bdbd fix(ci): drop unused google-chrome apt source before setup-r
Mirror to GitHub / mirror (push) Successful in 6s
R-CMD-check / check (push) Successful in 3m30s
r-lib/actions/setup-r@v2 runs `sudo apt-get update` before installing
R, which fails outright if any configured apt source is broken - even
one we never touch. The google-chrome source baked into GitHub's
ubuntu-latest image intermittently serves a stale Packages.gz that
doesn't match its own Release file's hash (a Google CDN sync race),
which took down every Linux leg of the R-CMD-check matrix on
2026-09-09 regardless of R version. Removing the source we don't need
makes this failure class impossible here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 13:47:42 -04:00
jaredandClaude Sonnet 5 9adb921170 docs: record session (#36)
Mirror to GitHub / mirror (push) Successful in 7s
R-CMD-check / check (push) Successful in 3m37s
Journal entries for the #33/#34 split and the #36 coverage counter
(two bugs caught before merge in the latter), plus the vignette
obligation filed as #72. Status board regenerated and republished.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 13:38:25 -04:00
jaredandClaude Sonnet 5 9f5cd988b2 docs(suggestions): restore dropped M/L-counterpart rationale (roborev job 224)
Mirror to GitHub / mirror (push) Successful in 7s
R-CMD-check / check (push) Successful in 3m46s
The #33 decomposition (0c7c7eb) silently dropped four roxygen lines
explaining why .attach_ig_counterparts()'s second flow-family check is
needed: condition 1 alone doesn't block ig_federal_b47_wide under
cog_revenue(), since its own "B" IS inside revenue's own flow_prefixes.
Restored them, plus the backticks around "B" that were also dropped as
an unstated formatting change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 13:35:18 -04:00
5 changed files with 85 additions and 36 deletions
+23
View File
@@ -44,6 +44,29 @@ jobs:
- uses: r-lib/actions/setup-pandoc@v2
- name: Drop the unused google-chrome apt source
if: runner.os == 'Linux'
run: |
set -x
grep -rl 'dl\.google\.com' /etc/apt/sources.list.d/ /etc/apt/sources.list 2>/dev/null || true
sudo sh -c "grep -rl 'dl\.google\.com' /etc/apt/sources.list.d/ /etc/apt/sources.list 2>/dev/null | xargs -r rm -f"
grep -rl 'dl\.google\.com' /etc/apt/sources.list.d/ /etc/apt/sources.list 2>/dev/null && exit 1 || true
# setup-r@v2 runs `sudo apt-get update` before installing R, and that
# command fails outright if ANY configured apt source is broken --
# even one we never use. The google-chrome source baked into GitHub's
# ubuntu-latest image intermittently serves a stale Packages.gz that
# doesn't match its own Release file's hash (a Google CDN sync race,
# not anything about R or this repo), which took down every ubuntu
# leg of this matrix on 2026-09-09. `rm -f` on a guessed filename
# (google-chrome.list) reported success but removed nothing -- the
# file it actually is on this image apparently doesn't match that
# name, since the source kept showing up in setup-r's `apt-get
# update` afterward. Find-by-content instead of guessing the
# filename, and fail loudly here (before setup-r even runs) if a
# matching source is still present, so a future runner-image change
# surfaces as a clear failure in this step instead of a confusing
# one in setup-r.
- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.config.r }}
+4 -3
View File
@@ -5,11 +5,12 @@
R package providing a curated reader API for the Civilytics US Census of
Governments finance corpus. Reads Hive-partitioned parquet + `manifest.json`
published by `cog_pipeline` via DuckDB (local path or remote URL). This is a
standalone Gitea repo, sibling to `cog_explorer/cog_pipeline/`.
standalone Gitea repo, sibling to the pipeline repo,
`Civilytics/census_of_governments_finance_pipeline`, cloned beside this one.
**Gitea remote:** `gitea.civilytics.org/Civilytics/uscogdata`
**Full implementation plan:** `../cog_pipeline/docs/plan_phase_n_tasks.md` (Tasks 2.1–2.8 + Phase 3)
**Reader contract spec:** `../cog_pipeline/docs/reader-specification.md`
**Full implementation plan (archived):** `../census_of_governments_finance_pipeline/docs/archive/plan_phase_n_tasks.md` (Tasks 2.1–2.8 + Phase 3)
**Reader contract spec:** `../census_of_governments_finance_pipeline/docs/reader-specification.md`
## Architecture
+5 -1
View File
@@ -390,7 +390,7 @@
#' `R/basis.R`). This blocks a recipe surfaced through a mis-scoped
#' category from ever reaching the M/L search, e.g. `cog_spending()`'s
#' flow_prefixes are `c("E","F","G")`, which `ig_federal_b47_wide`'s own
#' "B" is not part of.
#' `"B"` is not part of.
#' 2. `own_prefix %in% c("E","F","G")`: M/L only ever pairs with the
#' DIRECT-expenditure family, never with revenue (`cog_revenue()`'s
#' flow_prefixes already fold B/C/D in as ordinary revenue -- there is
@@ -398,6 +398,10 @@
#' adds one for spending) and never with ANOTHER M/L recipe (without
#' this check, `ige_local_m47_wide` would wrongly match sibling
#' `ige_state_l47_wide` on their shared {"47","94"} suffix set).
#' Condition 1 alone does not catch this: under `cog_revenue()`,
#' `ig_federal_b47_wide`'s own `"B"` IS inside revenue's own
#' `flow_prefixes`, so only this second, family-specific check blocks
#' the search.
#' @noRd
.attach_ig_counterparts <- function(con, suggestions, flow_prefixes) {
if (length(suggestions) == 0L) return(suggestions)
+37
View File
@@ -12,3 +12,40 @@ file carries the reasoning and the pointers.
- **Refs** — commits, issues, decision records.
---
## 2026-09-30 · repo · moved out of Nextcloud; code syncs only through Gitea
**Why:** Nextcloud was syncing this repo's `.git`, which risks conflict copies inside
it and stalls the client's cold scan across dozens of repos. Nothing untracked is
data, so nothing is linked (`--no-data`); the move changed no test result (1,101
pass, 2 skip before and after).
**Obligates:** #73, #74
**Refs:** 4bedf85 (ci/apt-https pushed to clear the move's preflight), 63b421a
## 2026-09-09 · ci · Linux R-CMD-check legs unblocked (recorded 2026-09-30)
**Why:** GitHub's ubuntu-latest image carries a google-chrome apt source that
intermittently fails its own hash check, and setup-r's `apt-get update` died on it for
every R version; the source is now found by URL and removed, and the step fails
loudly if it survives.
**Obligates:** (none)
**Refs:** d83301b, d1165a5
## 2026-09-09 · model · #36 coverage counter finished, two bugs caught before merge
**Why:** the drafted n_units_collected fix (uncommitted) derived its candidate
cohort from category-filtered results instead of the caller's full expected
cohort, and hardcoded a view name absent below schema_version 5 -- both
silent on the fixture, both would have shipped without an independent review
pass before merge.
**Obligates:** #72
**Refs:** #36, b41d5ee, PR#71
## 2026-09-09 · model · #33/#34 split into independently-tested commits
**Why:** #33's own branch had its tests sitting uncommitted, and quietly bundled
a behavior change (#34) into what its commit message called a pure refactor;
splitting them let each pass CI with its own tests instead of merging on a
false "tests pass" claim.
**Obligates:** (none)
**Refs:** #33, #34, 0c7c7eb, 392643b, d0d724c, 9f5cd98, PR#69, PR#70
+16 -32
View File
@@ -7,46 +7,30 @@
## Where this stands
uscogdata is at 0.4.0 and its public surface is settled: the query verbs, the cohort
predicates added in this release, and the provenance contract every verb returns.
The six open issues split cleanly. Two are API work carried out of the #9 review pass
and deliberately deferred there rather than fixed in that branch. Three concern the
corpus layer, and the largest of them, partition-level caching, was named the single
highest-leverage change on the remote path before being deferred. One, the
data-correction intake (#52), is a decision rather than a task: it was parked during
the 0.3.0 design, and the API announcement waits on it, because without it the corpus
cannot make the "traceable and correctable" claim that most distinguishes it from
Census's own files.
Nothing here is blocked on anything else, so the ordering is a judgement about value
rather than a dependency graph.
Compass's own files moved out of `docs/` this session. They were sitting inside
pkgdown's output directory, and `pkgdown::clean_site()` deletes every top-level entry
there except `CNAME` and `dev` — asked directly, it listed `docs/pm` and
`docs/decisions` among the 28 it would remove, with the guard that would have stopped
it satisfied by `docs/pkgdown.yml`. They are in `pm/` now. Nothing was lost: the
journal had no entries and there were no decision records yet, which made this the
cheapest moment to move. The `.gitignore` workaround that re-included two children of
an excluded `docs/` is gone with it.
uscogdata's code now lives only in Gitea; the old Nextcloud folder keeps just build
output and working files. The move changed no test result: 1,101 pass and the two
opt-in live-corpus tests skip, before and after. It turned up one defect, #73: the
fixture's reference docs have never been in git, because a `.gitignore` rule matches
too broadly. Next in line are #73 and #72 (a vignette for the coverage counters), with
#52 (data-correction intake) and #74 (the stale state section in `CLAUDE.md`) waiting
on a decision.
## Ready to work on next
- **#34** cog_revenue() offers expenditure recipes as suggestions: scope the candidate query by category_type · `ws/api` — nothing is blocking it; something is currently wrong
- **#36** n_units_reporting is category-conditional and cannot be read as a response rate · `ws/corpus` — nothing is blocking it; owed work from an earlier change
- **#73** The fixture corpus's docs/ never reaches git: .gitignore's docs/ rule is unanchored · `ws/corpus` — nothing is blocking it; something is currently wrong
- **#72** docs: add a vignette explaining provenance$coverage counters · `ws/docs` — nothing is blocking it; owed work from an earlier change
- **#2** Extend population data to be households as an alternate spending denominator · `ws/corpus` — nothing is blocking it
- **#33** Decompose .build_suggestions() (106 lines) into named helpers · `ws/api` — nothing is blocking it
- **#52** Release 11/11: design the data-correction intake (deferred; gates the API announcement) · `ws/corpus` — nothing is blocking it
- **#64** Partition-level caching: R/cache.R is still a stub, and the remote path pays for it every session · `ws/corpus` — nothing is blocking it
- **#74** CLAUDE.md's Current State section is frozen at 2026-08-03 · `ws/docs` — waiting on a person, not on other work
- **#52** Release 11/11: design the data-correction intake (deferred; gates the API announcement) · `ws/corpus` — waiting on a person, not on other work
## Workstreams
| Stream | Commits since | Open | Debt | Owes docs |
|---|---|---|---|---|
| Query verbs and results | 77 | 2 | 0 | no |
| Corpus, mirror, provenance | 39 | 4 | 1 | no |
| Vignettes and guides | 34 | 0 | 0 | **yes** |
| Query verbs and results | 0 | 0 | 0 | no |
| Corpus, mirror, provenance | 0 | 4 | 0 | no |
| Vignettes and guides | 0 | 2 | 2 | no |
## CI
@@ -58,8 +42,8 @@ an excluded `docs/` is gone with it.
_Nothing blocks anything else, so there is no graph to draw._
- Marker: `none` (no journal entry yet)
- Commits since: 165
- Marker: `9adb9211` (2026-09-09)
- Commits since: 3
- Open issues: 6
</details>