Compare commits
6
Commits
9eaa759ccb
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ba57f9cb27
|
||
|
|
63b421aea2
|
||
|
|
d1165a507e
|
||
|
|
d83301bdbd
|
||
|
|
9adb921170
|
||
|
|
9f5cd988b2
|
@@ -44,6 +44,29 @@ jobs:
|
|||||||
|
|
||||||
- uses: r-lib/actions/setup-pandoc@v2
|
- uses: r-lib/actions/setup-pandoc@v2
|
||||||
|
|
||||||
|
- name: Drop the unused google-chrome apt source
|
||||||
|
if: runner.os == 'Linux'
|
||||||
|
run: |
|
||||||
|
set -x
|
||||||
|
grep -rl 'dl\.google\.com' /etc/apt/sources.list.d/ /etc/apt/sources.list 2>/dev/null || true
|
||||||
|
sudo sh -c "grep -rl 'dl\.google\.com' /etc/apt/sources.list.d/ /etc/apt/sources.list 2>/dev/null | xargs -r rm -f"
|
||||||
|
grep -rl 'dl\.google\.com' /etc/apt/sources.list.d/ /etc/apt/sources.list 2>/dev/null && exit 1 || true
|
||||||
|
# setup-r@v2 runs `sudo apt-get update` before installing R, and that
|
||||||
|
# command fails outright if ANY configured apt source is broken --
|
||||||
|
# even one we never use. The google-chrome source baked into GitHub's
|
||||||
|
# ubuntu-latest image intermittently serves a stale Packages.gz that
|
||||||
|
# doesn't match its own Release file's hash (a Google CDN sync race,
|
||||||
|
# not anything about R or this repo), which took down every ubuntu
|
||||||
|
# leg of this matrix on 2026-09-09. `rm -f` on a guessed filename
|
||||||
|
# (google-chrome.list) reported success but removed nothing -- the
|
||||||
|
# file it actually is on this image apparently doesn't match that
|
||||||
|
# name, since the source kept showing up in setup-r's `apt-get
|
||||||
|
# update` afterward. Find-by-content instead of guessing the
|
||||||
|
# filename, and fail loudly here (before setup-r even runs) if a
|
||||||
|
# matching source is still present, so a future runner-image change
|
||||||
|
# surfaces as a clear failure in this step instead of a confusing
|
||||||
|
# one in setup-r.
|
||||||
|
|
||||||
- uses: r-lib/actions/setup-r@v2
|
- uses: r-lib/actions/setup-r@v2
|
||||||
with:
|
with:
|
||||||
r-version: ${{ matrix.config.r }}
|
r-version: ${{ matrix.config.r }}
|
||||||
|
|||||||
@@ -5,11 +5,12 @@
|
|||||||
R package providing a curated reader API for the Civilytics US Census of
|
R package providing a curated reader API for the Civilytics US Census of
|
||||||
Governments finance corpus. Reads Hive-partitioned parquet + `manifest.json`
|
Governments finance corpus. Reads Hive-partitioned parquet + `manifest.json`
|
||||||
published by `cog_pipeline` via DuckDB (local path or remote URL). This is a
|
published by `cog_pipeline` via DuckDB (local path or remote URL). This is a
|
||||||
standalone Gitea repo, sibling to `cog_explorer/cog_pipeline/`.
|
standalone Gitea repo, sibling to the pipeline repo,
|
||||||
|
`Civilytics/census_of_governments_finance_pipeline`, cloned beside this one.
|
||||||
|
|
||||||
**Gitea remote:** `gitea.civilytics.org/Civilytics/uscogdata`
|
**Gitea remote:** `gitea.civilytics.org/Civilytics/uscogdata`
|
||||||
**Full implementation plan:** `../cog_pipeline/docs/plan_phase_n_tasks.md` (Tasks 2.1–2.8 + Phase 3)
|
**Full implementation plan (archived):** `../census_of_governments_finance_pipeline/docs/archive/plan_phase_n_tasks.md` (Tasks 2.1–2.8 + Phase 3)
|
||||||
**Reader contract spec:** `../cog_pipeline/docs/reader-specification.md`
|
**Reader contract spec:** `../census_of_governments_finance_pipeline/docs/reader-specification.md`
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
|
|||||||
+5
-1
@@ -390,7 +390,7 @@
|
|||||||
#' `R/basis.R`). This blocks a recipe surfaced through a mis-scoped
|
#' `R/basis.R`). This blocks a recipe surfaced through a mis-scoped
|
||||||
#' category from ever reaching the M/L search, e.g. `cog_spending()`'s
|
#' category from ever reaching the M/L search, e.g. `cog_spending()`'s
|
||||||
#' flow_prefixes are `c("E","F","G")`, which `ig_federal_b47_wide`'s own
|
#' flow_prefixes are `c("E","F","G")`, which `ig_federal_b47_wide`'s own
|
||||||
#' "B" is not part of.
|
#' `"B"` is not part of.
|
||||||
#' 2. `own_prefix %in% c("E","F","G")`: M/L only ever pairs with the
|
#' 2. `own_prefix %in% c("E","F","G")`: M/L only ever pairs with the
|
||||||
#' DIRECT-expenditure family, never with revenue (`cog_revenue()`'s
|
#' DIRECT-expenditure family, never with revenue (`cog_revenue()`'s
|
||||||
#' flow_prefixes already fold B/C/D in as ordinary revenue -- there is
|
#' flow_prefixes already fold B/C/D in as ordinary revenue -- there is
|
||||||
@@ -398,6 +398,10 @@
|
|||||||
#' adds one for spending) and never with ANOTHER M/L recipe (without
|
#' adds one for spending) and never with ANOTHER M/L recipe (without
|
||||||
#' this check, `ige_local_m47_wide` would wrongly match sibling
|
#' this check, `ige_local_m47_wide` would wrongly match sibling
|
||||||
#' `ige_state_l47_wide` on their shared {"47","94"} suffix set).
|
#' `ige_state_l47_wide` on their shared {"47","94"} suffix set).
|
||||||
|
#' Condition 1 alone does not catch this: under `cog_revenue()`,
|
||||||
|
#' `ig_federal_b47_wide`'s own `"B"` IS inside revenue's own
|
||||||
|
#' `flow_prefixes`, so only this second, family-specific check blocks
|
||||||
|
#' the search.
|
||||||
#' @noRd
|
#' @noRd
|
||||||
.attach_ig_counterparts <- function(con, suggestions, flow_prefixes) {
|
.attach_ig_counterparts <- function(con, suggestions, flow_prefixes) {
|
||||||
if (length(suggestions) == 0L) return(suggestions)
|
if (length(suggestions) == 0L) return(suggestions)
|
||||||
|
|||||||
@@ -12,3 +12,40 @@ file carries the reasoning and the pointers.
|
|||||||
- **Refs** — commits, issues, decision records.
|
- **Refs** — commits, issues, decision records.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 2026-09-30 · repo · moved out of Nextcloud; code syncs only through Gitea
|
||||||
|
|
||||||
|
**Why:** Nextcloud was syncing this repo's `.git`, which risks conflict copies inside
|
||||||
|
it and stalls the client's cold scan across dozens of repos. Nothing untracked is
|
||||||
|
data, so nothing is linked (`--no-data`); the move changed no test result (1,101
|
||||||
|
pass, 2 skip before and after).
|
||||||
|
**Obligates:** #73, #74
|
||||||
|
**Refs:** 4bedf85 (ci/apt-https pushed to clear the move's preflight), 63b421a
|
||||||
|
|
||||||
|
## 2026-09-09 · ci · Linux R-CMD-check legs unblocked (recorded 2026-09-30)
|
||||||
|
|
||||||
|
**Why:** GitHub's ubuntu-latest image carries a google-chrome apt source that
|
||||||
|
intermittently fails its own hash check, and setup-r's `apt-get update` died on it for
|
||||||
|
every R version; the source is now found by URL and removed, and the step fails
|
||||||
|
loudly if it survives.
|
||||||
|
**Obligates:** (none)
|
||||||
|
**Refs:** d83301b, d1165a5
|
||||||
|
|
||||||
|
## 2026-09-09 · model · #36 coverage counter finished, two bugs caught before merge
|
||||||
|
|
||||||
|
**Why:** the drafted n_units_collected fix (uncommitted) derived its candidate
|
||||||
|
cohort from category-filtered results instead of the caller's full expected
|
||||||
|
cohort, and hardcoded a view name absent below schema_version 5 -- both
|
||||||
|
silent on the fixture, both would have shipped without an independent review
|
||||||
|
pass before merge.
|
||||||
|
**Obligates:** #72
|
||||||
|
**Refs:** #36, b41d5ee, PR#71
|
||||||
|
|
||||||
|
## 2026-09-09 · model · #33/#34 split into independently-tested commits
|
||||||
|
|
||||||
|
**Why:** #33's own branch had its tests sitting uncommitted, and quietly bundled
|
||||||
|
a behavior change (#34) into what its commit message called a pure refactor;
|
||||||
|
splitting them let each pass CI with its own tests instead of merging on a
|
||||||
|
false "tests pass" claim.
|
||||||
|
**Obligates:** (none)
|
||||||
|
**Refs:** #33, #34, 0c7c7eb, 392643b, d0d724c, 9f5cd98, PR#69, PR#70
|
||||||
|
|||||||
+16
-32
@@ -7,46 +7,30 @@
|
|||||||
|
|
||||||
## Where this stands
|
## Where this stands
|
||||||
|
|
||||||
uscogdata is at 0.4.0 and its public surface is settled: the query verbs, the cohort
|
uscogdata's code now lives only in Gitea; the old Nextcloud folder keeps just build
|
||||||
predicates added in this release, and the provenance contract every verb returns.
|
output and working files. The move changed no test result: 1,101 pass and the two
|
||||||
|
opt-in live-corpus tests skip, before and after. It turned up one defect, #73: the
|
||||||
The six open issues split cleanly. Two are API work carried out of the #9 review pass
|
fixture's reference docs have never been in git, because a `.gitignore` rule matches
|
||||||
and deliberately deferred there rather than fixed in that branch. Three concern the
|
too broadly. Next in line are #73 and #72 (a vignette for the coverage counters), with
|
||||||
corpus layer, and the largest of them, partition-level caching, was named the single
|
#52 (data-correction intake) and #74 (the stale state section in `CLAUDE.md`) waiting
|
||||||
highest-leverage change on the remote path before being deferred. One, the
|
on a decision.
|
||||||
data-correction intake (#52), is a decision rather than a task: it was parked during
|
|
||||||
the 0.3.0 design, and the API announcement waits on it, because without it the corpus
|
|
||||||
cannot make the "traceable and correctable" claim that most distinguishes it from
|
|
||||||
Census's own files.
|
|
||||||
|
|
||||||
Nothing here is blocked on anything else, so the ordering is a judgement about value
|
|
||||||
rather than a dependency graph.
|
|
||||||
|
|
||||||
Compass's own files moved out of `docs/` this session. They were sitting inside
|
|
||||||
pkgdown's output directory, and `pkgdown::clean_site()` deletes every top-level entry
|
|
||||||
there except `CNAME` and `dev` — asked directly, it listed `docs/pm` and
|
|
||||||
`docs/decisions` among the 28 it would remove, with the guard that would have stopped
|
|
||||||
it satisfied by `docs/pkgdown.yml`. They are in `pm/` now. Nothing was lost: the
|
|
||||||
journal had no entries and there were no decision records yet, which made this the
|
|
||||||
cheapest moment to move. The `.gitignore` workaround that re-included two children of
|
|
||||||
an excluded `docs/` is gone with it.
|
|
||||||
|
|
||||||
## Ready to work on next
|
## Ready to work on next
|
||||||
|
|
||||||
- **#34** cog_revenue() offers expenditure recipes as suggestions: scope the candidate query by category_type · `ws/api` — nothing is blocking it; something is currently wrong
|
- **#73** The fixture corpus's docs/ never reaches git: .gitignore's docs/ rule is unanchored · `ws/corpus` — nothing is blocking it; something is currently wrong
|
||||||
- **#36** n_units_reporting is category-conditional and cannot be read as a response rate · `ws/corpus` — nothing is blocking it; owed work from an earlier change
|
- **#72** docs: add a vignette explaining provenance$coverage counters · `ws/docs` — nothing is blocking it; owed work from an earlier change
|
||||||
- **#2** Extend population data to be households as an alternate spending denominator · `ws/corpus` — nothing is blocking it
|
- **#2** Extend population data to be households as an alternate spending denominator · `ws/corpus` — nothing is blocking it
|
||||||
- **#33** Decompose .build_suggestions() (106 lines) into named helpers · `ws/api` — nothing is blocking it
|
|
||||||
- **#52** Release 11/11: design the data-correction intake (deferred; gates the API announcement) · `ws/corpus` — nothing is blocking it
|
|
||||||
- **#64** Partition-level caching: R/cache.R is still a stub, and the remote path pays for it every session · `ws/corpus` — nothing is blocking it
|
- **#64** Partition-level caching: R/cache.R is still a stub, and the remote path pays for it every session · `ws/corpus` — nothing is blocking it
|
||||||
|
- **#74** CLAUDE.md's Current State section is frozen at 2026-08-03 · `ws/docs` — waiting on a person, not on other work
|
||||||
|
- **#52** Release 11/11: design the data-correction intake (deferred; gates the API announcement) · `ws/corpus` — waiting on a person, not on other work
|
||||||
|
|
||||||
## Workstreams
|
## Workstreams
|
||||||
|
|
||||||
| Stream | Commits since | Open | Debt | Owes docs |
|
| Stream | Commits since | Open | Debt | Owes docs |
|
||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| Query verbs and results | 77 | 2 | 0 | no |
|
| Query verbs and results | 0 | 0 | 0 | no |
|
||||||
| Corpus, mirror, provenance | 39 | 4 | 1 | no |
|
| Corpus, mirror, provenance | 0 | 4 | 0 | no |
|
||||||
| Vignettes and guides | 34 | 0 | 0 | **yes** |
|
| Vignettes and guides | 0 | 2 | 2 | no |
|
||||||
|
|
||||||
## CI
|
## CI
|
||||||
|
|
||||||
@@ -58,8 +42,8 @@ an excluded `docs/` is gone with it.
|
|||||||
|
|
||||||
_Nothing blocks anything else, so there is no graph to draw._
|
_Nothing blocks anything else, so there is no graph to draw._
|
||||||
|
|
||||||
- Marker: `none` (no journal entry yet)
|
- Marker: `9adb9211` (2026-09-09)
|
||||||
- Commits since: 165
|
- Commits since: 3
|
||||||
- Open issues: 6
|
- Open issues: 6
|
||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|||||||
Reference in New Issue
Block a user