feat(search): basket mode for cog_gov_search() #1

2026-04-28T14:47:28-04:00

jared commented

2026-04-28 14:47:28 -04:00

Summary

Adds basket mode to cog_gov_search() so users can resolve multiple place names to canonical govids in one call and pipe straight into cog_spending() / cog_revenue() / cog_geographic_rollup() / cog_peer_compare().

basket <- cog_gov_search(
  name  = c("BROWARD COUNTY", "SAN DIEGO CITY", "AUSTIN CITY"),
  state = c("FL",             "CA",             "TX")
)
basket |> cog_spending(years = 2019:2023, category = "Police")

Design

Mode dispatch on length(name). Length-1 keeps the existing regex utility behavior unchanged. Length->1 enters basket mode.
Resolution algorithm (per input row): exact case-insensitive equality on gov_name -> case-insensitive substring fallback (regex meta-chars escaped) -> disambiguation. Within-type ambiguity picks largest population_acs; cross-type ambiguity skips the row (no halt).
Soft-fail contract. Misses, ambiguous matches, and per-row excluded gov_types (4/5) do NOT abort. They're recorded in a sidecar tibble attached as attr(., \"resolution\"); the function emits a single cli::cli_inform summary when any row was unresolved.
Sidecar accessors. Two new exports: cog_basket_resolution(x, expand_candidates = FALSE) returns the audit tibble (drops the candidates list-col by default for readable printing), and cog_basket_unresolved(x) filters to ambiguous + no_match rows for iterative refinement.
Pipe-clean shape. Basket result is the same column shape as the existing utility-mode return (rows from canonical_fips_xwalk), in input order, with the sidecar carried as an attribute. .coerce_govid_input() already accepts any data frame with a canonical_govid column, so downstream verbs work without modification.

Breaking change

The first formal of cog_gov_search() was renamed pattern -> name. Audited 2026-04-28: every call site in this package and cog_explorer/ passes the first argument positionally, so the rename is non-breaking in practice. No lifecycle::deprecate_warn() alias added (pre-release v0.1.0, no external consumers).

Test plan

283 PASS / 0 FAIL / 0 SKIP via devtools::test() against the bundled fixture corpus.
R CMD check: 0 errors / 0 warnings (3 pre-existing notes about .gitea/, CLAUDE.md, timestamp).
Smoke test: basket of Broward FL + San Diego County CA pipes into cog_spending(years = 2019:2020, category = \"Police\") cleanly.
All four resolution outcomes covered (resolved, largest_pop, ambiguous, no_match) at both internal-helper and public-surface level.
Edge cases: empty/whitespace name, length-mismatched state/type, all-no-match basket, per-row excluded type, malformed regex name (e.g. unbalanced parens).

Files

R/search.R (381 lines): public cog_gov_search() mode dispatch, .validate_basket_args(), .resolve_basket_row(), .disambiguate(), .resolve_basket(), .escape_regex().
R/basket.R (138 lines, new): .build_sidecar(), .type_to_label(), .basket_summary_message(), plus exports cog_basket_resolution() and cog_basket_unresolved().
tests/testthat/test-search.R: +35 test cases covering basket mode.
tests/testthat/test-basket.R (new): 5 sidecar-accessor tests.
tests/testthat/test-spending.R: +1 pipe-through smoke test.
NEWS.md (new), _pkgdown.yml (extended reference index).

Deferred follow-ups

Make all-NA population_acs tiebreaker deterministic (add canonical_govid as secondary sort key). Rare in practice with the current corpus.
Extract .state_abbrev_to_fips to R/lookups.R for cohesion (~15 line reduction in search.R).

Spec & plan

Brainstormed and planned through superpowers: skills. Spec at docs/superpowers/specs/2026-04-28-cog-gov-search-basket-mode-design.md, plan at docs/superpowers/plans/2026-04-28-cog-gov-search-basket-mode.md (both gitignored). Implementation followed subagent-driven development: 7 implementer dispatches, 14 reviews (7 spec compliance + 7 code quality), 1 cross-task whole-branch review that caught two soft-fail-contract violations (F1: per-row excluded type aborting; F2: malformed regex name unhandled), both fixed in the final commit.

## Summary Adds **basket mode** to `cog_gov_search()` so users can resolve multiple place names to canonical govids in one call and pipe straight into `cog_spending()` / `cog_revenue()` / `cog_geographic_rollup()` / `cog_peer_compare()`. ```r basket <- cog_gov_search( name = c("BROWARD COUNTY", "SAN DIEGO CITY", "AUSTIN CITY"), state = c("FL", "CA", "TX") ) basket |> cog_spending(years = 2019:2023, category = "Police") ``` ## Design - **Mode dispatch on `length(name)`.** Length-1 keeps the existing regex utility behavior unchanged. Length->1 enters basket mode. - **Resolution algorithm** (per input row): exact case-insensitive equality on `gov_name` -> case-insensitive substring fallback (regex meta-chars escaped) -> disambiguation. Within-type ambiguity picks largest `population_acs`; cross-type ambiguity skips the row (no halt). - **Soft-fail contract.** Misses, ambiguous matches, and per-row excluded gov_types (4/5) do NOT abort. They're recorded in a sidecar tibble attached as ``attr(., \"resolution\")``; the function emits a single ``cli::cli_inform`` summary when any row was unresolved. - **Sidecar accessors.** Two new exports: ``cog_basket_resolution(x, expand_candidates = FALSE)`` returns the audit tibble (drops the candidates list-col by default for readable printing), and ``cog_basket_unresolved(x)`` filters to ambiguous + no_match rows for iterative refinement. - **Pipe-clean shape.** Basket result is the same column shape as the existing utility-mode return (rows from ``canonical_fips_xwalk``), in input order, with the sidecar carried as an attribute. ``.coerce_govid_input()`` already accepts any data frame with a ``canonical_govid`` column, so downstream verbs work without modification. ## Breaking change The first formal of ``cog_gov_search()`` was renamed ``pattern`` -> ``name``. Audited 2026-04-28: every call site in this package and ``cog_explorer/`` passes the first argument positionally, so the rename is non-breaking in practice. No ``lifecycle::deprecate_warn()`` alias added (pre-release v0.1.0, no external consumers). ## Test plan - [x] 283 PASS / 0 FAIL / 0 SKIP via ``devtools::test()`` against the bundled fixture corpus. - [x] R CMD check: 0 errors / 0 warnings (3 pre-existing notes about ``.gitea/``, ``CLAUDE.md``, timestamp). - [x] Smoke test: basket of Broward FL + San Diego County CA pipes into ``cog_spending(years = 2019:2020, category = \"Police\")`` cleanly. - [x] All four resolution outcomes covered (resolved, largest_pop, ambiguous, no_match) at both internal-helper and public-surface level. - [x] Edge cases: empty/whitespace name, length-mismatched state/type, all-no-match basket, per-row excluded type, malformed regex name (e.g. unbalanced parens). ## Files - ``R/search.R`` (381 lines): public ``cog_gov_search()`` mode dispatch, ``.validate_basket_args()``, ``.resolve_basket_row()``, ``.disambiguate()``, ``.resolve_basket()``, ``.escape_regex()``. - ``R/basket.R`` (138 lines, new): ``.build_sidecar()``, ``.type_to_label()``, ``.basket_summary_message()``, plus exports ``cog_basket_resolution()`` and ``cog_basket_unresolved()``. - ``tests/testthat/test-search.R``: +35 test cases covering basket mode. - ``tests/testthat/test-basket.R`` (new): 5 sidecar-accessor tests. - ``tests/testthat/test-spending.R``: +1 pipe-through smoke test. - ``NEWS.md`` (new), ``_pkgdown.yml`` (extended reference index). ## Deferred follow-ups - Make all-NA ``population_acs`` tiebreaker deterministic (add ``canonical_govid`` as secondary sort key). Rare in practice with the current corpus. - Extract ``.state_abbrev_to_fips`` to ``R/lookups.R`` for cohesion (~15 line reduction in ``search.R``). ## Spec & plan Brainstormed and planned through ``superpowers:`` skills. Spec at ``docs/superpowers/specs/2026-04-28-cog-gov-search-basket-mode-design.md``, plan at ``docs/superpowers/plans/2026-04-28-cog-gov-search-basket-mode.md`` (both gitignored). Implementation followed subagent-driven development: 7 implementer dispatches, 14 reviews (7 spec compliance + 7 code quality), 1 cross-task whole-branch review that caught two soft-fail-contract violations (F1: per-row excluded type aborting; F2: malformed regex name unhandled), both fixed in the final commit.

jared added 12 commits 2026-04-28 14:47:28 -04:00

refactor(search): rename cog_gov_search() first argument to name 68b72faeae

Pre-rename in preparation for basket mode. All existing callers in this
package and cog_explorer/ pass the first argument positionally, so this
rename is non-breaking. No deprecation alias added per design spec
(no external consumers; package is pre-release v0.1.0).

@param roxygen also updated to match the new formal; man/cog_gov_search.Rd
regenerated via devtools::document().

feat(search): add basket-mode argument validator 7fd29c919d

Internal .validate_basket_args() handles length validation and
recycling of state/type from length 1. Foundation for basket mode.

feat(search): add per-row resolver — exact match branch 4dcf1c72b2

.resolve_basket_row() handles the exact-match case. Substring fallback
and disambiguation branches follow in subsequent commits.

feat(search): add substring fallback and no_match handling 392e818f52

.resolve_basket_row() now falls back to case-insensitive substring
match when no exact match is found, and short-circuits empty/whitespace
input to no_match. Disambiguation stub raises pending Task 5.

feat(search): add disambiguation — largest_pop and ambiguous branches 2f47f7ae67

Multi-row matches within a single govs_type pick the largest-population
row (status=largest_pop). Multi-row matches spanning >=2 types return no
basket row (status=ambiguous) with all candidates preserved for the
sidecar.

feat(search): basket mode for cog_gov_search() 98ed6318a6

Vector name + state + type arguments dispatch to a per-row resolver
that produces a basket tibble with a 'resolution' sidecar attribute.
Utility mode (length-1 name) is unchanged.

feat(search): post-resolution summary message in basket mode d9bf0552b7

Emits a single cli::cli_inform when any input was ambiguous, missed,
or fell back to largest-population. Silent on clean baskets.

refactor(search): split sidecar helpers into R/basket.R ae04d4f48f

Moves .build_sidecar(), .type_to_label(), .basket_summary_message()
out of R/search.R and into a new R/basket.R. No behavior change;
brings R/search.R back under its 350-line budget. R/basket.R will
also host the public sidecar accessors added in the next commit.

feat: export cog_basket_resolution() and cog_basket_unresolved() 56fdd8e5b9

Sidecar accessors for basket-mode results. cog_basket_resolution()
returns the full resolution tibble (drops candidates list-col by
default for readable printing). cog_basket_unresolved() filters to
ambiguous/no_match rows for iterative refinement.

test: smoke test cog_gov_search basket -> cog_spending pipe 7475696853

Confirms a basket result pipes cleanly into the existing query verb
without any input-coercion friction.

docs: roxygen, NEWS, and pkgdown for basket mode 24e67791a0

Adds full @description, @details (algorithm), @examples on
cog_gov_search(); examples on cog_basket_resolution() /
cog_basket_unresolved(); NEWS.md entry covering the new mode and
the pattern->name rename; pkgdown reference entries for the two
new exports.

fix(search): soft-fail on per-row excluded type and malformed regex name

R-CMD-check / check (push) Successful in 1m37s

Details

R-CMD-check / check (pull_request) Successful in 1m37s

Details

efc0bd16b1

Cross-task review found two edge cases that violated the basket-mode
soft-fail contract:

- Per-row excluded type (e.g. type = c(NA, "special_district")) hit
  .coerce_type()'s abort inside the per-row resolver, killing the
  whole basket call. Now treated as no_match in the sidecar.
- Malformed regex in the substring fallback (e.g. name = "San(Diego")
  propagated DuckDB engine errors. .escape_regex() now backslash-
  escapes meta characters before the regexp_matches call. Utility-
  mode regex behavior is unchanged.

Plus a new public-surface test for the all-no-match case.

jared merged commit e7fa51eec7 into main

2026-04-28 15:09:44 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Civilytics/uscogdata#1