jared
98bf0d9720
#39 pushed pagination into SQL for cog_spending()/cog_revenue(); the other two verbs were left materializing everything and slicing in R -- the pattern behind the 2026-08-06 production incident. cog_gov_search() had no LIMIT at all, so an unfiltered call returns the entire 40,336-row crosswalk. Extracted the #39 machinery into R/pagination.R first (.validate_pagination(), .paginate_sql(), .take_pagination_total()) rather than growing a third inline copy: three definitions of what total_rows means is three places for it to drift. Conflict refusals stay at the call sites because each verb's conflict set differs. .verb_spendrev() now uses the shared helpers and is unchanged in behaviour. The empty-page fallback query is now passed as a thunk, so the unpaginated SQL is only BUILT when an offset actually lands past the end instead of on every paged call. Two things #57 did not anticipate: - cog_gov_search()'s ORDER BY was not a total order. population_acs DESC NULLS LAST leaves ties -- and the whole NULL block -- in scan order, so two requests can order them differently and a paged sweep duplicates one row while dropping another. Added canonical_govid as tiebreaker. Unpaginated output changes only in the relative order of already-tied rows. - Basket mode returns one resolved row per requested name plus a sidecar covering all of them, so a page of it is not a page of anything the caller asked for. Refused with uscogdata_basket_pagination_conflict rather than silently ignoring the arguments. Both default to NULL, so cog-api adopts them behind its existing formals() probe with no lockstep deploy. Suite: 1067 passed, 0 failed, 0 warnings (2 pre-existing live-corpus skips).
119 lines
4.4 KiB
R
119 lines
4.4 KiB
R
% Generated by roxygen2: do not edit by hand
|
|
% Please edit documentation in R/search.R
|
|
\name{cog_gov_search}
|
|
\alias{cog_gov_search}
|
|
\title{Search for governments by name, state, and/or type}
|
|
\usage{
|
|
cog_gov_search(
|
|
name = NULL,
|
|
state = NULL,
|
|
type = NULL,
|
|
limit = NULL,
|
|
offset = NULL
|
|
)
|
|
}
|
|
\arguments{
|
|
\item{name}{Character vector of place name(s). Length 1 = utility mode;
|
|
length >1 = basket mode.}
|
|
|
|
\item{state}{2-letter USPS abbreviation (e.g. `"FL"`), FIPS integer
|
|
(e.g. `12`), or `NULL`. In basket mode, length 1 recycles across
|
|
all entries; otherwise must match `length(name)`.}
|
|
|
|
\item{type}{Government type: integer in `0:3` or one of `"state"`,
|
|
`"county"`, `"city"`, `"township"`, or `NA`/`NULL`. Per-row optional
|
|
in basket mode (recycles from length 1). Excluded types `4`/`5` (or
|
|
`"special_district"` / `"school_district"`) trigger an explanatory
|
|
message and an empty result.}
|
|
|
|
\item{limit}{Maximum number of rows to return, applied in SQL. `NULL`
|
|
(default) returns every match -- which, with no other filter, is the
|
|
entire crosswalk. Utility mode only: pagination has no meaning in basket
|
|
mode, where the result is one resolved row per requested name in input
|
|
order, and is refused there with class
|
|
`uscogdata_basket_pagination_conflict`.}
|
|
|
|
\item{offset}{Rows to skip before `limit` starts counting (0-based).
|
|
Ignored if `limit` is `NULL`; defaults to `0L` when `limit` is set.}
|
|
}
|
|
\value{
|
|
A tibble of `canonical_fips_xwalk` rows. In utility mode, all
|
|
matches sorted by `population_acs` desc, ties broken by
|
|
`canonical_govid`. In basket mode, resolved rows in input order, with
|
|
`attr(., "resolution")` set to the sidecar tibble.
|
|
|
|
When `limit` is set, carries a `total_rows` attribute: the full
|
|
unpaginated match count, computed by the same query (`COUNT(*) OVER()`)
|
|
rather than a second scan.
|
|
}
|
|
\description{
|
|
Resolves human-readable place names into rows of `canonical_fips_xwalk`,
|
|
the cross-vintage canonical-government registry. Operates in two modes:
|
|
}
|
|
\details{
|
|
* **Utility mode** (single `name`, the original behavior): returns all
|
|
rows whose `gov_name` contains `name` as a **literal, case-insensitive
|
|
substring**, sorted by `population_acs` descending. Useful for
|
|
exploratory lookups. Regex metacharacters in `name` are escaped, so a
|
|
government is findable by its own complete name even when that name
|
|
contains parentheses or a period.
|
|
* **Basket mode** (`length(name) > 1`): resolves each input row to a
|
|
single canonical govid and returns a tibble in input order, suitable
|
|
for piping straight into [cog_spending()] / [cog_revenue()] /
|
|
[cog_geographic_rollup()]. Carries an audit sidecar accessible via
|
|
[cog_basket_resolution()] / [cog_basket_unresolved()].
|
|
|
|
|
|
**Basket-mode resolution algorithm** (per input row):
|
|
1. Filter `canonical_fips_xwalk` by `state` and (if non-NA) `type`.
|
|
2. **Exact pass:** case-insensitive equality against `gov_name`.
|
|
Single hit -> resolved. Multiple -> step 4.
|
|
3. **Substring fallback:** case-insensitive literal substring against
|
|
`gov_name` (metacharacters escaped).
|
|
Single hit -> resolved (`match_method = "substring"`). Zero hits ->
|
|
`status = "no_match"`. Multiple hits -> step 4.
|
|
4. **Disambiguation:** if matches share one `govs_type`, pick the
|
|
largest-population row (`status = "largest_pop"`). If they span >=2
|
|
types, no row is added (`status = "ambiguous"`); the user should
|
|
re-run with `type` specified.
|
|
|
|
Resolved rows form the returned tibble in input order. Unresolved
|
|
inputs (`ambiguous` / `no_match`) appear only in the sidecar.
|
|
}
|
|
\examples{
|
|
\dontrun{
|
|
# Utility mode — exploratory substring lookup
|
|
cog_gov_search("broward", state = "FL")
|
|
|
|
# Basket mode — resolve a known cohort
|
|
basket <- cog_gov_search(
|
|
name = c("BROWARD COUNTY", "SAN DIEGO CITY", "AUSTIN CITY"),
|
|
state = c("FL", "CA", "TX")
|
|
)
|
|
basket
|
|
|
|
# Inspect resolution audit
|
|
cog_basket_resolution(basket)
|
|
|
|
# Pipe into a spending query
|
|
library(dplyr)
|
|
basket |> cog_spending(years = 2019:2020, category = "Police")
|
|
|
|
# Iteratively refine ambiguous matches
|
|
partial <- cog_gov_search(
|
|
name = c("Broward", "San Diego"), # San Diego is ambiguous
|
|
state = c("FL", "CA")
|
|
)
|
|
cog_basket_unresolved(partial)
|
|
refined <- cog_gov_search(
|
|
name = c("Broward", "San Diego"),
|
|
state = c("FL", "CA"),
|
|
type = c(NA, "city") # disambiguate
|
|
)
|
|
}
|
|
}
|
|
\seealso{
|
|
[cog_basket_resolution()], [cog_basket_unresolved()],
|
|
[cog_spending()], [cog_revenue()].
|
|
}
|