2 Commits

Include renames

Author	SHA1	Message	Date
jared	0fbae00e27	feat: name a cohort by state/type predicate instead of a 40k-id IN list ... cog_spending(), cog_revenue() and cog_balances() gain optional state/type arguments. Both default to NULL, so every existing govid-based call is unchanged. The verbs took a cohort only as a govid vector, which .sql_lit_chr() rendered into a quoted IN list and .verb_spendrev() embedded into 5-8 separate statements per call: the scope check, the main aggregate, the per-capita join, the harmonization block, and the suggestion and suppression queries. For type = "city" that list is 301,589 characters, parsed and planned from scratch every time it appears. Passing state/type instead expresses the cohort as a subquery against canonical_fips_xwalk, so its size never enters the SQL string at all. Measured on the production corpus, same FY2022 aggregate over the 20,106-government city cohort, DUCKDB_THREADS=2, median of 5: IN (20,106 literals) -- 0.3.0 432 ms join against a temp cohort table 132 ms predicate on canonical_fips_xwalk 102 ms no cohort filter at all (the floor) 105 ms The predicate reaches the no-filter floor: the cohort restriction is now free. End to end through cog_spending(category = "Police"), 1080 ms -> 271 ms, 3.99x -- larger than the single-query saving, because the repetition across statements is what actually cost. Design decisions, both made explicitly rather than left implicit: - govid AND state/type INTERSECT. "These ids, narrowed to that state/type" is a real query, and an error here could never be relaxed later without breaking callers. - A predicate cohort has no id list to report, so provenance$scope$govids_found/govids_missing stay empty and a new scope$cohort block carries state, type and n_governments. Resolving the ids just to report them would put 20,000 govids in every fleet-scale response body -- the cost this change removes. A govid-named cohort's provenance is untouched. state/type are coerced with .coerce_state_to_fips()/.coerce_type(), the same helpers cog_gov_search() uses. That is load-bearing: the argument is a postal abbreviation ("WI") while fips_state holds a FIPS code ("55"), and a predicate on the raw parameter matches nothing and returns an empty result indistinguishable from "reported nothing". cog-api hit exactly this trap optimizing the same path. .attach_per_capita() now keys its population lookup on the govids present in the result rather than the requested cohort. Those are the only ones its LEFT JOIN can match, so the output is identical -- but it needs no id list, and on a paginated call it looks up one page instead of the fleet. Fixes uscogdata#58.	2026-08-09 14:15:21 -04:00
jared	4de915b557	feat: cog_recipes + recipe= + signposting suggestions ... Adds cog_recipes() to list the curated harmonization_recipes catalog (24 recipes / schema_version >= 5), and a recipe= argument on cog_spending()/ cog_revenue() that runs a recipe's generic multi-code join instead of the category view: SUM(amt * weight) across whichever component codes are present for a (year, canonical_govid), scoped by gov_type_scope. The join deliberately does not filter is_aggregate -- the wide era (<= 2011) exposes these split families (corrections 04+05, IG *89/*47, U4- rents, etc.) ONLY as aggregate rows, with leaf codes first appearing in 2012, so excluding aggregates would zero out the wide-era half of every recipe. This is safe by corpus construction: wide-era rows are aggregate-only, modern rows are leaf-only, and every component is year-scoped, so there is no double-counting. recipe= is mutually exclusive with category=; the result's subtype column reads "recipe" and category reads the recipe's label. Adds recipe-component-driven signposting: when a basis="harmonized" + category query comes back with zero rows in a requested year, and a harmonization recipe covering that category would actually produce rows for this government in that year (via the same join .run_recipe() uses), the recipe is surfaced in provenance$suggestions plus one cli::cli_inform() message. This is deliberately keyed off recipe components rather than harmonization_map's suggested_recipe_id column (which is empty on every live row -- the wide era's split families are NA-by-construction via aggregate exclusion, not an NA ruling to hang a suggestion off of). Also populates the previously-always-empty provenance$series_break_refs (schema v5 only: series_breaks_pq rows whose fin_code is among the observed codes and whose break_year falls in the requested span), and extends cog_explain() with Basis/Harmonization/Recipe/Suggestions/Series breaks sections.	2026-07-18 23:33:02 -04:00