Commit Graph
6 Commits
Author SHA1 Message Date
jared 013af5b0d2 fix: ship the vignettes
.Rbuildignore excluded ^vignettes$, ^doc$ and ^Meta$, so an installed
uscogdata had no vignettes at all -- while the README instructed users to
run vignette("total-spending"), which failed for every one of them.

Both build offline: total-spending points USCOGDATA_URL at the bundled
fixture, population-denominators is eval = FALSE. Confirmed present in the
built tarball as both source and rendered inst/doc/.

The test also pins the fixture as never-excluded -- it is what lets
R CMD check run with no credentials on r-universe and GitHub Actions.
2026-08-08 17:48:07 -04:00
jared 4a92f36d44 chore: add full MIT text, name the copyright holder properly
LICENSE held only the two-line stub and no LICENSE.md existed, so the
repo carried no license text for a human browsing it or for GitHub's
license detector.

usethis::use_mit_license() writes LICENSE.md but leaves an existing
LICENSE alone, so the stub kept saying 'Civilytics' while the full text
said 'Civilytics Consulting LLC'. Corrected by hand, with a test pinning
the two together.
2026-08-08 17:47:18 -04:00
jared a67735f121 chore: release metadata -- author of record, URLs, schema ceiling, 0.3.0
Authors@R was an org with no human, so citation() and the r-universe
maintainer page had nothing to render and ORCID could not collate this
with merTools. The given-name vector c("Jared", "E.") matches merTools
exactly; person("Jared", "E. Knowles") would render the same but put the
middle initial in the family-name slot.

MaxCorpusSchema claimed 5 while .validate_schema() accepts 4-7 and the
published corpus is 7 -- metadata contradicting code by two versions.

0.3.0 rather than 0.2.0: remote reads go from broken to working and the
default URL from placeholder to live, which is user-visible behaviour.
2026-08-08 17:46:39 -04:00
jared 4300b636b1 feat: default to the public corpus so the package works unconfigured
The default was a REPLACE_WITH_SHARE_TOKEN sentinel and no document in
the package supplied a working URL, so a new user installing uscogdata
had no path to a session at all -- just an actionable-looking error with
nothing actionable behind it.

The default is now the public HuggingFace mirror: CC-BY-4.0, no
credential, CDN-backed, and it keeps the origin's uplink out of the read
path. USCOGDATA_URL and options(uscogdata.url=) still override, so
Nextcloud and cog_mirror() copies are unaffected.

The sentinel guard stays for half-edited configs; the two tests covering
it set the URL explicitly, so they only needed renaming to stop calling
it 'the default'.
2026-08-08 17:18:56 -04:00
jared 748ca4a56e fix(#3): normalize the corpus URL's trailing slash at resolution
R-CMD-check / check (pull_request) Successful in 2m33s
R-CMD-check / check (push) Successful in 2m42s
Investigating "gov search doesn't work" (#3) turned up two separate things.

THE REPORTED SYMPTOM IS ALREADY FIXED.
#3 reported `cog_gov_search("Orange")` dying in jsonlite with
`lexical error: invalid char in json text. <html> <head>`. That was fixed the
same day the issue was filed, by 8743472 "fix(manifest): actionable errors when
USCOGDATA_URL is unset or returns non-JSON" (issue filed 2026-05-27 11:30;
commit 2026-05-27). The issue was simply never closed. Verified now: injecting
an HTML manifest.json raises a typed `uscogdata_invalid_manifest` condition
naming the likely causes, with the raw parse error demoted to a footnote, and
`cog_gov_search("Orange")` returns 62 rows against the live corpus.

THE ROOT CAUSE OF THAT HTML WAS STILL LIVE -- and is what this commit fixes.

Every consumer builds locations by CONCATENATION:
  manifest.R:95   paste0(url, "manifest.json")
  mirror.R:48,125 paste0(url, e$path)
  views.R         the parquet glob
and mirror.R:104 documents the invariant outright ('url ends in "/"'). The
error messages tell users to set `"<url-or-local-path>/"`. But `.resolve_url()`
was a bare `.cfg("url")` passthrough -- the invariant was assumed everywhere and
enforced nowhere.

So a URL entered without the slash failed silently and misleadingly:
  HTTPS -> ".../downloadmanifest.json"; the host answers with an HTML 404 page,
           which lands in the JSON parser as EXACTLY the #3 symptom -- and the
           guard then blames "login page / 404 / wrong share" when the real
           cause was one missing character.
  local -> ".../corpusdata/long/**/*.parquet" and a DuckDB "No files found".

Reproduced both: pointing USCOGDATA_URL at the bundled fixture without a
trailing slash gave
  No files found that match ".../fixture_corpusdata/long/**/*.parquet"

Normalizing once at resolution fixes every consumer at the same time, rather
than having each call site re-derive the invariant. An empty setting passes
through untouched so manifest.R's "not configured" guard still fires instead of
the value degrading into a bare "/" filesystem root.

RED->GREEN: 4 tests added, 2 failed first (append-missing-slash, local-path
normalization); the already-correct cases (slash present, empty setting) passed
throughout and pin them against regression. Same fixture path that produced the
DuckDB error above now returns 62 rows.

Suite: FAIL 0 | WARN 0 | SKIP 0 | PASS 471 (was 463; +8 = the new tests).
2026-07-25 18:37:42 -04:00
jared f7035049e7 feat: package skeleton — DESCRIPTION, NAMESPACE, session/manifest/cache/views
Minimal skeleton for uscogdata v0.1. Internal session layer with lazy
cog_open(), manifest fetch+validate+cache, view registration placeholder.
Depends on DuckDB >=1.0, httr2, jsonlite. inst/schemas/provenance-v1.json
ships the structured provenance JSON Schema.
2026-04-23 09:05:54 -04:00