fix(#3): normalize the corpus URL's trailing slash at resolution
Investigating "gov search doesn't work" (#3) turned up two separate things.
THE REPORTED SYMPTOM IS ALREADY FIXED.
#3 reported `cog_gov_search("Orange")` dying in jsonlite with
`lexical error: invalid char in json text. <html> <head>`. That was fixed the
same day the issue was filed, by 8743472 "fix(manifest): actionable errors when
USCOGDATA_URL is unset or returns non-JSON" (issue filed 2026-05-27 11:30;
commit 2026-05-27). The issue was simply never closed. Verified now: injecting
an HTML manifest.json raises a typed `uscogdata_invalid_manifest` condition
naming the likely causes, with the raw parse error demoted to a footnote, and
`cog_gov_search("Orange")` returns 62 rows against the live corpus.
THE ROOT CAUSE OF THAT HTML WAS STILL LIVE -- and is what this commit fixes.
Every consumer builds locations by CONCATENATION:
manifest.R:95 paste0(url, "manifest.json")
mirror.R:48,125 paste0(url, e$path)
views.R the parquet glob
and mirror.R:104 documents the invariant outright ('url ends in "/"'). The
error messages tell users to set `"<url-or-local-path>/"`. But `.resolve_url()`
was a bare `.cfg("url")` passthrough -- the invariant was assumed everywhere and
enforced nowhere.
So a URL entered without the slash failed silently and misleadingly:
HTTPS -> ".../downloadmanifest.json"; the host answers with an HTML 404 page,
which lands in the JSON parser as EXACTLY the #3 symptom -- and the
guard then blames "login page / 404 / wrong share" when the real
cause was one missing character.
local -> ".../corpusdata/long/**/*.parquet" and a DuckDB "No files found".
Reproduced both: pointing USCOGDATA_URL at the bundled fixture without a
trailing slash gave
No files found that match ".../fixture_corpusdata/long/**/*.parquet"
Normalizing once at resolution fixes every consumer at the same time, rather
than having each call site re-derive the invariant. An empty setting passes
through untouched so manifest.R's "not configured" guard still fires instead of
the value degrading into a bare "/" filesystem root.
RED->GREEN: 4 tests added, 2 failed first (append-missing-slash, local-path
normalization); the already-correct cases (slash present, empty setting) passed
throughout and pin them against regression. Same fixture path that produced the
DuckDB error above now returns 62 rows.
Suite: FAIL 0 | WARN 0 | SKIP 0 | PASS 471 (was 463; +8 = the new tests).
This commit is contained in:
+24
-1
@@ -21,7 +21,30 @@
|
||||
.uscogdata_defaults[[key]]
|
||||
}
|
||||
|
||||
.resolve_url <- function() .cfg("url")
|
||||
#' Resolve the corpus URL, guaranteeing the trailing slash the package assumes.
|
||||
#'
|
||||
#' Every consumer builds locations by CONCATENATION -- `paste0(url,
|
||||
#' "manifest.json")` in manifest.R, `paste0(url, e$path)` in mirror.R, and the
|
||||
#' parquet glob in views.R -- and mirror.R:104 documents the invariant outright
|
||||
#' ('url ends in "/"'). Nothing enforced it, so a URL entered without the slash
|
||||
#' failed silently and misleadingly:
|
||||
#'
|
||||
#' HTTPS -> ".../downloadmanifest.json"; the host answers with an HTML 404
|
||||
#' page, which lands in the JSON parser as the lexical error
|
||||
#' reported in issue #3 -- pointing the user at "login page / wrong
|
||||
#' share" when the real cause was one missing character.
|
||||
#' local -> ".../corpusdata/long/**/*.parquet" and a DuckDB "No files found".
|
||||
#'
|
||||
#' Normalizing here fixes every consumer at once, rather than each call site
|
||||
#' re-deriving the same invariant. An empty setting is passed through
|
||||
#' untouched so manifest.R's "not configured" guard still fires instead of the
|
||||
#' value degrading into a bare "/" filesystem root.
|
||||
#' @noRd
|
||||
.resolve_url <- function() {
|
||||
url <- .cfg("url")
|
||||
if (is.null(url) || !nzchar(url) || grepl("/$", url)) return(url)
|
||||
paste0(url, "/")
|
||||
}
|
||||
|
||||
.resolve_cache_dir <- function() {
|
||||
v <- .cfg("cache_dir")
|
||||
|
||||
Reference in New Issue
Block a user