That was fixed the same day the issue was filed, by 8743472"fix(manifest): actionable errors when USCOGDATA_URL is unset or returns non-JSON" (issue filed 2026-05-27 11:30; commit 2026-05-27). The issue was simply never closed.
Verified now, not assumed:
Injecting an HTML manifest.json raises a typed uscogdata_invalid_manifest condition that names the likely causes, with the raw jsonlite error demoted to a footnote.
cog_gov_search("Orange") returns 62 rows against the live corpus.
.fetch_or_cache_manifest() now parses before persisting, recovers from poisoned caches, and writes atomically — and manifest.json is the package's only JSON parse, so no unguarded path remains.
But the root cause of that HTML was still live
Every consumer builds locations by concatenation:
site
code
manifest.R:95
paste0(url, "manifest.json")
mirror.R:48,125
paste0(url, e$path)
views.R
the parquet glob
mirror.R:104 documents the invariant outright — url ends in "/" — and the error messages tell users to set "<url-or-local-path>/". But .resolve_url() was a bare .cfg("url") passthrough. The invariant was assumed everywhere and enforced nowhere.
A URL entered without the slash therefore failed silently and misleadingly:
HTTPS → .../downloadmanifest.json; the host answers with an HTML 404 page, which lands in the JSON parser as exactly the #3 symptom — and the guard then blames "login page / 404 / wrong share" when the real cause was one missing character.
local → .../corpusdata/long/**/*.parquet → DuckDB No files found.
Both reproduced. Pointing USCOGDATA_URL at the bundled fixture without a trailing slash gave:
No files found that match ".../fixture_corpusdata/long/**/*.parquet"
^^^^ missing separator
The fix
Normalize once, at resolution — so every consumer is fixed at the same time instead of each call site re-deriving the invariant. An empty setting passes through untouched, so manifest.R's "not configured" guard still fires rather than the value degrading into a bare / filesystem root.
Verification
RED→GREEN: 4 tests added; 2 failed first (append-missing-slash, local-path normalization). The already-correct cases (slash present, empty setting) passed throughout and now pin them against regression.
The fixture path that produced the DuckDB error above now returns 62 rows.
Suite: FAIL 0 | WARN 0 | SKIP 0 | PASS 471 (was 463; +8 = the new tests).
Investigating #3 turned up **two separate things**, and only one of them still needed fixing.
## The reported symptom was already fixed — in May
#3 reported:
```r
> cog_gov_search("Orange")
Error in `parse_con()`:
! lexical error: invalid char in json text.
<html> <head>
```
That was fixed **the same day the issue was filed**, by `8743472` *"fix(manifest): actionable errors when USCOGDATA_URL is unset or returns non-JSON"* (issue filed 2026-05-27 11:30; commit 2026-05-27). The issue was simply never closed.
Verified now, not assumed:
- Injecting an HTML `manifest.json` raises a typed `uscogdata_invalid_manifest` condition that names the likely causes, with the raw jsonlite error demoted to a footnote.
- `cog_gov_search("Orange")` returns **62 rows** against the live corpus.
- `.fetch_or_cache_manifest()` now parses **before** persisting, recovers from poisoned caches, and writes atomically — and `manifest.json` is the package's **only** JSON parse, so no unguarded path remains.
## But the root cause of that HTML was still live
Every consumer builds locations by **concatenation**:
| site | code |
|---|---|
| `manifest.R:95` | `paste0(url, "manifest.json")` |
| `mirror.R:48,125` | `paste0(url, e$path)` |
| `views.R` | the parquet glob |
`mirror.R:104` documents the invariant outright — *`url ends in "/"`* — and the error messages tell users to set `"<url-or-local-path>/"`. But `.resolve_url()` was a bare `.cfg("url")` passthrough. **The invariant was assumed everywhere and enforced nowhere.**
A URL entered without the slash therefore failed silently and misleadingly:
- **HTTPS** → `.../downloadmanifest.json`; the host answers with an HTML 404 page, which lands in the JSON parser as **exactly the #3 symptom** — and the guard then blames *"login page / 404 / wrong share"* when the real cause was one missing character.
- **local** → `.../corpusdata/long/**/*.parquet` → DuckDB `No files found`.
Both reproduced. Pointing `USCOGDATA_URL` at the bundled fixture *without* a trailing slash gave:
```
No files found that match ".../fixture_corpusdata/long/**/*.parquet"
^^^^ missing separator
```
## The fix
Normalize once, at resolution — so every consumer is fixed at the same time instead of each call site re-deriving the invariant. An empty setting passes through untouched, so `manifest.R`'s "not configured" guard still fires rather than the value degrading into a bare `/` filesystem root.
## Verification
- **RED→GREEN**: 4 tests added; **2 failed first** (append-missing-slash, local-path normalization). The already-correct cases (slash present, empty setting) passed throughout and now pin them against regression.
- The fixture path that produced the DuckDB error above now returns **62 rows**.
- Suite: **FAIL 0 | WARN 0 | SKIP 0 | PASS 471** (was 463; +8 = the new tests).
Closes #3.
Investigating "gov search doesn't work" (#3) turned up two separate things.
THE REPORTED SYMPTOM IS ALREADY FIXED.
#3 reported `cog_gov_search("Orange")` dying in jsonlite with
`lexical error: invalid char in json text. <html> <head>`. That was fixed the
same day the issue was filed, by 8743472 "fix(manifest): actionable errors when
USCOGDATA_URL is unset or returns non-JSON" (issue filed 2026-05-27 11:30;
commit 2026-05-27). The issue was simply never closed. Verified now: injecting
an HTML manifest.json raises a typed `uscogdata_invalid_manifest` condition
naming the likely causes, with the raw parse error demoted to a footnote, and
`cog_gov_search("Orange")` returns 62 rows against the live corpus.
THE ROOT CAUSE OF THAT HTML WAS STILL LIVE -- and is what this commit fixes.
Every consumer builds locations by CONCATENATION:
manifest.R:95 paste0(url, "manifest.json")
mirror.R:48,125 paste0(url, e$path)
views.R the parquet glob
and mirror.R:104 documents the invariant outright ('url ends in "/"'). The
error messages tell users to set `"<url-or-local-path>/"`. But `.resolve_url()`
was a bare `.cfg("url")` passthrough -- the invariant was assumed everywhere and
enforced nowhere.
So a URL entered without the slash failed silently and misleadingly:
HTTPS -> ".../downloadmanifest.json"; the host answers with an HTML 404 page,
which lands in the JSON parser as EXACTLY the #3 symptom -- and the
guard then blames "login page / 404 / wrong share" when the real
cause was one missing character.
local -> ".../corpusdata/long/**/*.parquet" and a DuckDB "No files found".
Reproduced both: pointing USCOGDATA_URL at the bundled fixture without a
trailing slash gave
No files found that match ".../fixture_corpusdata/long/**/*.parquet"
Normalizing once at resolution fixes every consumer at the same time, rather
than having each call site re-derive the invariant. An empty setting passes
through untouched so manifest.R's "not configured" guard still fires instead of
the value degrading into a bare "/" filesystem root.
RED->GREEN: 4 tests added, 2 failed first (append-missing-slash, local-path
normalization); the already-correct cases (slash present, empty setting) passed
throughout and pin them against regression. Same fixture path that produced the
DuckDB error above now returns 62 rows.
Suite: FAIL 0 | WARN 0 | SKIP 0 | PASS 471 (was 463; +8 = the new tests).
jared
merged commit e581e7360c into main2026-07-25 19:03:12 -04:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Investigating #3 turned up two separate things, and only one of them still needed fixing.
The reported symptom was already fixed — in May
#3 reported:
That was fixed the same day the issue was filed, by
8743472"fix(manifest): actionable errors when USCOGDATA_URL is unset or returns non-JSON" (issue filed 2026-05-27 11:30; commit 2026-05-27). The issue was simply never closed.Verified now, not assumed:
manifest.jsonraises a typeduscogdata_invalid_manifestcondition that names the likely causes, with the raw jsonlite error demoted to a footnote.cog_gov_search("Orange")returns 62 rows against the live corpus..fetch_or_cache_manifest()now parses before persisting, recovers from poisoned caches, and writes atomically — andmanifest.jsonis the package's only JSON parse, so no unguarded path remains.But the root cause of that HTML was still live
Every consumer builds locations by concatenation:
manifest.R:95paste0(url, "manifest.json")mirror.R:48,125paste0(url, e$path)views.Rmirror.R:104documents the invariant outright —url ends in "/"— and the error messages tell users to set"<url-or-local-path>/". But.resolve_url()was a bare.cfg("url")passthrough. The invariant was assumed everywhere and enforced nowhere.A URL entered without the slash therefore failed silently and misleadingly:
.../downloadmanifest.json; the host answers with an HTML 404 page, which lands in the JSON parser as exactly the #3 symptom — and the guard then blames "login page / 404 / wrong share" when the real cause was one missing character..../corpusdata/long/**/*.parquet→ DuckDBNo files found.Both reproduced. Pointing
USCOGDATA_URLat the bundled fixture without a trailing slash gave:The fix
Normalize once, at resolution — so every consumer is fixed at the same time instead of each call site re-deriving the invariant. An empty setting passes through untouched, so
manifest.R's "not configured" guard still fires rather than the value degrading into a bare/filesystem root.Verification
Closes #3.