docs: correct the error-bar convention and README to match the actual app
Deploy to git-pages / deploy (push) Successful in 22s

AGENTS.md's "Error Bar Convention" paragraph was wrong in four ways after the
last docs pass: it pointed "above" at a section that is below it, described the
fallback as "synthetic draw generation" when it generates no draws at all, cited
an `intervalWidth / 3.29` expression in RateDensityRidgeline.jsx that does not
exist, and listed ModelDrawsComparison.jsx, a file that was deleted when the
demo was reduced to 3 charts. Rewritten against the current code: the fallback
is fitSkewedInterval's analytic two-piece normal, whose divisor is
probit((1 + intervalMass) / 2) with intervalMass defaulting to 0.90, and the
"if you change this" list now names the three files that actually encode 90%.

README: 6 charts -> 3, the ridgeline is Chart 3 not Chart 5, Chart 2's use of
real draws is now mentioned, the D3.js v7 claim is dropped (d3 is not a
dependency), @duckdb/duckdb-wasm is listed in the tech stack, the file tree
matches src/, and the container-size note accounts for the wasm engine.
This commit is contained in:
2026-08-11 10:37:37 -04:00
parent 096455944e
commit db5111dab9
2 changed files with 30 additions and 24 deletions
+6 -3
View File
@@ -46,11 +46,14 @@ App.jsx (router)
### Error Bar Convention: 90% Intervals
The API returns 95% HPD intervals (`count_lower`, `count_upper`). However, all chart labels and calculations in this app use **90% intervals**. When converting HPD bounds to standard deviations for the synthetic draw generation fallback (used in the ridgeline chart if real draws can't be fetched — see "Real posterior draws via duckdb-wasm" above), the divisor used is **3.29** (corresponding to z = 1.645 for a two-tailed 90% interval).
The API returns 95% HPD intervals (`count_lower`, `count_upper`). However, all chart labels and calculations in this app use **90% intervals**.
The 90% convention is baked into the analytic fallback in `src/utils/distributionApprox.js` (the path used when real posterior draws can't be fetched — see the duckdb-wasm section below). That fallback generates **no draws**: `fitSkewedInterval` fits a two-piece normal directly from `{median, lower, upper}`, converting each half-interval to its own sigma with `z = probit((1 + intervalMass) / 2)` and `intervalMass` defaulting to `0.90` (so z ≈ 1.645). For a symmetric interval that is equivalent to the old fixed "full width / 3.29" divisor, but it is computed from `intervalMass` rather than hardcoded, and each side gets its own sigma so the fitted shape stays skewed.
If you change this convention, update:
- `RateDensityRidgeline.jsx` — SD calculation (`intervalWidth / 3.29`) and label text
- `ModelDrawsComparison.jsx` — Legend labels mentioning "90%"
- `distributionApprox.js` — the `intervalMass = 0.90` default in `fitSkewedInterval`, and its JSDoc claim that the API's bounds are a 90% interval
- `RateByGroupBar.jsx` — the caption and component JSDoc, both of which say "the model's reported 90% interval"
- `ArrestsOverTime.jsx` — the legend label "Modeled (median + 90% interval)"
- Any documentation referencing confidence/credible intervals
## Common Pitfalls & Gotchas
+24 -21
View File
@@ -18,22 +18,19 @@ npm run preview # serve built files locally
## What It Does
Visitors select a U.S. state, search for a school district (with suggestions of districts that have the most arrests), and see 6 charts comparing observed data against Bayesian model estimates:
Visitors select a U.S. state, search for a school district (with suggestions of districts that have the most arrests), and see 3 charts comparing observed data against Bayesian model estimates:
1. **Arrests over time** (Chart 1) — raw counts by CRDC wave with per-1k rate labels, built with inline SVG
2. **Rate by student group** (Chart 2) — bar chart, most recent year (observed vs. modeled), SVG
3. **District vs. national** (Chart 3) — highest-rate group compared to the U.S. average, SVG
4. **Model predictions vs. observed** (Chart 4) — four quadrants (one-year/three-year × baseline/covariate)
5. **Predicted rates by student group** (Chart 5) — density ridges built from each group's real 500-draw posterior (fetched client-side via duckdb-wasm from the public Hugging Face parquet dataset), with diamond markers for observed rates; falls back to an analytic approximation if the draws can't be fetched.
6. **Exceedance probability** (Chart 6) — P(district > national) per student group
2. **Rate by student group** (Chart 2) — box-and-whisker per race×sex group for the most recent year, split into Female/Male panels. The box is the 25th–75th percentile of that group's real 500-draw posterior (same duckdb-wasm fetch as Chart 3), the whisker is the model's reported 90% interval, and a diamond marks the observed rate; individual boxes fall back to an analytic approximation when a group's draws can't be fetched.
3. **Predicted rates by student group** (Chart 3) — density ridges built from each group's real 500-draw posterior (fetched client-side via duckdb-wasm from the public Hugging Face parquet dataset), with a model-selector dropdown for the four Bayesian specifications and diamond markers for observed rates; falls back to an analytic approximation if the draws can't be fetched.
## Architecture
### Tech Stack
- **React 19** + **Vite** (static site generation, no backend required)
- Plain CSS custom properties for styling (matches Civilytics design tokens exactly)
- D3.js v7 for data-driven visualizations (density ridges, scales, axes)
- Inline SVG rendering — charts are built with vanilla DOM/D3, not charting libraries
- Inline SVG rendering with hand-rolled scales — no charting library, no D3 dependency
- Embeds **DuckDB-Wasm** (`@duckdb/duckdb-wasm`, ~39MB uncompressed / ~8.8MB gzipped, loaded on demand only after a district is selected) to query real posterior draws client-side from a public Hugging Face Parquet dataset
- Calls the public read-only API directly from the browser
### File Structure
@@ -41,26 +38,32 @@ Visitors select a U.S. state, search for a school district (with suggestions of
crdc-demo/
├── index.html # Entry point
├── vite.config.mjs # Vite build config
├── public/ # Static assets (wordmark, favicon)
│ └── civilytics-wordmark.svg # Civilytics wordmark from civilyticsR package
├── public/ # Static assets (wordmark, favicon, fixtures)
│ ├── civilytics-wordmark.svg # Civilytics wordmark from civilyticsR package
│ └── data/national_rates.json # Static national rates fixture for comparisons
├── src/
│ ├── main.jsx # React entry
│ ├── App.jsx # Main router (state → search → loading → charts)
│ ├── hooks/useApi.js # API client with retry/backoff + endpoint wrappers
│ ├── hooks/
│ │ ├── useApi.js # API client with retry/backoff + endpoint wrappers
│ │ └── useDrawDistribution.js # Fetches real posterior draws (duckdb-wasm + HF parquet)
│ ├── components/
│ │ ├── StateSelector.jsx # Landing screen — state dropdown/grid
│ │ ├── DistrictSearch.jsx # Search + "interesting" district suggestions
│ │ ├── LoadingAnimation.jsx # Animated histogram grid during data fetch
│ │ └── ChartPanel.jsx # Orchestrates all 6 charts + data fetching
│ │ ├── ChartLegend.jsx # Shared legend row
│ │ ├── ApproxNote.jsx # "shape estimated from interval bounds" caption
│ │ └── ChartPanel.jsx # Orchestrates all 3 charts + data fetching
│ ├── charts/
│ │ ├── ArrestsOverTime.jsx # Chart 1 — line chart by wave (SVG)
│ │ ├── RateByGroupBar.jsx # Chart 2 — bar chart by group (SVG)
│ │ ├── DistrictVsNational.jsx # Chart 3 — comparison vs. national avg
│ │ ├── ModelDrawsComparison.jsx # Chart 4 — quadrant model comparison (D3)
│ │ ├── RateDensityRidgeline.jsx # Chart 5 — density ridges per student group (D3)
│ │ └── ExceedanceProbability.jsx # Chart 6 — P(district > nat) per group
│ ├── styles/tokens.css # Civilytics design tokens (colors, fonts, spacing)
│ └── data/national_rates.json # Static national rates fixture for comparisons
│ │ ├── ArrestsOverTime.jsx # Chart 1 — counts by wave (SVG)
│ │ ├── RateByGroupBar.jsx # Chart 2 — box-and-whisker by group (SVG)
│ │ └── RateDensityRidgeline.jsx # Chart 3 — density ridges per student group (SVG)
│ ├── utils/
│ │ ├── duckdbClient.js # Lazy duckdb-wasm bundle loader (dynamic import)
│ │ ├── kde.js # Empirical density from real draws (+ kde.test.js)
│ │ ├── drawGroups.js # Draw-map key format + coverage check (+ .test.js)
│ │ └── distributionApprox.js # Analytic fallback when draws are unavailable
│ └── styles/tokens.css # Civilytics design tokens (colors, fonts, spacing)
├── Dockerfile # Multi-stage build → nginx static server
├── docker-compose.yml # Local dev / self-hosted deployment
└── .gitea/workflows/pages.yml # CI/CD — builds and deploys to pages branch
@@ -114,7 +117,7 @@ docker buildx build --platform linux/amd64 -t registry.civilytics.org/crdc-demo:
docker push registry.civilytics.org/crdc-demo:latest
```
The container is ~5MB (nginx Alpine) and serves static files with immutable cache headers. A `/healthz` endpoint supports container orchestration health checks.
The container is nginx Alpine (~5MB) plus the built site, which is dominated by the ~39MB DuckDB-Wasm engine. `nginx.conf` serves that engine gzipped (~8.8MB on the wire) with immutable cache headers, so it is fetched once per browser. A `/healthz` endpoint supports container orchestration health checks.
### Environment Variables
| Variable | Default | Description |