crdc-demo/README.md

CRDC Arrests API Demo

A demonstration web application for the CRDC School Arrest Rate API, showing Bayesian model comparisons of school-based arrest rates across U.S. districts. Built with React + Vite, deployable as a static site or Docker container.

Quick Start (Development)

# Install dependencies
npm ci

# Start dev server on localhost:5173
npm run dev

# Build for production
npm run build      # outputs to dist/
npm run preview    # serve built files locally

What It Does

Visitors select a U.S. state, search for a school district (with suggestions of districts that have the most arrests), and see 3 charts comparing observed data against Bayesian model estimates:

1. Arrests over time (Chart 1) — raw counts by CRDC wave with per-1k rate labels, built with inline SVG
2. Rate by student group (Chart 2) — box-and-whisker per race×sex group for the most recent year, split into Female/Male panels. The box is the 25th–75th percentile of that group's real 500-draw posterior (same duckdb-wasm fetch as Chart 3), the whisker is the model's reported 90% interval, and a diamond marks the observed rate; individual boxes fall back to an analytic approximation when a group's draws can't be fetched.
3. Predicted rates by student group (Chart 3) — density ridges built from each group's real 500-draw posterior (fetched client-side via duckdb-wasm from the public Hugging Face parquet dataset), with a model-selector dropdown for the four Bayesian specifications and diamond markers for observed rates; falls back to an analytic approximation if the draws can't be fetched.

Architecture

Tech Stack

• React 19 + Vite (static site generation, no backend required)
• Plain CSS custom properties for styling (matches Civilytics design tokens exactly)
• Inline SVG rendering with hand-rolled scales — no charting library, no D3 dependency
• Embeds DuckDB-Wasm (@duckdb/duckdb-wasm, ~39MB uncompressed / ~8.8MB gzipped, loaded on demand only after a district is selected) to query real posterior draws client-side from a public Hugging Face Parquet dataset
• Calls the public read-only API directly from the browser

File Structure

crdc-demo/
├── index.html                    # Entry point
├── vite.config.mjs               # Vite build config
├── public/                       # Static assets (wordmark, favicon, fixtures)
│   ├── civilytics-wordmark.svg  # Civilytics wordmark from civilyticsR package
│   └── data/national_rates.json # Static national rates fixture for comparisons
├── src/
│   ├── main.jsx                  # React entry
│   ├── App.jsx                   # Main router (state → search → loading → charts)
│   ├── hooks/
│   │   ├── useApi.js             # API client with retry/backoff + endpoint wrappers
│   │   └── useDrawDistribution.js # Fetches real posterior draws (duckdb-wasm + HF parquet)
│   ├── components/
│   │   ├── StateSelector.jsx     # Landing screen — state dropdown/grid
│   │   ├── DistrictSearch.jsx    # Search + "interesting" district suggestions
│   │   ├── LoadingAnimation.jsx  # Animated histogram grid during data fetch
│   │   ├── ChartLegend.jsx       # Shared legend row
│   │   ├── ApproxNote.jsx        # "shape estimated from interval bounds" caption
│   │   └── ChartPanel.jsx        # Orchestrates all 3 charts + data fetching
│   ├── charts/
│   │   ├── ArrestsOverTime.jsx          # Chart 1 — counts by wave (SVG)
│   │   ├── RateByGroupBar.jsx            # Chart 2 — box-and-whisker by group (SVG)
│   │   └── RateDensityRidgeline.jsx      # Chart 3 — density ridges per student group (SVG)
│   ├── utils/
│   │   ├── duckdbClient.js       # Lazy duckdb-wasm bundle loader (dynamic import)
│   │   ├── kde.js                # Empirical density from real draws (+ kde.test.js)
│   │   ├── drawGroups.js         # Draw-map key format + coverage check (+ .test.js)
│   │   └── distributionApprox.js # Analytic fallback when draws are unavailable
│   └── styles/tokens.css          # Civilytics design tokens (colors, fonts, spacing)
├── Dockerfile                    # Multi-stage build → nginx static server
├── docker-compose.yml            # Local dev / self-hosted deployment
└── .gitea/workflows/pages.yml   # CI/CD — builds and deploys to pages branch

API Endpoints Used

Endpoint	Purpose	Frequency
/api/v1/models	List available Bayesian model specs	Once (cached)
/api/v1/districts?q=&state=	District name/geo lookup → LEAID	On keystroke
/api/v1/estimates/{leaid}?model=X&year=Y	Estimates for one district/model/year/group	~40 calls per district
/api/v1/draws?...	Locate raw-posterior Parquet shard	Not called from app — the app fetches shards directly from Hugging Face via duckdb-wasm; see src/hooks/useDrawDistribution.js
/data/national_rates.json	Static national rates fixture (committed)	Once per session

Deployment

Option A: Git Pages via Gitea Actions (recommended)

The app is a static site deployed automatically via Gitea Actions. On every push to main, the workflow in .gitea/workflows/pages.yml builds the Vite app and copies dist/ to a pages branch served by the git-pages webhook receiver.

URL: https://pages.civilytics.org/crdc-demo/

CORS note: The CRDC API (crdc-api.civilytics.org) does not send CORS headers. When deployed as a static site, browser-based fetch requests will be blocked by the same-origin policy. To fix this, deploy proxy.php to your git-pages host and set the VITE_PROXY_URL environment variable (e.g., /crdc-demo/proxy.php). The app automatically detects proxy availability — when unset, it attempts direct API calls (works via Docker/nginx or same-origin setups).

The workflow follows the same pattern as Civilytics/sln-school-comparison:

1. Triggers on push to main (only when source files change)
2. Builds with Node 22 + Vite (npm ci && npm run build)
3. Copies dist/ contents to an orphan pages branch
4. Forces push — the git-pages webhook picks it up automatically

Note

: Ensure ${{ secrets.GITHUB_TOKEN }} is configured in repo settings on gitea.civilytics.org.

Option A.5: CORS Proxy for Git Pages

If you can't deploy PHP, create a simple reverse proxy in nginx or use an edge function that:

1. Accepts ?target=/api/v1/... as a query parameter
2. Forwards the request to `https://crdc-api.civilytics.org/api/v1/...
3. Returns the response with Access-Control-Allow-Origin: *
4. Set VITE_PROXY_URL at build time to point at this endpoint.

Option B: Docker (self-hosted fleet) — includes CORS proxy nginx config:

docker compose up -d    # builds + serves on localhost:8080

# Or build manually for your registry:
docker buildx build --platform linux/amd64 -t registry.civilytics.org/crdc-demo:latest .
docker push registry.civilytics.org/crdc-demo:latest

The container is nginx Alpine (~5MB) plus the built site, which is dominated by the ~39MB DuckDB-Wasm engine. nginx.conf serves that engine gzipped (~8.8MB on the wire) with immutable cache headers, so it is fetched once per browser. A /healthz endpoint supports container orchestration health checks.

Environment Variables

Variable	Default	Description
VITE_API_BASE	https://crdc-api.civilytics.org/api/v1	Override to point at a staging API (set in Dockerfile build or docker-compose)

Visual Design

This app follows the Civilytics visual identity as defined in:

• theme/_tokens.scss — colors, typography, spacing tokens
• social_media_posts.md — chart patterns from published social media figures
• white_paper.qmd / R/paper_figures.R — white paper figure builders

Key design decisions:

• Color palette: Warm paper background (#FAF7F2), civic navy text (#0E1A2B), ember accent (#C25311)
• Data viz colors: Teal, plum, moss, brass from the supporting palette
• Fonts: Libre Franklin/Inter for display and body, JetBrains Mono for code
• Chart patterns: Pointrange comparisons (model vs. observed), ridgeline-style density bars, faceted small multiples

Citation & Attribution

The CRDC School Arrest Rate API: Knowles, J.E., & Miller, H. (2025). CRDC School Arrest Rate API v1. Civilytics Consulting. https://crdc-api.civilytics.org/api/v1/

Data source: US Department of Education, Office for Civil Rights. Civil Rights Data Collection (2021-22).

This research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation under NSF award NSF-DRL #1749275. Opinions reflect those of the author and do not necessarily reflect those AERA or NSF.