# Agent Guide — CRDC Demo App

This document captures context, decisions, and guidance for agents working on this codebase. It is the primary source of truth for how to make changes safely.

## Project Overview

A React + Vite static web app demonstrating the [CRDC School Arrest Rate API](https://crdc-api.civilytics.org/api/v1/). Visitors select a state, search for a school district, and see 6 charts comparing observed arrest data against Bayesian model estimates. Deployed via Gitea Actions to `pages.civilytics.org/crdc-demo/`.

## Architecture Summary

- **Frontend**: React 19 + Vite (static site generation)
- **Styling**: Plain CSS custom properties matching Civilytics design tokens (`src/styles/tokens.css`)
- **Charts**: Mixed approach:
  - Charts 1–3 use inline SVG with manual scales (no D3 dependency for these)
  - Chart 4 (ModelDrawsComparison) uses D3.js v7 for data-driven rendering of quadrant comparisons
  - Chart 5 (RateDensityRidgeline) uses D3.js v7 for density ridge visualizations
- **API**: Calls public read-only API directly from browser; no backend required
- **Deployment**: Static site deployed via Gitea Actions (`.gitea/workflows/pages.yml`)

## Key Components and Data Flow

```
App.jsx (router)
  → StateSelector (landing screen: state dropdown/grid)
    → DistrictSearch (search + "interesting" suggestions from /estimates?state=&year=)
      → LoadingAnimation (fetches all data in parallel, shows animated histogram grid)
        → ChartPanel (receives district object, fetches structured estimates for 6 charts)
          ├── ArrestsOverTime       — SVG line chart by wave (3 years)
          ├── RateByGroupBar         — SVG bar chart: observed vs modeled per group
          ├── DistrictVsNational     — SVG comparison to national average
          ├── ModelDrawsComparison    — D3 quadrant charts (4 model types × 1 year)
          ├── RateDensityRidgeline   — D3 density ridges per race×sex group
          └── ExceedanceProbability   — P(district > national) per student group
```

### Data Fetching Strategy (`ChartPanel.jsx`)

`ChartPanel` fetches all chart data on mount (after `LoadingAnimation` pre-fetched via batch calls):

1. **Wave data** for Charts 1–3: Fetches `unified_m3_mod` model estimates for years `['21-22', '17-18', '15-16']`.
   - Uses three-year models because they return observed arrest counts across all waves (one-year models only have data for the most recent wave).

2. **Quad data** for Charts 4–6: Fetches estimates from all four quadrant models (`unified_m1_mod`, `unified_m2_mod`, `unified_m3_mod`, `unified_m4_mod`) for year `21-22` only (one year, as the most recent wave).

3. **National rates**: Loaded once from a static JSON fixture or cached by `LoadingAnimation`.

### Error Bar Convention: 90% Intervals

The API returns 95% HPD intervals (`count_lower`, `count_upper`). However, all chart labels and calculations in this app use **90% intervals**.

The 90% convention is baked into the analytic fallback in `src/utils/distributionApprox.js` (the path used when real posterior draws can't be fetched — see the duckdb-wasm section below). That fallback generates **no draws**: `fitSkewedInterval` fits a two-piece normal directly from `{median, lower, upper}`, converting each half-interval to its own sigma with `z = probit((1 + intervalMass) / 2)` and `intervalMass` defaulting to `0.90` (so z ≈ 1.645). For a symmetric interval that is equivalent to the old fixed "full width / 3.29" divisor, but it is computed from `intervalMass` rather than hardcoded, and each side gets its own sigma so the fitted shape stays skewed.

If you change this convention, update:
- `distributionApprox.js` — the `intervalMass = 0.90` default in `fitSkewedInterval`, and its JSDoc claim that the API's bounds are a 90% interval
- `RateByGroupBar.jsx` — the caption and component JSDoc, both of which say "the model's reported 90% interval"
- `ArrestsOverTime.jsx` — the legend label "Modeled (median + 90% interval)"
- Any documentation referencing confidence/credible intervals

## Common Pitfalls & Gotchas

### 1. Null Safety in Chart Components

Several API responses may return empty arrays or missing fields for districts with no arrests:
```javascript
// Always use optional chaining and defaults:
const yearRow = (quadModels[q.key] || []).find(r => r.year === year)
const predMedian = yearRow?.count_median || 0
const enroll = row.stu_enroll || 1  // Prevent division by zero
```

### 2. D3 useEffect Dependency Arrays

When using `useEffect` for D3 rendering, always include all data dependencies to prevent stale renders:
```javascript
// Correct — includes all props used inside the effect
}, [quadData, selectedModel, rateByGroup])
```

### 3. SVG Dimensions and Responsiveness

Charts use fixed dimensions with responsive containers (`overflowX: 'auto'` for wide content):
- Chart cards have `max-width: 70rem` in `ChartPanel.jsx` (vs the default text width of ~60rem)
- SVG elements should set both `width="100%"` and a fixed `viewBox` for proper scaling

### 4. API Endpoint Availability

Not all endpoints are available to browser-based clients:
- `/api/v1/estimates/{leaid}` — ✅ Returns estimates summary (median, lower, upper bounds)
- `/api/v1/draws?...` — Returns a Parquet shard URL + DuckDB SQL, not draw data itself. The app **does** use the real draws in the shard it points to — see below.

### 5. Real posterior draws via duckdb-wasm

Charts 2 (`RateByGroupBar`) and 3 (`RateDensityRidgeline`) fetch the actual
500-draw-per-group posterior from the public Hugging Face parquet dataset
(`civilytics/crdc-school-arrest-rates`), queried client-side with
`@duckdb/duckdb-wasm` (`src/utils/duckdbClient.js` +
`src/hooks/useDrawDistribution.js`). No server-side draws endpoint is
involved. If that fetch fails (network, unsupported browser, HF outage),
both charts fall back to the `distributionApprox.js` analytic approximation
and show the "estimated shape" note — **do not delete `distributionApprox.js`
or `ApproxNote.jsx`**, they're the fallback path, not dead code.

See `docs/superpowers/specs/2026-08-11-empirical-draws-wasm-design.md` for
the full design.

The duckdb-wasm engine itself is ~39MB uncompressed / ~8.86MB gzipped (confirmed against the shipped `@duckdb/duckdb-wasm` package, not the design spec's original ~3-5MB estimate, which was wrong). It's loaded via dynamic `import()` only once a district is selected — never on initial page load — and cached by the browser thereafter, but it's a real one-time cost worth knowing about before touching this code path.

### 6. CORS Configuration

The CRDC API does not send CORS headers. When deployed to git-pages (static hosting), requests are blocked by same-origin policy unless a proxy is configured:
- The app auto-detects proxy availability via `VITE_PROXY_URL` environment variable
- If unset, the app attempts direct fetch — works when served from Docker/nginx or same-origin

## Deployment Checklist

Before pushing to production:

1. **Build succeeds**: `npm run build` (check for new errors)2. **No console errors in browser** after hard refresh3. **All 6 charts render** with sample districts (test "Denver", "Mobile County")
4. **Loading animation** appears briefly, then transitions to ChartPanel

### Commit Message Convention

Use descriptive commit messages that explain the *why*, not just the what:
```bash
# Good
git commit -m "Fix: use three-year model for wave data (returns observed counts across all years)"
git commit -m "Add D3 density ridges to Chart 5 with diamond markers for observed rates"

# Avoid vague messages
git commit -m "Fix charts"          # Too generic
git commit -m "Update code"         # No context
```

## Testing Strategy

There are no automated tests in this project. Manual verification is required:1. **Visual check**: Load a district and verify all 6 charts render correctly2. **Error console**: Check browser DevTools for JavaScript errors3. **Data accuracy**: Compare observed values against API response (check Network tab)
4. **Responsiveness**: Resize window to ensure layout adapts

## Style Guide References

- R code style: Follows tidyverse principles (`r-style-guide` skill in agent knowledge base)- Chart aesthetic decisions should match patterns from `social_media_posts.md` and `white_paper.qmd`
- Colors, typography, spacing are defined as CSS custom properties in `src/styles/tokens.css`

## Related Repositories

- **crdc-arrests** — The API server (Plumber/R) at `/home/jared/Nextcloud/Civilytics/Code/Civilytics/crdc-arrests/`
- **civilyticsR** — R package with wordmark and visualization functions at `/home/jared/Nextcloud/Civilytics/Code/Civilytics/civilyticsR/`

## Git Conventions

- Remote: `https://gitea.civilytics.org/Civilytics/crdc-demo.git`
- Default branch: `main` (not `master`)
- Gitea Actions workflow auto-deploys on push to `main` via `.gitea/workflows/pages.yml`
- Always pull before making changes: `git pull origin main`