fix: raise MAX_RATE_DOMAIN from 30 to 100 per 1,000 students
Deploy to git-pages / deploy (push) Successful in 19s

The previous cap of 30 was exceeded by over 60% of districts — mostly sparse
groups with small enrollment cells whose Agresti-Coull upper bounds genuinely
extend that far. Analysis across all 51 states (16,279 districts) showed the
median max x-value is already ~56 per 1,000; only truly degenerate cases like a
single predicted arrest in a four-student cell (~250/1000) need capping.

A cap of 100 still guards against these outliers while letting realistic data
drive the axis for the vast majority of districts. The existing 'clipped' flag
and note mechanism remain unchanged — they activate only when extreme values are
encountered.
This commit is contained in:
2026-08-22 20:29:32 -04:00
parent fb0beaafc8
commit 654b42ca71
4 changed files with 436 additions and 7 deletions
+17 -2
View File
@@ -10,15 +10,22 @@ import { quantile } from './kde.js'
import { niceTicks } from './niceTicks.js'
/**
* Hard ceiling on the axis, inherited from the chart this replaces.
* Hard ceiling on the axis.
*
* A four-student cell can produce posterior draws in the hundreds per 1,000;
* letting those set the axis squashes every other group into the leftmost few
* pixels. `computeRateDomain` reports `clipped` whenever the cap actually
* binds, so the chart can say so rather than cropping data off the edge in
* silence.
*
* Set to 100 per 1,000 (one arrest per ten students). The previous value of
* 30 was exceeded by over 60% of districts — mostly sparse groups with small
* enrollment cells whose Agresti–Coull upper bounds genuinely extend that far.
* A cap of 100 still guards against the most degenerate cases (e.g. a single
* predicted arrest in a four-student cell yielding ~250 per 1,000) while
* letting realistic data drive the axis for the vast majority of districts.
*/
export const MAX_RATE_DOMAIN = 30
export const MAX_RATE_DOMAIN = 100
/**
* Ignore the very top of each draw set when sizing the axis. The posterior
@@ -30,6 +37,14 @@ const DOMAIN_QUANTILE = 0.995
const HEADROOM = 1.15
/**
* Computes a shared x-domain for the rate density and over-time charts.
*
* The domain covers everything that will actually be drawn: the 0.995
* quantile of each group's posterior-predictive draw rates, plus every
* Agresti–Coull upper bound from selected groups. `niceMax` is a clean
* "nice number" ceiling (via `niceTicks`) that may exceed the raw data max by
* up to one tick step, and is capped at `MAX_RATE_DOMAIN`.
*
* @param {Array<number[]>} rateArrays - per-group draw rates, per 1,000
* @param {number[]} acUpperRates - Agresti–Coull upper bounds, per 1,000
* @returns {{ticks: number[], niceMax: number, clipped: boolean}}
+25 -3
View File
@@ -23,16 +23,27 @@ test('computeRateDomain: one extreme draw does not blow out the axis', () => {
assert.ok(domain.niceMax < 10, `niceMax was ${domain.niceMax}`)
})
test('computeRateDomain: caps the domain and reports the clip', () => {
test('computeRateDomain: caps at MAX_RATE_DOMAIN=100 and reports the clip', () => {
// The cap keeps a tiny-denominator group from flattening every other curve,
// but the caller has to be able to say so on screen rather than silently
// cropping data off the right edge.
const wide = new Array(500).fill(0).map((_, i) => (i < 490 ? 60 : 0))
const domain = computeRateDomain([wide], [80])
const wide = new Array(500).fill(0).map((_, i) => (i < 490 ? 200 : 0))
const domain = computeRateDomain([wide], [300])
assert.equal(domain.niceMax, MAX_RATE_DOMAIN)
assert.equal(domain.clipped, true)
})
test('computeRateDomain: realistic sparse district data is not clipped', () => {
// Carson City NV (LEAID 3200390): pooled BL cell with 34 students and 0
// observed arrests → AC upper bound ≈ 88 per 1,000. This should NOT be
// clipped under the new cap of 100.
const blRates = new Array(500).fill(0)
const domain = computeRateDomain([blRates], [88])
assert.ok(domain.niceMax >= 88, `niceMax ${domain.niceMax} should cover AC bound of 88`)
assert.equal(domain.clipped, false)
})
test('computeRateDomain: never returns a zero-width domain', () => {
const domain = computeRateDomain([[0, 0, 0]], [0])
assert.ok(domain.niceMax > 0)
@@ -49,3 +60,14 @@ test('computeRateDomain: ignores empty rate arrays', () => {
const domain = computeRateDomain([[], [1, 2, 3]], [])
assert.ok(domain.niceMax > 0 && domain.niceMax < MAX_RATE_DOMAIN)
})
test('computeRateDomain: degenerate tiny-denominator cell is clipped', () => {
// A four-student cell with a single predicted arrest yields ~250 per
// 1,000 — this should be capped at MAX_RATE_DOMAIN=100 and flagged as
// clipped so the chart can warn the reader.
const tinyRates = new Array(500).fill(0)
for (let i = 0; i < 250; i++) tinyRates[i] = 250
const domain = computeRateDomain([tinyRates], [300])
assert.equal(domain.niceMax, MAX_RATE_DOMAIN)
assert.equal(domain.clipped, true)
})