Model Evaluation Tools — Analysis Lab · consolidated app gallery
Eleven prototypes for exploring, visualizing, and interpreting MET-style verification
output — seven views, the STAT Ingest bridge that parses real MET output, the
MET Pipeline Explorer that recomputes MET’s verification live from real
data and checks it against MET’s own .stat, a Real-Data Demo that
streams real (de-identified) MET data from Cloudflare R2, and WebGPU FSS that runs
the Fractions Skill Score on the GPU. The first nine share the verification math
(lib/met-stats.mjs), are dependency-free, and work fully offline; the
Real-Data Demo needs the network and WebGPU FSS needs a WebGPU browser.
dist/<name>.html — it is fully inlined and opens straight from file://, no server, no network.apps/<name>/index.html versions are ES-module apps and must be served over HTTP (browsers block file:// module imports). From the worktree root run
python3 -m http.server 8000 then open http://127.0.0.1:8000/apps/<name>/index.html (or this gallery at http://127.0.0.1:8000/index.html).Documented MET verification math run entirely in the browser: live 2×2 contingency table, full categorical + continuous suites, a per-stat MET-fidelity formula table, a SQL-like query panel, and drag-drop CSV.
Direct-manipulation explorer with explicit Metric / Dimension / Filter / Facet vocabulary driving three cross-filtered linked views (brushable line plot, sortable table, faceted small multiples). Aggregates the correct ratio-of-sums way.
A gallery of four interactive verification visuals: a Performance Diagram (CSI contours + bias rays), a Reliability + Sharpness panel with Brier decomposition, a draggable threshold scrubber, and MODE-style object verification.
A cohesive 4-step diagnostic journey (Overview → Error vs lead → Categorical skill → Reliability) with a per-view provenance panel, shareable URL-hash sessions, and a deterministic rule-based narrator grounded in the on-screen numbers.
A pannable/zoomable Canvas error field over a synthetic gridded domain with forecast/obs/error layers, named region masks, per-mask continuous + categorical MET stats, hover readout, and a brush box. Shows ratio-of-sums vs mean-of-ratios on screen.
Four coherent ensemble diagnostics over one synthetic dataset — rank (Talagrand) histogram, spread-skill, probabilistic reliability + sharpness + Brier decomposition, and CRPS — all driven by a single under→calibrated→over dispersion knob.
An orbitable, tiltable 2.5D Canvas height-field rendering a verification metric over a lead × threshold × region cube, with scrubbable slicing planes, a synced 2D slice panel, hover-to-raw-counts, and a 12-metric live switcher.
.stat ingestion
The keystone parser: header-driven, version-tolerant ingestion of real MET .stat / per-type .txt / MODE output (CTC, CTS, SL1L2, CNT, PCT, PSTD, PRC, RHIST, ECNT, ORANK) feeding the shared lib. Paste or drop a real file; parsed PCT bins reproduce the paired PSTD Brier decomposition exactly. Ingested strings rendered injection-safe.
Walks the grid_stat/point_stat workflow stage by stage, flagging which run in the browser, which need an offline pre-encode (GRIB2/prepBUFR decode), and which stay server-side. The browser-computable stages run live on a real AIGFS-vs-URMA case — threshold → 2×2 contingency → partial sums → CNT/CTS scores → FSS — each checked against MET’s own .stat (counts bit-identical). Built for MET users to see exactly what’s portable to the client.
The networked app: streams real (de-identified) MET data from a private Cloudflare R2 bucket through a gated Worker. A deck.gl map animates AIGFS-forecast & URMA-analysis 2 m-T frames read chunk-by-chunk with zarrita.js (Zarr v3), a synced isobaric cross-section, and DuckDB-WASM runs SQL over the .stat Parquet (RMSE vs lead, by model) — all client-side. Needs network.
The follow-up: the Fractions Skill Score — MET’s one operator with no closed form over partial sums — computed by two real WebGPU compute shaders: a naive per-cell window (O(cells·r²)) and a separable two-pass sliding window (O(cells)). Both checked live against the integral-image CPU reference (counts exact, FSS ~1e-8), raced across grid size and neighborhood radius (~7× at 2048²; the r-sweep shows naive blowing up while separable stays flat), plus a Pf/Po field view that makes the neighborhood smoothing visible. Honest about the algorithm-vs-throughput tradeoff. Needs a WebGPU browser.