Model Evaluation Tools — Analysis Lab · consolidated app gallery
Eleven prototypes for exploring, visualizing, and interpreting MET-style verification
output — seven views, the STAT Ingest bridge that parses real MET output, the
MET Pipeline Explorer that recomputes MET’s verification live from real
data and checks it against MET’s own .stat, a Real-Data Demo that
streams real (de-identified) MET data from Cloudflare R2, and WebGPU FSS that runs
the Fractions Skill Score on the GPU. The first nine share the verification math
(lib/met-stats.mjs) and are dependency-free; the real-data apps stream MET output
from Cloudflare R2, and each of the first nine can still be run fully offline as a single
downloadable file. The Real-Data Demo needs the network and WebGPU FSS needs a WebGPU browser.
https://met-al-lab.pages.dev, no install and no clone. Most apps compute client-side; the
real-data apps stream MET output live from Cloudflare R2, so a network connection is expected.dist/<name>.html — download one and double-click; it opens straight from file://
with no server. (The Real-Data Demo streams from R2 and WebGPU FSS needs a WebGPU browser, so those two
want a connection / a capable GPU.)apps/<name>/index.html sources are
ES-module apps, so serve the tree over HTTP — python3 -m http.server 8000, then
http://127.0.0.1:8000/ — because browsers block file:// module imports.Documented MET verification math run entirely in the browser: live 2×2 contingency table, full categorical + continuous suites, a per-stat MET-fidelity formula table, a SQL-like query panel, and drag-drop CSV.
Direct-manipulation explorer with explicit Metric / Dimension / Filter / Facet vocabulary driving three cross-filtered linked views (brushable line plot, sortable table, faceted small multiples). Aggregates the correct ratio-of-sums way.
A gallery of four interactive verification visuals: a Performance Diagram (CSI contours + bias rays), a Reliability + Sharpness panel with Brier decomposition, a draggable threshold scrubber, and MODE-style object verification.
A cohesive 4-step diagnostic journey (Overview → Error vs lead → Categorical skill → Reliability) with a per-view provenance panel, shareable URL-hash sessions, and a deterministic rule-based narrator grounded in the on-screen numbers.
A pannable/zoomable Canvas error field over a synthetic gridded domain with forecast/obs/error layers, named region masks, per-mask continuous + categorical MET stats, hover readout, and a brush box. Shows ratio-of-sums vs mean-of-ratios on screen.
Four coherent ensemble diagnostics over one synthetic dataset — rank (Talagrand) histogram, spread-skill, probabilistic reliability + sharpness + Brier decomposition, and CRPS — all driven by a single under→calibrated→over dispersion knob.
An orbitable, tiltable 2.5D Canvas height-field rendering a verification metric over a lead × threshold × region cube, with scrubbable slicing planes, a synced 2D slice panel, hover-to-raw-counts, and a 12-metric live switcher.
.stat ingestion
The keystone parser: header-driven, version-tolerant ingestion of real MET .stat / per-type .txt / MODE output (CTC, CTS, SL1L2, CNT, PCT, PSTD, PRC, RHIST, ECNT, ORANK) feeding the shared lib. Paste or drop a real file; parsed PCT bins reproduce the paired PSTD Brier decomposition exactly. Ingested strings rendered injection-safe.
Walks the grid_stat/point_stat workflow stage by stage, flagging which run in the browser, which need an offline pre-encode (GRIB2/prepBUFR decode), and which stay server-side. The browser-computable stages run live on a real AIGFS-vs-URMA case — threshold → 2×2 contingency → partial sums → CNT/CTS scores → FSS — each checked against MET’s own .stat (counts bit-identical). Built for MET users to see exactly what’s portable to the client.
The networked app: streams real (de-identified) MET data from a private Cloudflare R2 bucket through a gated Worker. A deck.gl map animates AIGFS-forecast & URMA-analysis 2 m-T frames read chunk-by-chunk with zarrita.js (Zarr v3), a synced isobaric cross-section, and DuckDB-WASM runs SQL over the .stat Parquet (RMSE vs lead, by model) — all client-side. Needs network.
The follow-up: the Fractions Skill Score — MET’s one operator with no closed form over partial sums — computed by three real WebGPU compute kernels: a naive per-cell window (O(cells·r²)), a separable two-pass sliding window (O(cells)), and a prefix-scan integral image built with a workgroup shared-memory block scan (O(cells)). All checked live against the integral-image CPU reference (counts exact, FSS ~1e-8) and raced across grid size and neighborhood radius (~7× at 2048²; the r-sweep shows naive blowing up while the O(cells) kernels stay flat). Plus the Pf/Po fields rendered GPU-to-screen (compute→texture→canvas, no readback). Honest about the algorithm-vs-throughput tradeoff. Needs a WebGPU browser.
