# MEME-outcome gate — package for the DM3 team (XGBoost)

Retrain + full validation of the pre-run MEME badge. Fixes the calibration issue (badge said
"unlikely" on alignments where MEME found a site 72% of the time) + veg/datamonkey3#151
(non-monotonicity, dead feature). Model is XGBoost — ships its OWN native JSON, no converter.

## START HERE
1. README-ML.md                  — the ML deliverable: model, how to retrain, operating characteristics.
2. SUMMARY-what-is-possible.md   — results + the product/band decision.
3. MODEL-and-relationship-to-axomeme.md — the gate vs the AxoMEME neural net.

## SHIP ARTIFACT
- meme_gate.json      — the model, XGBoost native format (booster.save_model). SINGLE artifact; the
                        browser runtime reads these bytes directly (no ONNX, no pkl, no converter).
                        Features [num_seqs, num_sites, median_pos_dist]; prep: median_pos_dist floored
                        0.001 / capped 10; output = P(MEME reports a site at p<=0.1).
- meme_gate.meta.json — provenance sidecar (features, hyperparams, AUC). Not needed at runtime.
  ** Bands (cuts 0.30 / 0.70): Unlikely 26% / Uncertain 53% / Likely 93% — see calibration tables. **

## NUMBERS (for true UI copy)
- calibration_full_universe.json — observed MEME-hit rate per threshold, ALL /data jobs (8,981) + bands.
- calibration_report.json        — same, held-out production test split.
- GATE-V2-REPORT.md              — technical write-up.

## REPRODUCE (ML side)
- train_gate_xgb.py     — trains meme_gate.json (XGBClassifier, monotone_constraints=(1,1,1)).
- validate_full.py      — scores ALL /data MEME jobs (extracts p<=0.1 labels).
- validate_full_xgb.py  — full-corpus calibration table + bands.
  (Raw per-job data is real user submissions — OFF this share. See DATA-NOTE.txt.)

## MONOTONICITY (#151)
Native XGBoost monotone_constraints=(1,1,1): 0 violations, guaranteed by construction. frac_p_defined
dropped (#151b). No custom converter to keep in sync — that was the root cause of the earlier churn.

## DEV NOTE
Version the filename on each retrain (meme_gate_YYYYMMDD.json) — a prior package overwrote an earlier
model under an identical name with a different family inside. XGBoost's stable JSON makes drift avoidable.