# MEME-outcome gate — ML deliverable (XGBoost)

Scope: this is the **ML team's** deliverable — the trained model + how it was made + its operating
characteristics. Browser runtime / DM3 integration / versioning-in-app is the dev team's; not here.

## The artifact
- **`meme_gate.json`** — the model, in XGBoost's own native format (`booster.save_model`). This is the
  single artifact. No pkl, no ONNX, no converter — the dev team's runtime reads this JSON directly.
- `meme_gate.meta.json` — provenance sidecar (features, label, hyperparams, AUC). Not needed at runtime.

## What it predicts
P(MEME reports >=1 site at p<=0.1) from three features, in this exact order:
`[num_seqs, num_sites, median_pos_dist]`. Apply the same prep as training before scoring:
median_pos_dist floored 0.001, capped 10.0. Output is a probability (binary:logistic).

## Model
`XGBClassifier(max_depth=6, n_estimators=500, learning_rate=0.05, min_child_weight=1,
monotone_constraints=(1,1,1), subsample=0.9)`.
- **Monotone by construction** in all three features (native `monotone_constraints`) — verified 0
  violations. Adding sequences / codons / depth can only raise the score. (Closes veg/datamonkey3#151a.)
- `frac_p_defined` dropped — it was a post-hoc, inert feature (#151b).
- **AUC: 0.914 held-out test / 0.87 full /data corpus** (matches the earlier HistGBM 0.891 and beats
  untuned XGBoost 0.8785 — the XGBoost switch is not a downgrade).

## Operating characteristics (for the dev team to write UI copy against)
- `calibration_report.json` — held-out production test split.
- `calibration_full_universe.json` — ALL /data MEME jobs (8,981), the definitive numbers.

3-band structure (cuts 0.30 / 0.70), full /data corpus, observed MEME-hit rates:
| Band | Score | Observed hit rate | Fires on |
|---|---|---|---|
| Unlikely | <0.30 | 26% | 5.2% |
| Uncertain | 0.30-0.70 | 53% | 11.2% |
| Likely | >=0.70 | 93% | 83.6% |
At an 85% base rate, "unlikely" bottoms out at ~26% — honest copy is "often finds little", not "won't".
(The band/copy decision is the dev+product team's; these are the true rates to write against.)

## Reproduce
- `train_gate_xgb.py`  — trains meme_gate.json from meme_validation.tsv (3,000 production MEME jobs).
- `validate_full.py`   — scores ALL /data MEME jobs (extracts p<=0.1 labels from the MEME JSONs).
- `validate_full_xgb.py` — full-corpus calibration table + bands on the XGBoost model.
Run: `python train_gate_xgb.py` then `python validate_full_xgb.py`.

Raw per-job data (meme_validation.tsv, full_validation.tsv) is real user submissions — kept local, not
shared. See ../dm3-gate-validation on silverback.

## Note for the dev team
XGBoost's JSON is a stable first-party format; the browser reads the bytes exported here with no
conversion step. Please version the filename on each retrain (e.g. meme_gate_YYYYMMDD.json) so a new
model can't silently overwrite an old one under the same name.