# Phase 4 rankings and robustness methodology

Model: `ranking/v1.0.0` (metrics `metrics/v1.0.0`, pressure `pressure/v1.0.0`,
canonical `canon/v1`). Scope: IPL primary, T20I corroborative, combined view
labelled as a sensitivity check. Status: frozen for Phase 4 review. The
dashboard remains blocked until the Phase 5 approval gate.

## 1. What is ranked

Two lists are published:

1. **Top 10 batters** by reliability-adjusted pressure batting rating.
2. **Top 10 bowlers** by reliability-adjusted pressure bowling rating.

The rating answers one question: how much better than the player's own
context-adjusted baseline does the player perform when the pre-delivery
pressure state is high? It is not a total batting or bowling quality metric,
and it is not a projection of future runs or wickets.

Pressure states come from `pressure/v1.0.0`: the high-pressure label uses the
frozen threshold of 70 on the pre-delivery pressure index. Baselines and deltas
come from `metrics/v1.0.0`; the ranking layer only aggregates and standardises
those deltas.

## 2. Frozen configuration

The single source of truth is `configs/models/ranking_v1.toml`.

| Choice | Frozen value |
|---|---|
| Metric direction | Higher is always better (direction-adjusted deltas) |
| Winsorising | 2.5% / 97.5% within the eligible pool |
| Standardisation | Median / scaled MAD, scale factor 1.4826 |
| z clamp | -3 to +3 |
| Reliability prior | 150 pressure balls (both disciplines) |
| Battery eligibility | 30 pressure balls |
| Bowler eligibility | 36 pressure legal balls |
| Bootstrap | Match-level, 1000 replicates, seed 20261110, 95% |

Metric weights:

| Batting metric | Weight | Bowling metric | Weight |
|---|---:|---|---:|
| Strike-rate delta | 0.45 | Economy delta | 0.40 |
| Dot-ball avoidance delta | 0.15 | Wicket-rate delta | 0.20 |
| Boundary-rate delta | 0.20 | Dot-ball delta | 0.10 |
| Dismissal-risk control delta | 0.20 | Boundary-suppression delta | 0.20 |
| | | Extras-control delta | 0.10 |

Weights were fixed before inspecting the final player names. They express a
cricket judgement: the primary run-scoring or run-prevention change carries
most weight, with secondary skills splitting the remainder. The sensitivity
run tests equal, primary-heavy and secondary-heavy alternatives.

## 3. Data pipeline and parity

Rust is authoritative (ADR-0001). `scout rank` rebuilds pressure states,
metrics and rankings from the canonical tables, then writes:

- `data/rust/rankings/rankings_{batting,bowling}_{ipl,t20i,combined}.csv`
- `data/rust/rankings/top10_{batting,bowling}_{ipl,t20i,combined}.csv`
- `data/rust/rankings/ranking_match_{batting,bowling}.csv` (match components)
- `data/rust/rankings/rankings_diagnostics.json` (transform bounds per scope)
- `data/rust/rankings/rankings_run_manifest.json` (checksums and model ids)

The Python research layer re-scores the exported match components with the same
frozen parameters. Across all six discipline/scope pairs the maximum absolute
score difference is **5.3e-10** and there are **zero** rank mismatches, inside
the `1e-9` float tolerance. A clean baseline re-run also reproduces the
published scores exactly (`score_max_abs_diff = 0`).

## 4. Metric definitions

Each metric is a pressure-vs-baseline delta. Positive is always better.

**Batting**

- `primary`: pressure strike rate minus the player's shrunk expected strike
  rate in the same scope, innings type and role bucket.
- `dot`: expected dot rate minus the observed pressure dot rate.
- `boundary`: observed pressure boundary rate minus expected.
- `dismissal`: expected dismissal rate minus the observed pressure dismissal
  rate (survival is positive).

**Bowling**

- `primary`: expected economy minus observed pressure economy.
- `wicket`: observed bowler-credited wicket rate minus expected.
- `dot`: observed dot rate minus expected.
- `boundary`: expected boundary rate minus observed (suppression is positive).
- `extras`: expected extras rate minus observed.

The expected value blends the player's own baseline with the stratified context
baseline using a 300-ball prior (frozen in `metrics_v1.toml`). That prior is a
*metric* shrinkage, distinct from the 150-ball *ranking* reliability prior.

## 5. Standardisation

For every metric inside the eligible pool of a scope:

1. Winsorise at the 2.5th and 97.5th percentiles.
2. Compute the median and the scaled median absolute deviation
   (`1.4826 * MAD`).
3. Fall back to the sample standard deviation if MAD is numerically zero, then
   to a `1e-9` floor.
4. Compute a robust z-score and clamp it to ±3.

This keeps a single extreme innings from setting the scale and prevents one
metric from dominating through an unbounded tail. The transform bounds for each
scope are recorded in `rankings_diagnostics.json`. IPL bounds are never reused
for T20I or the combined view.

## 6. Reliability and eligibility

The composite is shrunk by

`reliability = balls / (balls + 150)`

so a player with 30 balls keeps only 17% of the raw composite, 150 balls keeps
50%, and 300 balls keeps 67%. The final rating is

`score = reliability × Σ(weight × z)`.

Eligibility is applied before ranking: batters need 30 pressure balls, bowlers
need 36 pressure legal balls in the scope being ranked. The pools are:

| Scope | Eligible batters | Eligible bowlers |
|---|---:|---:|
| IPL (primary) | 100 | 85 |
| T20I (corroborative) | 155 | 123 |
| Combined (sensitivity) | 235 | 177 |

Players below the gate are retained in the full table with `eligible = false`
and no rank. They are never inserted to fill a top ten.

## 7. Ranking and tie-breaks

Eligible players are sorted by the reliability-adjusted score. Ties are broken,
in order, by reliability, distinct pressure matches, share of pressure matches
above baseline, pressure balls and player id. The last key is a stable
identifier only and has no cricket meaning. Ranks are contiguous from 1 to the
eligible pool size.

## 8. Evidence labels

The operational rule is fixed in `ranking_v1.toml`:

- **High**: at least 12 distinct pressure matches and reliability at least 0.50.
- **Moderate**: at least 6 matches and reliability at least 0.30.
- **Limited**: anything below those gates.

These labels describe sample adequacy, not the width of the bootstrap interval.
The interval is reported separately so that a high-evidence player with a wide
interval is not mistaken for a precise estimate.

## 9. Published top ten (IPL)

**Batting**

| Rank | Player | Score | Pressure balls | Matches | Evidence |
|---:|---|---:|---:|---:|---|
| 1 | Rinku Singh | 0.604 | 200 | 14 | high |
| 2 | Tim David | 0.518 | 184 | 19 | high |
| 3 | Dinesh Karthik | 0.445 | 116 | 13 | moderate |
| 4 | Shubham Dubey | 0.371 | 49 | 7 | limited |
| 5 | Ashutosh Sharma | 0.368 | 136 | 16 | moderate |
| 6 | Sherfane Rutherford | 0.338 | 66 | 6 | moderate |
| 7 | David Miller | 0.332 | 150 | 21 | high |
| 8 | Vipraj Nigam | 0.312 | 59 | 7 | limited |
| 9 | Sanju Samson | 0.311 | 75 | 5 | limited |
| 10 | Ravindra Jadeja | 0.296 | 210 | 22 | high |

**Bowling**

| Rank | Player | Score | Pressure balls | Matches | Evidence |
|---:|---|---:|---:|---:|---|
| 1 | Sunil Narine | 0.576 | 134 | 18 | moderate |
| 2 | Mohit Sharma | 0.454 | 164 | 14 | high |
| 3 | Andre Russell | 0.414 | 98 | 14 | moderate |
| 4 | Prasidh Krishna | 0.403 | 148 | 19 | moderate |
| 5 | Matheesha Pathirana | 0.380 | 228 | 15 | high |
| 6 | Harshit Rana | 0.367 | 78 | 12 | moderate |
| 7 | T Natarajan | 0.347 | 166 | 17 | high |
| 8 | Umran Malik | 0.341 | 47 | 6 | limited |
| 9 | Mustafizur Rahman | 0.341 | 126 | 11 | moderate |
| 10 | Yash Dayal | 0.321 | 97 | 16 | moderate |

The T20I list is corroborative, not a second selector. Only one name
(Mustafizur Rahman) appears in both IPL and T20I top tens, which is expected:
the two competitions have different samples, roles and baseline contexts. The
combined list is reported only as a labelled sensitivity view and never as the
published order.

## 10. Uncertainty: match-level bootstrap

The bootstrap resamples **matches**, not deliveries, and recomputes the entire
pipeline in each replicate: player aggregation, winsorising, robust
standardisation, weighting, reliability shrink and ranking. The seed is
20261110 with 1000 replicates.

The published top ten all carry wide rank intervals: the 95% rank interval
reaches beyond 20th place for every top-ten player, and P(top 10) ranges from
0.30 to 0.86. Under the pre-declared stability rule (interval width ≤ 5 and
P(top 10) ≥ 0.8) every top-ten card is labelled **volatile**; the rest of the
pool is almost entirely volatile or moderate. This is an honest statement about
the sample, not a reason to reorder the frozen list. The point order is the
headline; the interval is the uncertainty statement.

The highest P(top 10) values belong to Rinku Singh (0.86), Sunil Narine (0.86)
and Tim David (0.76). The lowest in the top ten are Yash Dayal (0.30) and
Ravindra Jadeja (0.32), whose places are the most contestable.

## 11. Robustness checks

All variants were run through the Rust engine, not reimplemented in Python.
Results are in `data/rust/research/rank_sensitivity.csv` and
`reports/research/phase4_sensitivity.json`.

### 11.1 High-pressure threshold

| Threshold | Batting top-10 overlap | Batting Spearman | Bowling overlap | Bowling Spearman |
|---:|---:|---:|---:|---:|
| 55 | 3/10 | 0.686 | 3/10 | 0.674 |
| 60 | 4/10 | 0.762 | 3/10 | 0.738 |
| 65 | 5/10 | 0.854 | 6/10 | 0.856 |
| **70 (frozen)** | **10/10** | **1.000** | **10/10** | **1.000** |
| 75 | 5/10 | 0.909 | 5/10 | 0.857 |
| 80 | 5/10 | 0.875 | 5/10 | 0.823 |

The order is **materially threshold-sensitive**. Moving the gate by five points
changes roughly half the top ten. This is the largest robustness threat in the
study and the reason the threshold must stay frozen and visible in the model
registry. It is not evidence for choosing a different threshold post hoc.

### 11.2 Metric weights and ablation

| Variant | Batting overlap | Batting Spearman | Bowling overlap | Bowling Spearman |
|---|---:|---:|---:|---:|
| Equal weights | 8/10 | 0.985 | 8/10 | 0.980 |
| Primary-heavy | 9/10 | 0.997 | 9/10 | 0.993 |
| Secondary-heavy | 9/10 | 0.982 | 9/10 | 0.987 |

Single-metric ablations retain 7–9 of 10 in the affected discipline. Batting
loses the most when the primary (strike-rate) or dismissal component is removed
(7/10, Spearman 0.947 and 0.975). Bowling loses the most when the primary
(economy) or wicket component is removed (7/10, Spearman 0.947 and 0.962). No
single component is solely responsible for the list, but the primary component
is clearly load-bearing.

### 11.3 Eligibility gates

Lowering the batter gate to 24 or 36 leaves all ten unchanged; 45 retains nine;
60 retains eight. Lowering the bowler gate to 24/36/45 leaves all ten; 60
retains nine. The frozen gates are therefore not knife-edge choices for the
published top ten, although the 60-ball variants begin to remove players with
genuinely thin samples.

### 11.4 Priors

Baseline prior 100 and 500 retain 9–10 of 10; ranking reliability prior 75 and
300 retain 9–10 of 10. The order is insensitive to reasonable prior changes.

### 11.5 Leave-one-season-out (IPL 2022–2026)

| Season removed | Batting overlap | Batting Spearman | Bowling overlap | Bowling Spearman |
|---|---:|---:|---:|---:|
| 2022 | 8/10 | 0.938 | 6/10 | 0.932 |
| 2023 | 8/10 | 0.946 | 7/10 | 0.882 |
| 2024 | 6/10 | 0.897 | 7/10 | 0.876 |
| 2025 | 7/10 | 0.961 | 8/10 | 0.852 |
| 2026 | 7/10 | 0.967 | 9/10 | 0.953 |

No single season carries the list. The weakest case is batting with 2024
removed (6/10) and bowling with 2025 removed (8/10 but Spearman 0.852). The
2026 season is the most recent and the most influential for batting, as
expected for current form.

### 11.6 Remove-one-match influence and concentration

For each published top-ten player the whole match with their largest pressure
ball count was removed and the full ranking recomputed. The largest rank movement
was 20 places (Umran Malik), followed by 14 (Prasidh Krishna) and 13 (Shubham
Dubey). Several players actually improve when their largest-volume match is
removed, which means the match was a drag rather than the foundation of the
ranking.

No card has more than 50% of its pressure balls in one match (maximum 0.47,
Sherfane Rutherford). One card exceeds the 75% season-concentration flag:
Vipraj Nigam at 0.90, driven by 2025. He is retained in the published list but
flagged in the evidence card and should be read as a limited-evidence,
season-concentrated case. All other cards are below 0.67 season share.

### 11.7 Correlation and collinearity

The IPL metric deltas are correlated as expected:

- Batting: primary vs boundary **0.92**, primary vs dot **0.77**.
- Bowling: primary vs boundary **0.93**, primary vs dot **0.68**.

The primary and boundary components therefore carry overlapping information,
and the effective weighting is more concentrated than the nominal weights
suggest. The ablation results quantify the consequence: removing either primary
or dismissal for batting, or primary or wicket for bowling, changes 3 of 10
places. The weights are retained because they are cricket-legible and the
ablations do not overturn the core list, but the overlap is a documented
limitation rather than a hidden assumption.

## 12. Threats to validity

- **Threshold sensitivity** dominates every other robustness result. The frozen
  70 is a Phase 2 decision and must not be tuned against these rankings.
- **Wide intervals.** Match-level resampling with 47–228 pressure balls per
  player cannot produce narrow rank intervals. Treat ranks 3–10 as a band, not
  a strict order.
- **Limited-evidence entries.** Shubham Dubey, Vipraj Nigam, Sanju Samson and
  Umran Malik are below the high-evidence gate. They are eligible and ranked,
  but their cards should not be read as settled.
- **Correlated components.** Primary and boundary deltas overlap; nominal
  weights overstate independence.
- **Scope mismatch.** T20I corroboration shares almost no names with the IPL
  list. That is not a contradiction, but it means T20I cannot rescue or
  overturn an IPL placement.
- **No opposition-quality adjustment.** The baseline is context-adjusted by
  scope, innings type, role/phase and match history, not by opponent strength.
- **No outcome leakage.** Pressure states are pre-delivery. The ranking uses
  only realised performance under those states, never the match result.

## 13. Model registry entry

| Field | Value |
|---|---|
| Model id | `ranking/v1.0.0` |
| Metrics model | `metrics/v1.0.0` |
| Pressure model | `pressure/v1.0.0` |
| Canonical version | `canon/v1` |
| Config path | `configs/models/ranking_v1.toml` |
| High-pressure threshold | 70.0 (index) |
| Transform | 2.5/97.5 winsor, median/MAD 1.4826, z ±3 |
| Reliability prior | 150 balls |
| Eligibility | 30 batter / 36 bowler pressure balls |
| Bootstrap | 1000 match-level replicates, seed 20261110, 95% |
| Run date | 2026-11-11 |
| Outputs | `data/rust/rankings/*`, checksummed manifest |

## 14. Reproduction

```powershell
$env:Path = "$env:USERPROFILE\.cargo\bin;$env:Path"

# Authoritative rankings from a clean engine run
cargo run --release --manifest-path engine/Cargo.toml --bin scout -- rank

# Score/rank parity, bootstrap intervals and remove-one-match influence
python research/rank_stability.py

# Threshold, weight, ablation, eligibility, prior and LOSO variants
python research/rank_sensitivity.py

# Evidence cards for the published top ten
python research/evidence_cards.py

# Tests
cargo test --manifest-path engine/Cargo.toml --workspace
python -m pytest -q tests/unit
```

Expected artefacts:

- `data/rust/research/rank_stability.csv`
- `data/rust/research/rank_sensitivity.csv`
- `data/rust/research/top10_evidence_cards.csv`
- `reports/research/phase4_sensitivity.json`
- `reports/research/phase4_evidence_cards.md`
- `reports/qa/phase4_validation.json`
- `reports/qa/phase4_tests.txt`

## 15. Sign-off status

- Rankings reproduce from a clean Rust run: **yes**.
- Published tables match the frozen engine: **yes**, zero rank mismatches.
- Uncertainty and robustness reported: **yes**, with the threshold caveat
  stated explicitly.
- No ranking dominated by an unexamined single match/season: **yes** for
  matches; one season-concentrated card is flagged and examined.
- All top-ten evidence cards complete: **yes**.
- QA sign-off: recorded in `reports/qa/phase4_tests.txt` and
  `reports/qa/phase4_validation.json`.

Open review item for the cricket reviewer: whether the threshold sensitivity is
acceptable for a scouting shortlist, or whether the published list should be
presented as a wider band (for example top 15) in Phase 5. The frozen order
itself is not changed by this report.
