Failed experiments do not disappear

Can a replacement-baseline cumulative value ledger be published honestly before fielding is solved?

“fielding/catching NULL (fielding blocked by defense entanglement) … runs-per-win 10.485 held-out at 3.65 wins MAE … never-better-than-WAR recorded”

Fielding is blocked by defense entanglement; runs-per-win 10.485 with a held-out 3.65-win MAE. The one provisional metric in a registry where nothing is validated. CROWN v3 (2026-08-08) is a disclosure-only successor: "standalone terminal scope is stated exactly and every numerical player value is unchanged."

SRC Hardball Signal ROADMAP.md MS-14 row (repo doc, not public) + data/registry/metrics.yaml (player_value_cumulative) methodology — CROWN cumulative value — a provisional ledger with visible holes ↗

Does the shrunken batting arm beat the baseline and a league-mean-only control?

“shrunken arm beat baseline AND league-mean-only (which beat the unpooled baseline — small-sample trap measured)”

After the M-13 paired bootstrap it "publishes as supported_but_marginal_after_multiplicity" — Bonferroni upper bound -6.17e-06, 0.29% of the effect, "where an independent auditor's run landed on the other side."

SRC Hardball Signal ROADMAP.md at commit c7816be (historical MS-10 row; repo doc, not public) validation — Batting shrinkage tournament — a win that shrank under multiplicity ↗

Does lagged CORE beat BPM, PER, WS/48, and VORP-rate at a frozen prediction task?

“Benchmarks pilot complete — verdict NULL, 8/16 criteria, no promotion.”

"No completed diagnostic or experiment nominates a production formula change." CORE remains provisional by its own registry.

SRC Court Signal ROADMAP.md header + docs/ANALYTICS_EVIDENCE_CHECKPOINT_2026-07-29.md, receipt ae454ca9… (repo docs, not public) evidence — Lagged CORE vs. public metrics on a matched future-outcome task ↗

How does CORE align with RAPTOR's on/off and box components?

“CORE, O-CORE and D-CORE each align more closely with RAPTOR on/off than RAPTOR box, but this is descriptive shared-input agreement—not a forecast, accuracy, or superiority claim.”

EPM, LEBRON, DARKO, xRAPM, and ESPN Net Points appear only as honest blocked-availability states.

SRC Court Signal README.md, 2026-07-29 verified research checkpoint (repo doc, not public) + docs/RAPTOR_CORE_EDITION_C_VERIFIED_RESULTS_2026-07-30.md signal matrix — RAPTOR–CORE Edition C — descriptive agreement, not superiority ↗

What do points buy in wins, and what does replacement look like, before FORGE claims any player value?

“FORGE foundations: calibration + cohort evidence recorded; FORGE stays scaffold”

Evidence only, constants pre-declared before results were inspected: 36.11 points per win (90% CI [34.98, 37.18]); fitted Pythagorean exponent 2.65 beats the folklore 2.37 on a 2024 test-once. Audit overlay 2026-08-08: foundations "RETAIN_DESCRIPTIVE_ONLY", v1 role conclusions "SUPERSEDED_UNSUPPORTED" — the product's release notes state "FORGE v1 role conclusions are superseded by a season-aware eligibility scaffold." FORGE remains scaffold, and no values exist anywhere.

SRC gridironsignal.rodericrinehart.com/data/experiments.json (forge_foundations_v1) + Gridiron Signal DECISIONS.md D-014 (repo doc, not public) methodology — FORGE foundations — the constants before any value claim ↗

Does the CORE-WAR accounting and scale validation earn promotion?

“CORE-WAR receipt assembled — verdict NULL, 10/13 criteria, no promotion.”

Receipt 2b4c4d02… (CORE WAR ACCOUNTING SCALE VALIDATION, 10/13 criteria) is listed publicly in evidence.json. This entry previously conflated two receipts; the conditional-prediction receipt (2e8b9131…, 15/16) now has its own entry below.

SRC Court Signal DECISIONS.md 2026-07-27 ("CORE-WAR accounting/scale pilot: NULL, no promotion"), receipt 2b4c4d02… (repo doc, not public; receipt listed publicly in courtsignal.rodericrinehart.com/data/evidence.json ) evidence — CORE-WAR accounting/scale receipt — a 12-hour NULL ↗

Can possession-level attribution beat the frozen criteria it preregistered?

“Attribution receipt assembled — verdict NULL, 6/10 criteria, no promotion.”

The final experiment reproduced all 11 scientific outputs byte-for-byte before returning its NULL — reproducibility and promotion are separate bars, and only one was cleared.

SRC Court Signal ROADMAP.md header block, receipt 62df00c4… (repo doc, not public) evidence — Attribution experiment receipt — reproducible, and NULL ↗

Does the product’s analytical record survive an independent hostile audit?

“Pre-repair verdict: `WITHHOLD`, frozen in commit `33f21f9` … RESEARCH-PREVIEW-RETAINED”

The audit reversed three prior promotions, corrected the published corpus counts (4,289 game payloads and 510 team-seasons rather than the builder’s 4,830/512), revised season conservation to 22/27 PASS, and declared 2025 permanently spent as model evidence. Per-model current dispositions are public in /data/experiments.json independent_audit blocks; the producer states builder verdicts display only as superseded history.

SRC Gridiron Signal docs/audits/2026-08-08_CODEX_FINAL_CERTIFICATION.md + DECISIONS.md D-019 (repo docs, not public) + gridironsignal.rodericrinehart.com/release-notes ("Publication posture: bounded research preview.") research lab — The GS-18 hostile audit — WITHHOLD, repaired, retained ↗

Is kicking points above expected ready to claim anything?

“SCAFFOLD_COMPLETE_FUTURE_GATE_REQUIRED”

The product’s release notes: "LEG kicking points above expected is a scaffold with uncertainty and replay contracts. Its genuinely unopened future-season gate is intentionally not passed." A gate deliberately left unopened is different from a gate failed.

SRC gridironsignal.rodericrinehart.com/data/experiments.json (kicking_expected_v1) + /release-notes methodology — LEG kicking scaffold — the future gate intentionally not passed ↗

Does the BUCKETS formula earn a production nomination under frozen criteria?

“the BUCKETS pilot EXECUTED: first result-complete formula validation; verdict NULL”

Terminal NULL with no production nomination. The product’s public evidence file states: "every completed pilot to date returned NULL."

SRC Court Signal DECISIONS.md 2026-07-24 heading (repo doc, not public); receipt a701dc41… listed publicly in courtsignal.rodericrinehart.com/data/evidence.json evidence — BUCKETS formula pilot — first result-complete validation, NULL ↗

Does the CHEF formula earn a production nomination under frozen criteria?

“The CHEF pilot EXECUTED: second result-complete formula validation; verdict NULL”

Terminal NULL. CHEF was also relabeled "Three-Point Production" after a 2026-08-03 integrity correction found the prior label misdescribed the formula.

SRC Court Signal DECISIONS.md 2026-07-25 heading (repo doc, not public); receipt f33f9585… listed publicly in courtsignal.rodericrinehart.com/data/evidence.json evidence — CHEF formula pilot — second result-complete validation, NULL ↗

Does the MAMBA (Creation Load) formula earn a production nomination?

“The KOBE pilot EXECUTED: third result-complete formula validation; verdict NULL”

The program ran as kobe-formula-validation; its receipt displays publicly as MAMBA after the 2026-07-30 metric-identity migration (MAMBA — Creation Load). Terminal NULL, no nomination.

SRC Court Signal DECISIONS.md 2026-07-25 heading (repo doc, not public); receipt a72ff69b… listed publicly (display label MAMBA) in courtsignal.rodericrinehart.com/data/evidence.json evidence — MAMBA formula pilot — third result-complete validation, NULL ↗

Does an independent defensive component justify changing ALIEN — or removing its DBPM dependency?

“ALIEN primary/comparator complete — both exact replay, both NULL.”

"Removing DBPM materially weakens ALIEN. Retain the dependency and disclose it; no production formula change is nominated." The retained limitation — a disclosed DBPM dependency — is itself the finding.

SRC Court Signal ROADMAP.md (repo doc, not public); receipts 3772bbf9… (primary, 13/19) and 36cdc277… (comparator, 7/14) listed publicly in courtsignal.rodericrinehart.com/data/evidence.json evidence — ALIEN pilots — primary and DBPM-free comparator, both NULL ↗

Does conditional CORE-WAR prediction earn promotion at a frozen, matched task?

“Conditional CORE-WAR prediction complete — exact replay, 15/16, NULL.”

"Full-precision CORE materially beats the training-mean null for margin and win probability; it is practically indistinguishable from the frozen two-decimal ablation." 2,767 games / 5,534 team-game rows with 5,000 paired whole-game bootstrap draws — distinct from the accounting/scale receipt above.

SRC Court Signal ROADMAP.md, receipt 2e8b9131… (repo doc, not public) evidence — Conditional CORE-WAR prediction — exact replay, 15/16, NULL ↗

Can a market-evidence ledger avoid laundering partial checks into full sweeps?

“**The three facts behind a price claim are kept apart** (`evidence-integrity.mjs`): which books were CHECKED, which prices were RETAINED, which retained price WON. None is derived from another.”

Private system — posture facts only. Probability, fair-price, EV, Kelly, and portfolio-risk surfaces remain blocked fail-closed; pre-attribution legacy evidence is quarantined as historical and inadmissible; per-promotion eligibility replaced any global minimum-odds rule.

SRC Hedgebook ROADMAP.md "2026-08-08 implementation update — NBA awards and evidence integrity" — private system, no public link

Can any independent expected-points or win-probability arm beat replaying the nflverse EP/WP columns under frozen gates?

“EP/WP tournament v1: RETAIN_BASELINE (NULL result, recorded) … A NULL result is a publishable outcome, not a failure to be massaged.”

Per the packet's decision field: "no independent arm beat source replay on the frozen gates." Production displays source EP/WP with nflverse attribution; no independent EP/WP model is claimed. The 2026-08-08 independent audit retained the disposition — challenger_status "REWORK", current_verdict "RETAIN_BASELINE" — pending a new predeclared future-data tournament.

SRC gridironsignal.rodericrinehart.com/data/experiments.json (ep_wp_tournament_v1) + Gridiron Signal DECISIONS.md D-010 (repo doc, not public) research lab — EP/WP tournament v1 — challengers vs. the nflverse baseline ↗

Does a context-ridge QB impact estimate earn research status against frozen gates?

“RETAIN_BASELINE … All three audited candidate-minus-raw effects are small and their player-cluster 95 percent intervals span zero; alpha 1000 also fails the split-half comparison with raw EPA/dropback.”

The builder’s D-011 promotion is superseded: D-019 states "D-011 does not promote `passing_impact_rate`." The metric is scaffold, candidate use is research-workbench only, and no leaderboard or public values are authorized.

SRC gridironsignal.rodericrinehart.com/data/experiments.json (qb_core_pilot_v1.independent_audit, audit_id qb_core_pilot_v1_codex_hostile_audit_2026-08-08) + Gridiron Signal DECISIONS.md D-019 (repo doc, not public) methodology — QB CORE pilot v1 — context-adjusted quarterback impact ↗

Does an EWMA form estimate beat transparent trailing windows?

“Transparent windows retained (EWMA lost)”

PULSE is descriptive-only, never a forecast. The v1 forward-test numbers once quoted here were superseded when the 2026-08-08 post-moonshot audit found the test had leaked target information (PM-01); the corrected, sealed v2 evidence retains the anti-forecast finding — see the forward-test correction entry.

SRC Hardball Signal ROADMAP.md at commit cc8854e (historical MS-13 row; repo doc, not public) methodology — PULSE EWMA challenger — exponential decay vs. transparent windows ↗

Do the pitching impact-rate challengers hold up under paired bootstrap?

“starter 95% [-0.003991, +0.000468] and reliever [-0.001910, +0.000483] both SPAN ZERO and are now labelled not_distinguishable_from_zero”

"Green" was demoted to meaning the tournament executed soundly — not that a claim holds.

SRC Hardball Signal docs/operations/DEPLOYMENT.md, deployment f54f3f56 row (repo doc, not public) validation — Starter and reliever pitching tournaments — claims dissolved ↗

Does any candidate MLB team-strength model earn promotion over simple baselines?

“No contestant separated from the field. elo took 3 of 4 outer seasons, and its overall margin_mae (3.4241) differs from champion-ridge (3.4229) by 0.0013 runs, inside the preregistered materiality floor of 0.01. On this evidence the models are not distinguishable, so nothing is promoted and no ranking between them is claimed.”

Corrected evidence, 2026-08-08: the earlier bootstrap-CI framing is withdrawn — "these are sensitivity ranges—not 95% confidence intervals or tests of separation." The correction enforced strict completion-date availability and exactly-equal contestant masks over 8,841 common games. mlb-champion-ridge-v0.1.0 stays research only, never a rank claim.

SRC Rinehart Ratings docs/audits/2026-08-08_MLB_CHAMPION_TOURNAMENT_CORRECTED.json (repo doc, not public; supersedes the 2026-08-06 artifact, which remains as historical evidence of the superseded method and claim) methodology — MLB champion model tournament — nothing promoted ↗

Should a startup-order calibration candidate activate after passing every numeric gate?

“Qualitative decision: do not activate this candidate.”

Private system — posture fact only. A candidate passed its preregistered numeric checks and was still rejected for correlated component reuse and incomplete source lineage. No private metric value, comparison population, or identity count is published.

SRC Roster Command docs/DECISIONS.md (2026-08-01) + docs/MODEL_CHANGELOG.md — private system, no public link

Is a CFP Path model justified on current evidence?

“STOP. Research III is not justified on current evidence.”

Both preregistered decisive experiments failed; either alone would have ended the campaign. Research I remains fail-closed before full implementation with the owner hypothesis UNRESOLVED — "the current evidence does not authorize a target-team probability." No forecast snapshot exists and no public surface was deployed.

SRC Rinehart Ratings docs/research/2026_CFP_PATH_RESEARCH_II_DECISION.md + docs/research/2026_CFP_PATH_MODEL_DECISION.md (repo docs, not public) methodology — CFP Path research — stopped by its own stopping rule ↗

Did the forward test behind PULSE’s anti-forecast lede hold up under audit?

“PULSE trailing-50 and trailing-100 were worse than the sealed league reference, season-to-date was not distinguishable, and no EWMA survived selection-aware promotion”

PM-01 (critical) found the v1 test estimated its league comparator from post-cutoff outcomes; two re-attacks found further leakage. The v2 evidence seals a target-independent cutoff reference, and the product’s live lede now reads: "The league reference is now frozen entirely before the target period, and paired intervals find the short windows have higher error. Season-to-date is not distinguishable from that reference."

SRC Hardball Signal docs/releases/FINAL_REPORT_2026-08-08.md + docs/audits/POST_MOONSHOT_AUDIT_2026-08-08.md PM-01 (repo docs, not public) methodology — PULSE forward test — leakage found, evidence corrected, finding retained ↗

Does partial pooling at K=25 earn promotion over the unpooled baserunning baseline?

“`K=25` beat unpooled but ranked behind league-only and its league interval spanned zero, so the arm was not promoted; v1 realized values remain unchanged”

v1’s selection had scored aggregate full-sample means with no real split and no interval (PM-16); v2 reran a nested chronological evaluation before declining to promote.

SRC Hardball Signal docs/audits/POST_MOONSHOT_AUDIT_2026-08-08.md PM-16 disposition (repo doc, not public) validation — Baserunning shrinkage tournament v2 — the winning arm was still not promoted ↗

What happens when a published claim or display is found wrong after release?

“The event policy produces 3,439 SB and 981 CS versus official gamelog credits of 3,440 and 989 across exactly nine clean games.”

The exact official-gamelog conservation claim was withdrawn — the data was not (D-014); the producer chose versioned disclosure over reclassifying events to force agreement. The same successor wave removed the numeric ip field after impossible displays like "187.7" innings for 563 outs, which render as "187.2" in baseball notation (D-015). Every correction ships as a versioned successor artifact with predecessors preserved.

SRC Hardball Signal DECISIONS.md D-014 · D-015 (repo docs, not public) + hardballsignal.rodericrinehart.com/release-notes data & sources — season_stats corrections — a withdrawn claim and an impossible display ↗

Does park adjustment improve the batting impact-rate model enough to promote?

“park arm FAILED (-2.3e-05 vs 0.001 threshold) → incumbent retained, failed arm published”

A reformulated additive arm later landed at -0.000305 — "still a null, but a real one," with power proven on synthetic data.

SRC Hardball Signal ROADMAP.md A2 row + docs/operations/DEPLOYMENT.md 3d6eb6d0 row (repo docs, not public) validation — Park-adjusted batting tournament: challenger failed, result published ↗

Can a resume-based model reproduce the CFP committee’s final top-12 well enough to build a path model on?

“9.75/12 — FAIL”

Preregistered bar: mean final top-12 overlap of at least 10.0/12. The nuance is real — a censored Plackett–Luce on generic resumes beats Elo-only and record-then-Elo in 11 of 12 held-out seasons, and still misses more than two field slots per season; in-season committee persistence beats every resume model.

SRC Rinehart Ratings docs/research/2026_CFP_PATH_RESEARCH_II_DECISION.md + docs/research/2026_CFP_COMMITTEE_RANKABILITY.json (repo docs, not public) methodology — CFP committee rankability — the ordering did not validate ↗

Can tiebreak ambiguity be bounded tightly enough to publish a target-team path probability?

“0.208 / 0.180 — FAIL, under every policy reading”

Miami’s title-game probability is identified only to [0.631, 0.839] against a preregistered width bar of 0.10. "An ACC clarification would narrow the bounds and still not rescue them." 44.9% of simulated seasons reach a tiebreak rung only SportSource can observe.

SRC Rinehart Ratings docs/research/2026_CFP_PATH_RESEARCH_II_DECISION.md + docs/research/2026_ACC_MIAMI_PARTIAL_IDENTIFICATION.json (repo docs, not public) methodology — ACC/Miami partial identification — the bounds are not narrow ↗

Is individual offensive-line attribution identifiable enough to publish research values?

“WITHHOLD_INDIVIDUAL … quantitative results mix participation seasons withheld by policy”

D-019: "D-013 does not permit individual offensive-line values. Current disposition is `WITHHOLD_INDIVIDUAL`; only unit and identifiability research is authorized." The identifiability measurements stand as recorded history; the publication permission does not.

SRC gridironsignal.rodericrinehart.com/data/experiments.json (trench_visibility_v1.audit_posture, publication_status "withheld") + Gridiron Signal DECISIONS.md D-019 (repo doc, not public) methodology — Trench visibility v1 — can offensive linemen be identified at all? ↗

Do defensive-front research values survive a cross-family agreement gate?

“UNIT_IDENTIFIABILITY_ONLY”

D-019: "D-015 does not permit individual defensive-front values. Current disposition is `UNIT_IDENTIFIABILITY_ONLY`; missing assignment evidence remains unmeasured." The family-spread finding (mean cross-family agreement 0.544) remains recorded history.

SRC gridironsignal.rodericrinehart.com/data/experiments.json (trench_visibility_v2_front.audit_posture, publication_status "withheld") + Gridiron Signal DECISIONS.md D-019 (repo doc, not public) methodology — Trench wave 2 — interior defensive line and off-ball linebackers ↗

Can a successor rank model ship without a governed exposure estimator?

“Outcome: **WITHHOLD**.”

No candidate board was ever generated: the required player-level exposure and workload estimator does not exist as governed evidence, and the producer declined to guess. Startup and Auction stay disabled; authoritative rank surfaces fail closed. Private system — posture facts only.

SRC Roster Command docs/MODEL_CHANGELOG.md ("Fundamental v0.2 Session A — Outcome B, withheld") — private system, no public link

Which historical seasons survive validation against source play-by-play defects?

“The former 24/27 conservation claim did not check reverse schedule coverage or fail on errors. The repaired v2 result is 22/27 PASS.”

Five seasons are publication-ineligible under conservation-v3 — 2001 and 2002 each contain 8 fatal games (transposed scores in source play-by-play), 2011 has one phantom-TD-class fatal game, and 1999–2000 fail the repaired reverse-coverage checks. Quarantined seasons stay quarantined rather than being repaired by guesswork.

SRC gridironsignal.rodericrinehart.com/release-notes (GS-18 section) + live /data/freshness.json season verdicts (FAIL: 1999, 2000, 2001, 2002, 2011) data & sources — Season conservation under a hostile audit — 22 of 27 PASS ↗

What happens when the production model fails its own validity audit?

“RC-KI-047 — RESOLVED BY QUARANTINE: Fundamental 0.1.8 is analytically invalid”

Private system — posture facts only, no values or identities. The producer quarantined its own production model, retained the 0.1.8 rows as immutable historical evidence, and redeployed the same day to an honest unavailable state.

SRC Roster Command docs/KNOWN_ISSUES.md RC-KI-047 + docs/MODEL_CHANGELOG.md — private system, no public link

May a forecast ever act on a market price?

“blocked by real evidence, executable in code … Passing one gate never compensates for another. Certification requires seven of seven.”

The pilot's shadow ledger begins empty by design. No probability, fair-price, EV, or stake surface renders until certification.

SRC Hedgebook docs/V1_CERTIFICATION_PILOT.md — private system, no public link

Can pitch-level Statcast metrics ship?

“The seam is complete and tested; **no admitted lawful source exists**”

The product's /pitches page serves an honest unavailable state with a named unblocking condition rather than scraping.

SRC Hardball Signal docs/releases/FINAL_REPORT_2026-08-07.md line 72 (repo doc, not public) data & sources — Statcast-derived metrics — complete, tested, and blocked ↗

Can shot-coordinate spatial surfaces ship?

“parked external dependency, not a failed product”

v3.1.2 tightened the epistemic wording: "No source is admitted. The request is reported sent by Roderic and awaiting a response; that is not permission or proof of delivery." Remaining at Tier 0 permanently is declared an acceptable final state.

SRC Court Signal ROADMAP.md CONTROLLING POSTURE + docs/releases/SHOT_CHARTS_TIER0_COMPLETION_2026-07-31.md (repo docs, not public) methodology — Shot charts and spatial analytics — parked on permission, not failure ↗

Is there an approved live MLB data source for a 2026 Current Board?

“Status: BLOCKED. No live MLB source is approved, and none has been requested from a provider.”

The MLB 2026 Current Board is UNAVAILABLE by construction; the historical/research vertical remains active. "The honest answer up front: no candidate has clean written permission for this use." Clarified 2026-08-08: "“No live source exists” should be read as “no live source has been reviewed and approved.”" An MLBAM permission inquiry is drafted and unsent.

SRC Rinehart Ratings docs/mlb-live-source-decision.md + docs/mlb-live-source-review-2026-08-06.md (repo docs, not public) methodology — MLB live data source — blocked until someone says yes ↗