Failed experiments do not disappear

Question Can a replacement-baseline cumulative value ledger be published honestly before fielding is solved?

Decision Keep the research. Publish with the holes still visible.

Why it matters Fielding is blocked by defense entanglement; runs-per-win 10.485 with a held-out 3.65-win MAE. The one provisional metric in a registry where nothing is validated. CROWN v3 (2026-08-08) is a disclosure-only successor: "standalone terminal scope is stated exactly and every numerical player value is unchanged."

Question Does the shrunken batting arm beat the baseline and a league-mean-only control?

Decision Keep the research. Publish with the holes still visible.

Why it matters After the M-13 paired bootstrap it "publishes as supported_but_marginal_after_multiplicity" — Bonferroni upper bound -6.17e-06, 0.29% of the effect, "where an independent auditor's run landed on the other side."

Question Does lagged CORE beat BPM, PER, WS/48, and VORP-rate at a frozen prediction task?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters "No completed diagnostic or experiment nominates a production formula change." CORE remains provisional by its own registry.

Question How does CORE align with RAPTOR's on/off and box components?

Decision Keep the research. Do not treat it as a finished public number.

Why it matters EPM, LEBRON, DARKO, xRAPM, and ESPN Net Points appear only as honest blocked-availability states.

Question What do points buy in wins, and what does replacement look like, before FORGE claims any player value?

Decision Keep the research. Do not treat it as a finished public number.

Why it matters Evidence only, constants pre-declared before results were inspected: 36.11 points per win (90% CI [34.98, 37.18]); fitted Pythagorean exponent 2.65 beats the folklore 2.37 on a 2024 test-once. Audit overlay 2026-08-08: foundations "RETAIN_DESCRIPTIVE_ONLY", v1 role conclusions "SUPERSEDED_UNSUPPORTED" — the product's release notes state "FORGE v1 role conclusions are superseded by a season-aware eligibility scaffold." FORGE remains scaffold, and no values exist anywhere.

Question Does the CORE-WAR accounting and scale validation earn promotion?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters Receipt 2b4c4d02… (CORE WAR ACCOUNTING SCALE VALIDATION, 10/13 criteria) is listed publicly in evidence.json. This entry previously conflated two receipts; the conditional-prediction receipt (2e8b9131…, 15/16) now has its own entry below.

Question Can possession-level attribution beat the frozen criteria it preregistered?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters The final experiment reproduced all 11 scientific outputs byte-for-byte before returning its NULL — reproducibility and promotion are separate bars, and only one was cleared.

Question Does the product’s analytical record survive an independent hostile audit?

Decision Keep the research. Do not treat it as a finished public number.

Why it matters The audit reversed three prior promotions, corrected the published corpus counts (4,289 game payloads and 510 team-seasons rather than the builder’s 4,830/512), revised season conservation to 22/27 PASS, and declared 2025 permanently spent as model evidence. Per-model current dispositions are public in /data/experiments.json independent_audit blocks; the producer states builder verdicts display only as superseded history.

Question Is kicking points above expected ready to claim anything?

Decision Keep the research. Do not treat it as a finished public number.

Why it matters The product’s release notes: "LEG kicking points above expected is a scaffold with uncertainty and replay contracts. Its genuinely unopened future-season gate is intentionally not passed." A gate deliberately left unopened is different from a gate failed.

Question Does the BUCKETS formula earn a production nomination under frozen criteria?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters Terminal NULL with no production nomination. The product’s public evidence file states: "every completed pilot to date returned NULL."

Question Does the CHEF formula earn a production nomination under frozen criteria?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters Terminal NULL. CHEF was also relabeled "Three-Point Production" after a 2026-08-03 integrity correction found the prior label misdescribed the formula.

Question Does the MAMBA (Creation Load) formula earn a production nomination?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters The program ran as kobe-formula-validation; its receipt displays publicly as MAMBA after the 2026-07-30 metric-identity migration (MAMBA — Creation Load). Terminal NULL, no nomination.

Question Does an independent defensive component justify changing ALIEN — or removing its DBPM dependency?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters "Removing DBPM materially weakens ALIEN. Retain the dependency and disclose it; no production formula change is nominated." The retained limitation — a disclosed DBPM dependency — is itself the finding.

Question Does conditional CORE-WAR prediction earn promotion at a frozen, matched task?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters "Full-precision CORE materially beats the training-mean null for margin and win probability; it is practically indistinguishable from the frozen two-decimal ablation." 2,767 games / 5,534 team-game rows with 5,000 paired whole-game bootstrap draws — distinct from the accounting/scale receipt above.

Question Can any independent expected-points or win-probability arm beat replaying the nflverse EP/WP columns under frozen gates?

Decision Record the NULL. Do not treat no-change as a hidden success.

Why it matters Per the packet's decision field: "no independent arm beat source replay on the frozen gates." Production displays source EP/WP with nflverse attribution; no independent EP/WP model is claimed. The 2026-08-08 independent audit retained the disposition — challenger_status "REWORK", current_verdict "RETAIN_BASELINE" — pending a new predeclared future-data tournament.

Question Does a context-ridge QB impact estimate earn research status against frozen gates?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters The builder’s D-011 promotion is superseded: D-019 states "D-011 does not promote `passing_impact_rate`." The metric is scaffold, candidate use is research-workbench only, and no leaderboard or public values are authorized.

Question Does an EWMA form estimate beat transparent trailing windows?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters PULSE is descriptive-only, never a forecast. The v1 forward-test numbers once quoted here were superseded when the 2026-08-08 post-moonshot audit found the test had leaked target information (PM-01); the corrected, sealed v2 evidence retains the anti-forecast finding — see the forward-test correction entry.

Question Do the pitching impact-rate challengers hold up under paired bootstrap?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters "Green" was demoted to meaning the tournament executed soundly — not that a claim holds.

Question Does any candidate MLB team-strength model earn promotion over simple baselines?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters Corrected evidence, 2026-08-08: the earlier bootstrap-CI framing is withdrawn — "these are sensitivity ranges—not 95% confidence intervals or tests of separation." The correction enforced strict completion-date availability and exactly-equal contestant masks over 8,841 common games. mlb-champion-ridge-v0.1.0 stays research only, never a rank claim.

Question Is a CFP Path model justified on current evidence?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters Both preregistered decisive experiments failed; either alone would have ended the campaign. Research I remains fail-closed before full implementation with the owner hypothesis UNRESOLVED — "the current evidence does not authorize a target-team probability." No forecast snapshot exists and no public surface was deployed.

Question Did the forward test behind PULSE’s anti-forecast lede hold up under audit?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters PM-01 (critical) found the v1 test estimated its league comparator from post-cutoff outcomes; two re-attacks found further leakage. The v2 evidence seals a target-independent cutoff reference, and the product’s live lede now reads: "The league reference is now frozen entirely before the target period, and paired intervals find the short windows have higher error. Season-to-date is not distinguishable from that reference."

Question Does partial pooling at K=25 earn promotion over the unpooled baserunning baseline?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters v1’s selection had scored aggregate full-sample means with no real split and no interval (PM-16); v2 reran a nested chronological evaluation before declining to promote.

Question What happens when a published claim or display is found wrong after release?

Decision Keep the incumbent. Do not promote the challenger.

Why it matters The exact official-gamelog conservation claim was withdrawn — the data was not (D-014); the producer chose versioned disclosure over reclassifying events to force agreement. The same successor wave removed the numeric ip field after impossible displays like "187.7" innings for 563 outs, which render as "187.2" in baseball notation (D-015). Every correction ships as a versioned successor artifact with predecessors preserved.

Question Does park adjustment improve the batting impact-rate model enough to promote?

Decision Keep the failure on the record. Do not publish this approach as a success.

Why it matters A reformulated additive arm later landed at -0.000305 — "still a null, but a real one," with power proven on synthetic data.

Question Can a resume-based model reproduce the CFP committee’s final top-12 well enough to build a path model on?

Decision Keep the failure on the record. Do not publish this approach as a success.

Why it matters Preregistered bar: mean final top-12 overlap of at least 10.0/12. The nuance is real — a censored Plackett–Luce on generic resumes beats Elo-only and record-then-Elo in 11 of 12 held-out seasons, and still misses more than two field slots per season; in-season committee persistence beats every resume model.

Question Can tiebreak ambiguity be bounded tightly enough to publish a target-team path probability?

Decision Keep the failure on the record. Do not publish this approach as a success.

Why it matters Miami’s title-game probability is identified only to [0.631, 0.839] against a preregistered width bar of 0.10. "An ACC clarification would narrow the bounds and still not rescue them." 44.9% of simulated seasons reach a tiebreak rung only SportSource can observe.

Question Is individual offensive-line attribution identifiable enough to publish research values?

Decision Keep the research. Do not publish.

Why it matters D-019: "D-013 does not permit individual offensive-line values. Current disposition is `WITHHOLD_INDIVIDUAL`; only unit and identifiability research is authorized." The identifiability measurements stand as recorded history; the publication permission does not.

Question Do defensive-front research values survive a cross-family agreement gate?

Decision Keep the research. Do not publish.

Why it matters D-019: "D-015 does not permit individual defensive-front values. Current disposition is `UNIT_IDENTIFIABILITY_ONLY`; missing assignment evidence remains unmeasured." The family-spread finding (mean cross-family agreement 0.544) remains recorded history.

Question Which historical seasons survive validation against source play-by-play defects?

Decision Isolate from publication. Keep the disposition as history.

Why it matters Five seasons are publication-ineligible under conservation-v3 — 2001 and 2002 each contain 8 fatal games (transposed scores in source play-by-play), 2011 has one phantom-TD-class fatal game, and 1999–2000 fail the repaired reverse-coverage checks. Quarantined seasons stay quarantined rather than being repaired by guesswork.

Question Can pitch-level Statcast metrics ship?

Decision Stop until a lawful source is admitted.

Why it matters The product's /pitches page serves an honest unavailable state with a named unblocking condition rather than scraping.

Question Can shot-coordinate spatial surfaces ship?

Decision Stop until a lawful source is admitted.

Why it matters v3.1.2 tightened the epistemic wording: "No source is admitted. The request is reported sent by Roderic and awaiting a response; that is not permission or proof of delivery." Remaining at Tier 0 permanently is declared an acceptable final state.

Question Is there an approved live MLB data source for a 2026 Current Board?

Decision Stop until a lawful source is admitted.

Why it matters The MLB 2026 Current Board is UNAVAILABLE by construction; the historical/research vertical remains active. "The honest answer up front: no candidate has clean written permission for this use." Clarified 2026-08-08: "“No live source exists” should be read as “no live source has been reviewed and approved.”" An MLBAM permission inquiry is drafted and unsent.