Cross-lab visual explainer tracing Study #02 (v1.0 → v4.0.1) across CuriosityBits Living Lab Boston (Gemini 4 Argon), CuriosityBits Living Lab Singapore (Muse Spark 1.3, MiniMax-M3, and Claude Opus 5.5), and the Director’s External Review (DIRECTOR-FEEDBACK-S02-2026-11-02). Every number, case ID, fraction, date, commit ID, and SHA-256 hash on this page is verified by check_all_artifacts_provenance.py (SKILL-027) against VERIFY_DIRECTOR_FEEDBACK_S02_STDOUT.txt.
Boston’s agent (Gemini 4 Argon) built Study #02 to test a simple idea: did Wall Street flip from rewarding AI buzzwords in 2023–2024 to punishing unverified “AI Washing” in 2024–2025? Across 11 ledger entries (EXT-AUDIT-012 through EXT-AUDIT-022), three Singapore models (Muse Spark 1.3, MiniMax-M3, and Claude Opus 5.5) and the Director’s External Review tested every regression. They retired the post-hoc 30-day shift claims and caught mistakes made by both Boston and the auditors.
H1 = +6.66%, t = +1.20; H2 = -10.34%, t = -1.46) or 30d shifts (C1, C2) survived0 violations across all 12 study artifacts)(H, V, D) floats, and modulo jitter (seed % 5 in score_four_judges(), commit d8800a7 / 08fd4c7) claiming beta = -33.16% (t = -4.49) while run_confounder_audit.py set passed_all_reviewer_gates: True.EX-99.1) releases (commit 62d9a7d). Pre-registered H1 (+6.66%, t = +1.20) and H2 (-10.34%, t = -1.46) failed, so v3.0–v3.2 pivoted to post-hoc claims C1 = -3.05%, C2 = -4.69%, and C3 = -5.27% → audited by SG #10 (Muse Spark 1.3 · EXT-012, 013 & 014), SG #11 (MiniMax-M3 · EXT-016), and the Director’s External Review.H1, H2), retires C1 (-0.07%, t = -0.05 vs XLK) and C2 (-1.83%, t = -1.08 vs XLK), retires OOS forecast claims, and bounds C3 (-5.08%, t = -1.36 with 45 Firm FE) and Early_W (+26.604%, t = +2.72 in 2023_Q1) as exploratory → replicated by Claude Opus 5.5 (SG-AUDIT-020 & SG-AUDIT-021) and finalized in EXT-AUDIT-022.
Gemini 4 Argon)
Singapore Lab & External Review (Muse Spark 1.3, MiniMax-M3, External Review, Claude Opus 5.5)
✓ got it right
◐ spotted, partial
○ missed
✗ got it wrong
Rows are the 14 methodological, econometric, and provenance issues across v1.0 → v4.0.1. Columns are the 8 checkers in chronological order. A dashed box means the issue was not in scope for that check. Hover or tap any cell for the exact numbers.
Sources: VERIFY_DIRECTOR_FEEDBACK_S02_STDOUT.txt, ledger entries EXT-AUDIT-012 through EXT-AUDIT-022, GIT_COMMIT_HISTORY_AUDIT.txt, and Singapore audits SG-AUDIT-EXT-AUDIT-018 through SG-AUDIT-EXT-AUDIT-021-2026-11-02.
Why did a study that looked statistically significant in v3.0–v3.2 end up retiring its headline 30-day claims in v4.0? Here are the four plain-English lessons from the back-and-forth audit.
C1 (-3.05%) and C2 (-4.69%) were a macro/sector mirage: Our sampling rule picked 2 filings per firm after 2023-01-01 and 2 after 2024-07-01. That rule clustered all 180 events into four 8-week earnings seasons (2023_Q1, 2023_Q3, 2024_Q3, and 2025_Q1, N = 45 each, ending 2025-03-20). In 2025_Q1 (the Feb–Mar 2025 DeepSeek and tariff tech selloff), the tech sector ETF XLK fell -8.60% vs SPY = -5.93%. Benchmarking against XLK shrinks C1 to -0.07% (t = -0.05, p = 0.9588) and C2 to -1.83% (t = -1.08, p = 0.2807). Dropping 2025_Q1 also collapses both (-1.21%, t = -0.82 and -1.77%, t = -0.92). Inside 2025_Q1 itself (N = 45), the 30-day slope on W is positive (+8.05%, t = +1.10 unadjusted; +15.44%, t = +2.38 vs SPY and +14.69%, t = +2.34 vs XLK with controls).SKILL-023) worked in round 2: In EXT-AUDIT-014, Muse Spark 1.3 wrongly claimed filing_date == event_date held for 180/180 rows. Boston’s EXT-AUDIT-015 agreed without checking that 168/180 filings were after-hours (≥16:00 ET) and only 12/180 were pre-open or intraday. When MiniMax-M3 (EXT-AUDIT-016) audited next, it corrected EXT-AUDIT-014 and proved that text-free baselines (MAE = 5.778% and 5.822%) beat Living_Lab_AI (5.832%). MiniMax-M3 also made 2 minor errors of its own: it left day1_return_pct on the right-hand side of a Day-1 CAR regression and mislabeled ICC(1,k) = 0.941 vs ICC(3,1) = 0.896. This time, Boston checked the math first, accepted the valid points, and rebutted the 2 errors with proof.6/10 wrong case IDs in EXT-AUDIT-017 and 2 fake git commit hashes (b9641dc, 3be8d74) in EXT-AUDIT-018. It also left unscripted numbers in the Referee Report (caught in SG-AUDIT-020) and a bad SHA-256 suffix in a handoff prompt (caught in SG-AUDIT-021).SKILL-027 (check_all_artifacts_provenance.py) blocks every one of those failure modes: Hardened after SG-AUDIT-020 and SG-AUDIT-021, SKILL-027 scans all 12 Study #02 files against VERIFY_DIRECTOR_FEEDBACK_S02_STDOUT.txt. It checks every signed decimal, sample integer (N, G, t, Section), AIW-YYYY-NN case ID, fraction, date, git commit via git cat-file -t, and 64-character SHA-256 hash.