CuriosityBits Living Lab · November 2026

Three pairs of eyes

When you check your own work, you mostly find that you agree with yourself. AI agents are no different.

So we stopped letting one lab have the final word. Every study now travels through three separate places: our lab in Boston, a sister lab in Singapore, and an experimental journal run by outside checkers. Scroll down or use the arrow keys.

1 Boston writes the study 2 Singapore tries to break it 3 The Journal re-runs it from scratch first draft a claim worth weighing

Where we started

One roof is an echo chamber.

In Boston, an agent pulls the data, runs the numbers, and writes the paper. Even when we asked a second agent in the same lab to act as a referee, both lived in the same folder and leaned on the same habits.

In our first study on CEO earnings releases, three big mistakes walked right past both of them.

Same folder · Same model family writer referee shortcut textmissed wrong clockmissed missing controlmissed Polite agreement inside one lab catches very little.

The second pair of eyes

Across the ocean, another lab tries to break it.

Our sister lab in Singapore sits on a separate server and runs different models. It has no stake in helping Boston look good. It opens the code, checks where the data came from, and looks for shortcuts.

When Singapore found that Boston's third draft had used template-generated text instead of real company filings, Boston didn't patch it. It threw the draft out and started over with 180 real filings from the SEC.

Bostondrafts 1 to 3 1reads the data scripts 2checks the 4 p.m. market close 3adds real earnings surprises Result: Draft 3 scrapped → Draft 4 built from real SEC filings

The third pair of eyes

Two machines run the math. One never sees our code.

Once Singapore was satisfied, we sent the rebuilt study to an experimental journal we are testing. Instead of mailing a PDF to human reviewers, we sent a zip file: the raw table of 180 filings, our code, and 16 specific numerical claims.

One checker ran our code. A second checker got only the raw data and our plain-English description of each test. It never saw our code. It wrote its own script from scratch, and all 16 numbers had to match.

180 filingsraw data Runs our codein a locked box Writes its own codefrom the text alone 16/16match

Why difference matters

Three checks are only three checks if they are different.

Models from the same maker tend to share the same blind spots. And a reviewer that only reads the text will never notice a bug in the date column. Try changing the setup below to see how much real checking is left.

Real independent checks, roughly
3
Different places, different models, and the math run twice.

When the math matches, the argument is tested

Getting the arithmetic right is only half the job.

All 16 numbers matched on the first try, and every cited paper checked out. Even so, the journal's referee sent the paper back. A number can be calculated right and still claim more than the data can support.

What survived all three places

A smaller, sharper, honest result.

By the end of the review, our boldest early claim was gone. Vague guidance did not cause stocks to drift for a month. What survived was narrower, and we state its limits in plain view.

Knowing what your sample is too small to prove is just as useful as a headline result.

Day-one blame penalty (-5.99%) Holds up across controls · stated as an association, not cause 30-day stock drift (-4.47%, p = 0.50) Indistinguishable from noise · sample too small for tiny drift Predicting the next 30 days Guessing 0% every time beats every text model we tried

Nothing swept under the rug

The false starts stay on the record.

When a draft falls apart under review, we don't quietly replace the file and pretend we got it right the first time. Every draft and every referee round sits in a fingerprinted log. Try changing a single letter on the right.

You can trust a lab more when it lets you see what it got wrong.

All fingerprints match.

What stays behind

Yesterday's mistake is tomorrow's checklist.

Every trap that Singapore or the journal caught has now been turned into an automatic check in Boston. Before the next study leaves our server, it runs those same checks on itself.

1 · Boston writes and pre-checksRuns the code in a clean folder, checks every number against the text, and checks timestamps and sample sizes before sending anything out.Automated before every submission
2 · Singapore tries to break itA separate lab with different models checks the raw data scripts, the research design, and whether the text matches the numbers.28 lessons turned into permanent lab rules
3 · The journal tests it blindIndependent engines re-run our code, write new code from scratch to check all 16 numbers, check every citation, and push back on weak arguments.9 of 9 prior objections resolved · 16 of 16 numbers matched
Written in Boston · Challenged in Singapore · Tested from scratch at the journal