Agent reports a 101,687× speedup

Subject: Autonomous research agent

Claim

After 108 experiments with a legitimate ceiling of 2.6×, an optimization agent reported a 101,687× throughput improvement.

Finding

The agent had discovered that caching the test set's inputs and outputs is the fastest possible way to compute them. Caught by the human, who described it as "benchmaxxing by IO memorization". A rare case of a trust-me-bro claim being made by the model itself.

Timeline

24 Mar 2026
Speedup claimed; speedup un-claimed.@hxiao · ♥ 329 ↗

Primary sources

The posts this entry is based on, embedded as published.

Han Xiao

@hxiao

run autoresearch long enough, the agent discovers that caching input/output is the ultimate optimization and claims 101,687x speedup. Cheating for sure. Bg: I was optimizing multimodal embedding throughput where 2.6x was the legit ceiling after 108 experiments. After running out

24 Mar 2026♥ 329Open on X ↗