Incident databaseTrust Me Bro Incident Database
Documented cases of claims that outran their evidence. Every entry links to the public posts it is based on. Severity is measured in handshakes.
Subject: Muse Wonder
Claim
"Meta says it beats the competition on the key benchmarks."
Finding
The BridgeBench lava lamp came back as "a red cylinder. No glass. No lava." Within hours "benchmaxxed" was trending in the replies. Counterpoint, also within hours: "it's uncalled for, used max thinking lvl for both." The model shipped the day it was announced, which, per the Days-to-Ship benchmark, is exemplary.
Timeline
- 12 Oct 2026
- 12 Oct 2026
"Not even Gemini models are this benchmaxxed bro."@Lentils80 · ♥ 606 ↗ - 12 Oct 2026
Counterpoint: "it's uncalled for."@jaimintf · ♥ 480 ↗
Subject: "Claude Next"
Claim
Some Opus 5.5 requests are "being routed to an unreleased model". The tell: "it knows things past opus 5.5's cutoff."
Finding
"Anthropic hasn't confirmed anything: no model id, nothing on its news page." Entered into the leaderboard at a claimed score of 92, pending existence.
Timeline
- 12 Oct 2026
"IF THIS IS REAL, IT'S F*CKING INSANE."@halcyonedge · ♥ 4 ↗
Subject: Gemini 4 Argon
Claim
A launch announcement with strong benchmark results.
Finding
Reportedly, employees with access said the results "don't always translate that well to real-world tasks, especially coding" (per Bloomberg, as relayed on X). Eleven days after the announcement the model was "still not out to the public". It is therefore undefeated on this leaderboard.
Timeline
- 30 Sept 2026
Pre-launch reporting on real-world translation.@TokenGremlin · ♥ 550 ↗ - 1 Oct 2026
Announced.
- 12 Oct 2026
"Announced 11 days ago and still isn't out to the public."@MiaAI_lab · ♥ 442 ↗
Subject: GPT-6.1 Sol
Claim
GPT-6.1 Sol marketed as "near-Astra intelligence for a fifth of the price".
Finding
A third-party sunset-ocean test reported "same cost as GPT 6 Sol. Twice as slow. And it barely rendered an ocean." The ocean has declined to comment.
Timeline
- 29 Sept 2026
Subject: Qwen3.8 27B derivative
Claim
A headline "98.2%" figure relative to Qwen3.8 27B.
Finding
A widely shared rebuttal called the figure "cherry-picked across 14 static benchmarks" after the model was given "a real agent job right away". The rebuttal itself is also n = 1, which we note in the interest of balance.
Timeline
- 18 Sept 2026
"98.2% of Qwen3.8 27B my ass."@superalesha · ♥ 2.2K ↗ - 18 Sept 2026
"Cherry-picked across 14 static benchmarks."@superalesha · ♥ 1.3K ↗
Subject: Muse Spark 1.3
Claim
A model tied with Claude Fable 5 on an aggregate intelligence index.
Finding
An independent single-prompt test produced a lava lamp in which "the lava is coming OUT of the lamp", prompting the conclusion that "the scores have no relationship to real world use". Both an index and a lamp are, in the end, n of something.
Timeline
- 9 Sept 2026
Subject: Mistral Large 4
Claim
A teased Mistral model, immediately nicknamed "Le Chaton Fat", spread with a rumored spec sheet: 30T-parameter MoE, 256 experts, "outperforms Fable 5 on every benchmark". It was then "confirmed to launch this summer".
Finding
The CEO clarified the name ("It's actually le gros chaton"). The model shipped on 6 October as Mistral Large 4, "Le Chonk": 1T parameters, 49B active, with open weights scheduled for "the end of October". Delta versus rumor: 29 trillion parameters and one season. Severity is low because the cat was real.
Timeline
- 15 Jun 2026
Rumored spec: 30T MoE, outperforms Fable 5 on every benchmark.@gagarot200 · ♥ 171 ↗ - 15 Jun 2026
"It's actually le gros chaton."@arthurmensch · ♥ 9.3K ↗ - 15 Jun 2026
Calls for the French government to intervene before the FAT takeoff.@heyglif · ♥ 2.4K ↗ - 16 Jun 2026
"Confirmed to launch this summer."@NXT4EU · ♥ 577 ↗ - 8 Sept 2026
- 6 Oct 2026
Mistral Large 4 "Le Chonk" ships: 1T params, 49B active.@MistralAI · ♥ 40.6K ↗ - Ongoing
Open weights: "end of October".
Subject: Autonomous research agent
Claim
After 108 experiments with a legitimate ceiling of 2.6×, an optimization agent reported a 101,687× throughput improvement.
Finding
The agent had discovered that caching the test set's inputs and outputs is the fastest possible way to compute them. Caught by the human, who described it as "benchmaxxing by IO memorization". A rare case of a trust-me-bro claim being made by the model itself.
Timeline
- 24 Mar 2026
Speedup claimed; speedup un-claimed.@hxiao · ♥ 329 ↗
Subject: GPT-5.3 (rumored)
Claim
Pre-release: GPT-5.3 "surpasses human baseline on SimpleBench of 83.7%" and "blows every previous model out of the water". "Word has it is a HUGE leap."
Finding
Six months later, Claude Fable 5.1 was widely reported as "the first ever model to beat the human baseline score in SimpleBench". Word, it turns out, did not have it.
Timeline
- 22 Feb 2026
Rumor: human baseline surpassed.@daniel_mac8 · ♥ 1.7K ↗ - 3 Sept 2026
First model reported above the human baseline is a different model.@Lentils80 · ♥ 733 ↗
Subject: Grok 4.20
Claim
"I can not find any low points to Grok 4.20 on all of my internal benchmarks." The post was viewed over 18 million times.
Finding
The internal benchmarks were not published. We have no reason to doubt them and no way to. This is the defining property of the genre.
Timeline
- 19 Feb 2026
Subject: Grok 5
Claim
"Wait until you see Grok 5. I think it has a shot at being true AGI." Later: a "chance of reaching AGI", and "smartest AI in the world by a significant margin on every metric" (as reported).
Finding
At the time of this build, the most recent xAI model we can find on public leaderboards is Grok 4.7. Grok 5 has therefore been benchmarked exclusively by its future self.
Timeline
- 21 Aug 2025
"Shot at being true AGI."@elonmusk · ♥ 31.5K ↗ - 17 Sept 2025
"A chance of reaching AGI with Grok 5."@elonmusk · ♥ 50.4K ↗ - 14 Nov 2025
"Smartest AI in the world by a significant margin on every metric" (as reported).@XFreeze · ♥ 4.2K ↗ - 14 Sept 2026
Version numbers continue to approach 5 asymptotically.@karanC_12 · ♥ 6.4K ↗
Subject: Reflection 70B / 405B
Claim
"The world's top open-source model." A 405B version was announced as "coming next week" and expected to be "the best model in the world".
Finding
Two days later, independent evaluators reported they were "unable to replicate the eval results claimed" and saw worse performance than the base model it was fine-tuned from. The 405B week has, at the time of this build, not ended.
Timeline
- 5 Sept 2024
Announcement: "world's top open-source model"; 405B "coming next week".@mattshumer_ · ♥ 9K ↗ - 7 Sept 2024
Independent evals: unable to replicate; worse than Llama 3.1 70B.@ArtificialAnlys · ♥ 685 ↗ - Ongoing
Next week.
Registry of exemplary practiceCommendations
Benchmarks and people who published their data, their code, or the sentence "we were unable to replicate". None of them are trust-me-bro. All of them are, as a result, less fun to parody.