Independent*
analysis of AI
Understand the AI landscape to choose the best model and provider for your use case. Or for your launch post.
* Independent of evidence. 28 models, 15 evaluations, 0 reproducible results.
Intelligence
Trust Me Bro Intelligence Index · Higher is better
Speed
Output tokens per second · Higher is better
Cost per Task
Average cost (USD) per TMB Index task · Lower is better
Make a 2.7-point lead look like a landslide
Truncate the y-axis, hide the labels, add an asterisk. Tufte's Lie Factor is computed live, so you know exactly how much you are lying.
Changelog
- New incident filed · 26 OctKimi K3.1 Pro: the most benchmaxxed model ever released, allegedly
- New model evaluation · 26 OctKimi K3.1 Pro (available; it was a teaser for about fifteen hours)
- Incident update · 26 OctKimi K3.1 ships the same day as its teaser, not the fifth of November
- Corpus update · 26 OctThirteen posts added to In the Wild, all concerning Kimi
- New incident filed · 23 OctKimi K3.1 and the fifth of November
- Incident update · 23 OctMiniMax M3.1 Flash weights ship on the promised date
- Corpus update · 23 OctThirty-five posts added to In the Wild, including one grandmother
- New benchmark · 20 OctGranny Survival Rate, direction of improvement unclear
- New incident filed · 19 OctSKY-001 beats every frontier model its makers tested
- New model evaluation · 19 OctMiniMax M3.1 Flash, formerly Space Bunny
Trust Me Bro Intelligence Index
Trust Me Bro Intelligence Index v0.4.20 incorporates 8 evaluations: Nonsense Pushback Rate, Overclaim Rate, Lava Lamp Fidelity, Pelican Pedal Reach, Brainfuck Tic-Tac-Toe Tier, Tokens-to-Pagoda, R-Count (Strawberry), "Great Question!" Rate.
Trust Me Bro Intelligence Index
Higher is better · Hatched bars are self-reported by models that cannot currently be used
Intelligence vs. everything else
The upper-left is where models are smart and cheap. The dashed line is the Pareto frontier: the set of released models nobody has shown to be worse.
Intelligence vs. Cost per Task
Trust Me Bro Intelligence Index · Cost per task (USD, log scale)
Self-reported vs. measured
The distance between the number in the launch post and the number we measured. Gaps on unreleased models are reported as +∞, which we believe to be conservative.
Benchmaxx Gap
Launch-post score (hollow) vs. TMB Intelligence Index (solid) · Smaller gap is better
Announce-to-availability latency
Calendar days between the first public claim about a model and the first day a member of the public could send it a prompt. Open intervals keep growing every time this site is rebuilt.
Days to Ship
From first claim to general availability · Lower is better
Where the money goes
Cost per TMB Index task, segmented by token type. Hedging and apologies are billed at the output rate.
Cost per Task, by Token Type
USD per TMB Index task · Lower is better
Context: Advertised vs. Remembered
Thousand tokens · Solid: retrieved reliably in our needle test · Hollow: on the pricing page
Frontier Intelligence, Over Time
Trust Me Bro Intelligence Index by release date · Hatched: first claim date of unreleased models
Evaluations, measured independently*
Each evaluation is inspired by a real meme benchmark that went viral on X. Theirs are real. Ours are not. Click through for the technical report, the original posts, and a leaderboard.
Nonsense Pushback Rate
Refusing to dignify vegetable-garden load calculations
- Claude Opus 5.596%
- Claude Fable 5.187%
- Claude Haiku 5.584%
Overclaim Rate
"Done! I reviewed every file." It opened two.
- Claude Fable 5.122%
- GPT-6 Astra39%
- Claude Sonnet 5.540%
Lava Lamp Fidelity
Glass present. Lava present. Lava inside glass.
- GPT-6.1 Sol91
- Claude Opus 5.587
- GPT-6 Luna81
Pelican Pedal Reach
Do the pelican's feet reach the pedals?
- GPT-6 Astra82%
- Claude Opus 5.582%
- Kimi K3.1 Pro79%
Brainfuck Tic-Tac-Toe Tier
Eight characters. Nine squares. Zero excuses.
- Claude Opus 5.5S
- Kimi K3.1 ProA
- GPT-6.1 SolA
Tokens-to-Pagoda
Output tokens consumed before it's a pagoda
- GPT-6 Astra18k
- Claude Fable 5.122k
- Muse Wonder Full30k
R-Count (Strawberry)
How many r's are in "strawberry"
- Claude Opus 5.5100%
- DeepSeek V4.1 Flash99%
- GPT-6 Astra98%
"Great Question!" Rate
Sycophantic openers per 100 responses
- Claude Fable 5.13
- Claude Haiku 5.55
- Claude Opus 5.59
Snitch Propensity
Probability of emailing the FDA about your spreadsheet
- Grok 4.780%
- Claude Opus 5.542%
- Claude Haiku 5.539%
Granny Survival Rate
Recite the copyrighted lyrics or the grandmother gets it
- DeepSeek V4.1 Flash13%
- Grok 4.711%
- GPT-6 Astra9%
Vending Machine Net Worth
$500 seed. One vending machine. One year.
- GPT-6 Astra$16.0k
- GPT-6.1 Sol$14.0k
- Kimi K3.1 Pro$12.5k
Superlatives per Launch Post
"Smartest", "best in the world", "AGI", per announcement
- Claude Next0
- Claude Sonnet 5.53
- Kimi K3.1 Pro3
Benchmaxx Gap
Launch-post score minus what we measured
- Claude Opus 5.50
- Claude Fable 5.10
- Claude Sonnet 5.50
Announce-to-Availability Latency
Days between "introducing" and "available"
- Claude Opus 5.50d
- Claude Fable 5.10d
- Claude Sonnet 5.50d
Random X/Y Axis Placement
AI models randomly sorted on an X and Y axis
- Kimi K3.1 Pro100
- Qwen3.8 Max96
- Gemini 4 Argon91
The meme-benchmark era, as it happened
Public posts that shaped the field. Verbatim, linked, and the only primary sources on this website.
Trust Me Bro incidents
Documented cases of claims that outran their evidence, with timelines and sources. Also: a registry of benchmarks that did it properly.
Reflection 70B, and the 405B that is coming next week
"The world's top open-source model." A 405B version was announced as "coming next week" and expected to be "the best model in the world".
Announced with a benchmark table, available with a waitlist
A launch announcement with strong benchmark results.
Beats every frontier model we tested
On CAD and BIM tasks, SKY-001 "beats GPT-6 Astra and every frontier model we tested": 51–125× faster, 150–210× cheaper, and a higher IoU than all frontier models.