Fable 5 思维基准测试引争议
推荐指数 64.0 NO. 015 · 2026.09.22
发布2026/09/21Score296Comments199
为什么值得看
AI 基准测试平台 Fable 5 的"Median thinking"指标在 8 月出现下降，引发 HN 社区 199 条评论激辩。这暴露了当前 AI 能力评估体系的脆弱性，做模型选型的工程师需要警惕单一指标的误导。
编辑判断
Fable 5 这类第三方评测平台正在成为模型厂商的必争之地，但它的评分波动往往反映的是评测方法本身的迭代，而非模型能力的真实退化。之前 LMSYS 的 Chatbot Arena 也经历过类似的公信力危机，后来靠引入人类专家盲测才稳住。
对做模型选型的团队来说，关键不是看某个指标的月度涨跌，而是理解这个指标到底测了什么——"Median thinking"的题集构成、评分标准、是否对特定模型过拟合，这些信息比分数本身更重要。
建议把 Fable 5 当作参考系之一，而不是决策依据，同时自建核心业务场景的私有评测集。
社区反馈
负面 173 条评论
核心争论：模型性能波动是真实退化还是用户感知偏差，厂商是否存在暗中降质
I seem to recall Anthropic going on record saying that they don't do anything to model performance to stretch their compute capacity. I've anecdotally noticed massive peaks and troughs in performance week to week (albeit with Opus, not Fable). I wonder what their official explanation for this behavi
Last time they were called out, it was a regression in Claude code itself. At least that's their explanation. Either way, it wasn't a good look for "vibecoding" but it got brushed over.
When something is new, its capabilities feel incredible. Over time, those same capabilities become mundane, and you start to notice the flaws. (Now, if TFA is actually measuring reasoning tokens, that's quite different! It's not entirely obvious to me how he is measuring.)