Kimi K3 开源 2.8T 参数 MoE 架构
推荐指数 78.0 NO. 008 · 2026.07.29
发布2026/07/28Score125Comments13
为什么值得看
月之暗面开源了目前最大的开源权重模型 Kimi K3，参数量从去年的 48B 扩展到 2.8T，核心创新是引入 LatentMoE 压缩大规模线性层。对需要部署超大规模模型的团队有参考价值，但 2.8T 参数的实际推理成本仍需评估。
编辑判断
LatentMoE 并非 Kimi 原创，而是直接复用了 Nemotron 3 Ultra 的相同设计，这说明超大规模 MoE 的架构收敛正在加速。真正值得关注的是 2.8T 参数下如何保持推理效率——如果 LatentMoE 的压缩比不够激进，实际 serving 成本可能接近闭源 API 价格。
对于想做私有化部署的创业公司，建议先等社区的量化方案和 vLLM 适配进度，2.8T 的显存需求在没有优化前几乎不可承受。Kimi 这次开源更像是秀肌肉而非实用工具，实际落地可能要看后续的蒸馏版本。
社区反馈
意见分歧 12 条评论
核心争论：K3 的 NoPE 架构能否在超大规模下成立，以及活跃参数与总参数的实际差距
相关内容
Interesting that they went NoPE everywhere — everyone else hedges with RoPE in the local layers. Feels like the linear-attention stuff (Kimi Delta) is quietly doing the positional work so they can get away with it. Curious to see if it holds up at frontier scale.
Is frontier scale larger than this? Kimi K3 seems to benchmark in the same range as Opus and Fable. I would have expected they are all in the 2-4T range, with quality of the training and architecture differences as the major differentiators
The number of active parameters is vastly different. Deepseek CEO hinted that he estimates it as an order of magnitude difference in one of his recent interviews. > Seems to benchmark yes, but in human usage the differences show up