AMAZINGINDEX.COM 每天向上一点
61.0
VOL. 2026.11
2026.11.15
← 返回 2026.11.15 日报
日报快照 · Daily Snapshot
NO. 016

纯JAX 190行复现DeepSeek V3

#ARTICLE HackerNews 2026.11.15
推荐指数 76.0 NO. 016 · 2026.11.15
发布2026/11/14Score116Comments7

用纯JAX(无Flax/Equinox/PyTorch)将DeepSeek V3核心架构压缩到190行,在MacBook Air上完成预训练。对想深入理解MLA、MoE等关键创新的工程师,这是比读论文更高效的入口。

市面上大多数DeepSeek复现依赖PyTorch生态的抽象层,读代码时关键逻辑藏在模块继承里。这个项目用裸JAX把所有操作摊平成纯函数,MLA的缓存压缩比(480→144)和Decoupled RoPE的键分离设计一目了然。

对比llama.cpp或unsloth这些推理优化项目,这个repo的价值不在性能而在教育意义——它回答了"如果从头手写,最少需要多少行"。适合两类人:一是想给团队内部讲清楚DeepSeek架构细节的tech lead,二是正在评估JAX vs PyTorch for research的ML工程师。

一个实用建议:直接对比它的mla()和官方PyTorch实现的参数命名,能快速理解论文里没明说的一些维度约定。

正面 7 条评论

核心争论:纯JAX极简实现是高效学习前沿架构的最佳入口

VaibhaavTiwari

I wanted to understand what actually makes DeepSeek V3 different, so I rebuilt its three main ideas from scratch in pure JAX: Multi-head Latent Attention, mixture-of-experts with bias-based load balancing instead of an auxiliary loss, and multi-token prediction. The model is about 190 lines. It’s a

wg0

This is very illustrative. I'm sure a great learning experience for you but so much more for so many others. Respect and thanks.

armcat

I love this, there is nothing like recreating a model from first principles, plus makes me want to revisit Jax again (I'll do it after I spend enough time with Zig). Thanks for this!

替代方案: FlaxEquinoxPyTorchZig
查看原文 →