How DeepSeek Rewrote the Transformer [MLA]
Why Hacker News recommends it
HN commenters recommend it as one of the clearest visual explanations of attention and transformers in general, and as a pointer to DeepSeek's real architectural efficiency gain.
Recommended by 2 people across 2 different months, 2025 to 2026.
What it is
Welch Labs explains DeepSeek's Multi-Head Latent Attention (MLA), walking through multi-head, multi-query and grouped-query attention with animations before showing how MLA compresses the KV cache. It builds up the transformer attention mechanism visually, including a GPT-2 visualization, and covers the claimed KV cache reduction.
What people said
Strong rec credit_guy · 2025-09 · Hacker News · on “Understanding Transformers Using a Minimal Example”
Here's the best video I have seen about transformers [1]. It is made by Welch Labs. It talks about DeepSeek and what their main innovation was, but it covers transformers in general too, and I couldn't find any other better description of transformers. Also, here's an interactive "transformer explainer" that is absolutely mind-blowing [2]. [1] [this video] [2] [LINK]
Recommends karmakaze · 2025-05 · Hacker News · on “TransMLA: Multi-head latent attention is all you need”
I'm not "in the field" though I like to read about and use LLMs. This video "How DeepSeek Rewrote the Transformer [MLA]"[0] is really good at explaining MHA, MQA, GQA, and MLA with clear visuals/animations and how DeepSeek MLA is 57x more efficient. [0] [this video]
Cites karmakaze · 2025-04 · Hacker News · on “Recent AI model progress feels mostly like bullshit”
> [...] But I would nevertheless like to submit, based off of internal benchmarks, and my own and colleagues' perceptions using these models, that whatever gains these companies are reporting to the public, they are not reflective of economic usefulness or generality. [...] Seems like they're looking at how they fail and not considering how they're improving in how they succeed. The efficiency in DeepSeek's Multi-Head Latent Attention[0] is pure advancement. [0] [this video]
Cites andblac · 2026-09 · Hacker News · on “Transformers Explained Visually”
Look at this poster [1] (its low-res, full res is paid). Also see this video for context [2] - it talks about deepseek's innovation, but explains attention well. Above I was pointing to the moment where multiplication AV happens. In normal Dense layer in MLPs when you go through the layer you compute y=Wx, where x is an input and y is output (before gactivation) and W is a weight matrix. This W is usually what's produced through training process. This multiplication AV looks like Wx. If we take one column from matrix V in the poster and call it v, we can pretend for one moment that W=A and x=v [3]. So this multiplication Av works like linear transformation in ordinary network Wx. But in case