Large Language Models from scratch
Why Hacker News recommends it
HN recommends it as a quick, non-dumbed-down way to get up to speed on how LLMs are built.
What it is
A short cartoon explaining how language models like ChatGPT work, from autocomplete and conditional probabilities to neural networks, gradient descent and backpropagation. It shows how networks act as universal function approximators.
Recommended by 3 HN commenters across 2 different months, 2023 to 2024.
What Hacker News said
Strong rec sainez · 2023-04 · on “A Completely Non-Technical Explanation of Deep Learning”
If anyone is looking for a quick overview of how LLMs are built, I highly recommend this video by Steve Seitz: [this video] . It does an excellent job of taking you from 0 to a decent understanding without dumbing down the content or abusing analogies.
Strong rec peter_d_sherman · 2023-05 · on “Large Language Models from scratch (2022)”
Consider the following frame/time from the video: [this video] At this point in time in the video, we see the following 3 things: 1) Fourier Series 2) Taylor Series 3) Neural Network The remainder of the video goes on to show how Neural Networks can approximate functions -- much like the Fourier Series and Taylor Series can... Now, I don't hold an advanced Ph.D. degree in Math, and prior to watching this video, I had never made the connection between Fourier Series, Taylor Series, and Neural Networks... I knew about each of those independently -- but prior to watching the video, I never made the connection that they can be used to approximate one another. And that's the one of the key words -- ap
Recommends runetech · 2023-05 · on “Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper”
Not sure I can give an ELI5 explanation, but for me this the closest to what you are after in two short videos on the topic. At least they helped me get up to speed fast. Hope they will do the same for you ;-) Large Language Models from scratch [this video] Large Language Models: Part 2 [LINK]
Cites mistermann · 2023-01 · on “Globalization Is Dead and No One Is Listening”
> What is so unserious, disingenuous, and illegitimate about the “system”? Some examples (I could write indefinitely): [LINK] (section "Bank Error In Our Favor" is particularly offensive, to me) [LINK] (this sort of thing (which is fairly common) is offensive to me, as is our culture's complacency towards it) How it is (in part) perpetuated, in my estimation: - I’m sure whatever the system is it isn’t perfect. But here I am using the “system” and living my life. [rationalization] - I see so many strange comments on the internet that sound like “red-pilled” nonsense. [training-based heuristic pattern matching to ~bad/dumb/wrong] - Could it be better and is there room for improvement? Absolute
Cites mistermann · 2024-03 · on “The Starship or the Canoe: Where will our future adaptations be? (2017)”
(Note: I am a different Human than the one you are replying to - apologies for interjecting.) > "Physically preventable" is a moving target. So are rockets and artillery shells, but check this out: [LINK] Humans can be very clever and generous ($60,000 USD per interceptor missile, hoo baby!! Is "democracy" not the greatest thing ever!!!???) when they set their minds to something. Unfortunately, delusion, untruthfulness, deceit, misdirection, etc are also somethings , and apparently very desirable somethings, seemingly irresistible to some. > The more resources are invested in preventing currently preventable deaths, the more farther away the target moves, because more and more causes of deat