Backpropagation calculus | Deep Learning Chapter 4

What Hacker News said

Recommends nouveaux · 2017-11 · on “What is backpropagation and what is it doing? [video]”

This was very timely for me and for anyone else learning, here are the first few videos of the series: [LINK] [LINK] [LINK] (Original video) [this video]

Recommends hal9000xp · 2017-11 · on “Ask HN: Best way to get started with AI?”

I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must. A few months ago, I stumbled upon very amazing YouTube Channel 3Blue1Brown which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus. Just recently he published 4 videos about deep neural networks: [LINK] [LINK] [LINK] [VIDEO

Recommends melvinroest · 2020-06 · on “From scratch: reverse-mode automatic differentiation (in Python)”

I'm just an enthusiast. With that said, I think the author's first image image really captures the core of forward propagation and back propagation. For people who want to get more of a feel on autodiffing, I find these 2 sources amazing [1, 2]. They don't use the word autodiffing, but looking at it, I can see it's the same thing. I once made an attempt at animating feed forward propagation [3]. If I'd animate back propagation, I'd probably make an animation on autodiffing as well as that is the most intuitive way I know on how to explain back propagation right now. It's still on my big to do list ;-) [1] [LINK] [2] 3Blue1Brown does in episode 4 of his Deep Learning series: [this video] [3] [LINK

Cites seanhunter · 2024-10 · on “What P vs. NP is about”

It's not satire. If you have some function with several inputs, the Jacobian is a matrix of all the partial derivatives of this function with respect to each input variable[1]. Since derivatives give the slope of a function, if you think about your function as being like a bumpy surface with the height at each point being the output, this matrix tells you which way (and how far) to change any input if you want to go "up" or "down" in the output. Backpropogation is a way to optimise a neural network. You want to know how best to nudge the weights of the network to optimise some loss function, so what you do is compute the gradient (ie partial derivative) of that function with respect to each