Recent Developments in Deep Learning
Why Hacker News recommends it
Cited as a watchable early introduction to deep learning from its pioneer, and remembered as a mind-blowing moment in the field.
What it is
Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.
Recommended by 3 HN commenters across 2 different months, 2010 to 2026.
What Hacker News said
Recommends Devilboy · 2012-06 · on “Building high-level features using large scale unsupervised learning”
Have you seen the online courses? [LINK] (From one of the authors of this paper!) [LINK] [LINK] Prof. Hinton's videos are very watchable: [LINK] [this video]
Recommends tylerhobbs · 2012-06 · on “Microsoft Research make breakthrough in audio speech recognition”
There's a very good Google Tech Talk by Geoff Hinton (who has worked closely with Dahl on a lot of this research and developed some of the key algorithms in this field) that explains how to build deep belief networks using layers of RBMs: [LINK] That video focuses on handwritten digit recognition, but it's great for understanding the basics. There's a second Google Tech Talk video from a few years later that talks directly about phoneme recognition as well: [this video]
Recommends markovbling · 2014-03 · on “Is Google Cornering the Market on Deep Learning?”
There's a great Google Tech Talk by Geoff Hinton that serves as a slightly technical introduction to Deep Learning if anyone's interested: [this video]
Cites Dn_Ab · 2010-11 · on “Jeff Hawkins talk on modeling neocortex and its impact on machine intelligence”
I would remove one maybe both muches. I agree that in general SVMs are better but they are also every bit as black box. Deep belief networks though, are on another level. literally, svms are depth 2, dbn's are depth unbounded, i think. [this video]
Cites utunga · 2012-07 · on “Machine Learning Video Library - Learning From Data (Abu-Mostafa)”
If you enjoyed Geoff Hinton's talk you will probably find the theano 'deep learning' library to be of use. Still undergoing quite a lot of iteration but powerful and you get to run your stuff on the GPU for added fun. [LINK] . Incidentally Hinton gave another google tech talk in 2010 [this video] .
Cites iandanforth · 2012-10 · on “Large Scale Visual Recognition Challenge 2011 - Results”
Hinton's team (SuperVision) uses an interesting 'dropout' technique. He gave a Google Tech Talk on this back in June. [LINK] And an older talk that covers some of what a deep convolutional net is: [this video]
Cites cs702 · 2013-03 · on “Geoffrey Hinton has joined Google”
Something really interesting must be happening with AI at Google, because in the past few months both Ray Kurzweil (the best-known proponent of the singularity) and Geoff Hinton (the crazy-talented individual who invented deep-belief networks using interconnected "restricted Boltzman machines"[1]) have joined the company. -- [1] For an overview of deep belief networks, see these videos: [LINK] , [this video] , and [LINK]
Cites zackmorris · 2015-01 · on “Research Priorities for Artificial Intelligence: An Open Letter”
I think the top priority should be how a hierarchy of weak AI agents can form strong AI with the ability to rewrite its own notions and not become unstable. I've seen very little about this, except for a video where (as I recall) a multilayer neural network can be trained while holding certain connections at a constant value. That way the network can be trained to do different tasks and then put back into those modes by setting those training connections to various values during runtime: [this video] Eventually another neural net could be trained to know which values to hold to get different behaviors. It sounds like one of the goals is to reuse logic for similar tasks, and eventually be able to