Gradient descent, how neural networks learn | Chapter 2, Deep learning

Enjoy these videos? Consider sharing one or two.
Help fund future projects: https://www.patreon.com/3blue1brown
Special thanks to these supporters: http://3b1b.co/nn2-thanks
Written/interactive form of this series: https://www.3blue1brown.com/topics/ne...

This video was supported by Amplify Partners.
For any early-stage ML startup founders, Amplify Partners would love to hear from you via [email protected]

To learn more, I highly recommend the book by Michael Nielsen
http://neuralnetworksanddeeplearning....
The book walks through the code behind the example in these videos, which you can find here:
https://github.com/mnielsen/neural-ne...

MNIST database:
http://yann.lecun.com/exdb/mnist/

Also check out Chris Olah's blog:
http://colah.github.io/
His post on Neural networks and topology is particular beautiful, but honestly all of the stuff there is great.

And if you like that, you'll love the publications at distill:
https://distill.pub/

For more videos, Welch Labs also has some great series on machine learning:

• Learning To See [Part 1: Introduction]

• Neural Networks Demystified [Part 1: ...

"But I've already voraciously consumed Nielsen's, Olah's and Welch's works", I hear you say. Well well, look at you then. That being the case, I might recommend that you continue on with the book "Deep Learning" by Goodfellow, Bengio, and Courville.

Thanks to Lisha Li (@lishali88) for her contributions at the end, and for letting me pick her brain so much about the material. Here are the articles she referenced at the end:
https://arxiv.org/abs/1611.03530
https://arxiv.org/abs/1706.05394
https://arxiv.org/abs/1412.0233

Music by Vincent Rubinetti:
https://vincerubinetti.bandcamp.com/a...

-------------------
Video timeline
0:00 - Introduction
0:30 - Recap
1:49 - Using training data
3:01 - Cost functions
6:55 - Gradient descent
11:18 - More on gradient vectors
12:19 - Gradient descent recap
13:01 - Analyzing the network
16:37 - Learning more
17:38 - Lisha Li interview
19:58 - Closing thoughts
------------------