Comprehensive Guide to Building Neural Networks from Scratch with Python

When I started learning Neural Networks from scratch a few years ago, I did not think about just looking at some Python code or similar. I found it quite hard to understand all the concepts behind Neural Networks (e.g. Bias, Backpropagation, ...). Now I know that it all looks quite more complicated when you see it written mathematically compared to looking at the code. In this video, I try to provide you an intuitive understanding through Python code and detailed animations. Hope it helps you :)

Code:
https://github.com/Bot-Academy/NeuralNetworkFromScratch

Find me on:
Patreon: https://www.patreon.com/botacademy
Discord: https://discord.gg/6fRE4DE
Twitter: https://twitter.com/bot_academy
Instagram: https://www.instagram.com/therealbotacademy/

Citation:
[1] https://www.datasciencecentral.com/m/blogpost?id=6448529%3ABlogPost%3A489568

Additional Notes:
1. You might’ve seen that we haven’t used the variable e at all.
This is for two reasons. First, normally we would’ve used it to calculate ‘delta_o’,
but due to some tricks, it is not needed here. Second, it is sometimes helpful to print the average error during training to see if it decreases.

2. To see how it performs on images not seen during training, you could only use just the first 50000 images for training and then analyze the results on the remaining 10000 samples. I haven’t done it in this video for simplicity. The accuracy, however, shouldn’t change that much.

3. It seems like some people have a hard time understanding the shape lines [e.g. x.shape += (1,)]. So let me try to explain:

To create a 1-tuple in python we need to write x=(1,). If we would just write x=(1), it gets converted to the integer 1 in Python.

Numpy introduces the shape attribute for arrays. Because the shape of a matrix has to be represented by a tuple like (2, 5) or (2, 4, 7), it is a good idea to represent a vector as a 1-tuple instead of an integer for consistency. So it is (X,).

If we want to use this vector in a matrix multiplication with a matrix, it doesn't work because you can't matrix multiply a vector with a matrix in numpy. So we need to add this 'invisible' second dimension of size 1. The line basically adds a (1,) vector to the shape of the (X,) vector which results in a matrix of size (X, 1). That's also why it doesn't work with (2,) because that would require more values. For example (5,) and (5, 1) both contain 5 values while (5, 2) would contain 10 values.

I should've shown the shapes in the shape information box as (X,) instead of just X. I think that also made it more confusing.

Credits:
17.08 - End
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Music: Ansia Orchestra - Hack The Planet
Link: https://youtu.be/fthcBrJY5eg
Music provided by: MFY - No Copyright
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

The animations are created with a python library called manim. Manim was first created by Grant Sanderson also known as 3blue1brown (YouTube) and is now actively developed by the manim community. Special thanks to everyone involved in developing the library!
Github: https://github.com/manimcommunity/manim

Contact: smarter.code.yt@gmail.com

Chapters:
00:00 Basics
02:55 Bias
04:00 Dataset
05:25 One-Hot Label Encoding
06:57 Training Loops
08:15 Forward Propagation
10:22 Cost/Error Calculation
12:00 Backpropagation
15:30 Running the Neural Network
16:55 Where to find What
17:17 Outro