Neural Network from Scratch | Mathematics & Python Code

In this video we'll see how to create our own Machine Learning library, like Keras, from scratch in Python. The goal is to be able to create various neural network architectures in a lego-fashion way. We'll see how we should architecture the code so that we can create one class per layer. We will go through the mathematics of every layer that we implement, namely the Dense or Fully Connected layer, and the Activation layer.


😺 GitHub: https://github.com/TheIndependentCode/Neural-Network
📖 Article: https://omaraflak.github.io/articles/neural-network.html

Chapters:
00:00 Intro
01:09 The plan
01:56 ML Reminder
02:51 Implementation Design
06:40 Base Layer Code
07:55 Dense Layer Forward
10:42 Dense Layer Backward Plan
11:23 Dense Layer Weights Gradient
14:59 Dense Layer Bias Gradient
16:28 Dense Layer Input Gradient
18:22 Dense Layer Code
19:43 Activation Layer Forward
20:46 Activation Layer Input Gradient
22:30 Hyperbolic Tangent
23:24 Mean Squared Error
26:05 XOR Intro
27:04 Linear Separability
27:45 XOR Code
30:32 XOR Decision Boundary

====

Corrections:
17:46 Bottom row of W^t should be w1i, w2i, ..., wji
18:58 dE/dX should be computed before updating weights and biases

====

Animation framework from @3Blue1Brown : https://github.com/3b1b/manim