Transformers Explained: The AI Behind ChatGPT and More 🤖

Transformer Neural Networks are the heart of pretty much everything exciting in AI right now. ChatGPT, Google Translate and many other cool things, are based on Transformers. This StatQuest cuts through all the hype and shows you how a Transformer works, one-step-at-a time.

NOTE: If you're interested in learning more about Backpropagation, check out these 'Quests:
The Chain Rule: https://youtu.be/wl1myxrtQHQ
Gradient Descent: https://youtu.be/sDv4f4s2SB8
Backpropagation Main Ideas: https://youtu.be/IN2XmBhILt4
Backpropagation Details Part 1: https://youtu.be/iyn2zdALii8
Backpropagation Details Part 2: https://youtu.be/GKZoOHXGcLo

If you're interested in learning more about the SoftMax function, check out:
https://youtu.be/KpKog-L9veg

If you're interested in learning more about Word Embedding, check out: https://youtu.be/viZrOnJclY0

If you'd like to learn more about calculating similarities in the context of neural networks and the Dot Product, check out:
Cosine Similarity: https://youtu.be/e9U0QAFbfLI
Attention: https://youtu.be/PSs6nxngL6k

For a complete index of all the StatQuest videos, check out:
https://statquest.org/video-index/

If you'd like to support StatQuest, please consider...

Patreon: https://www.patreon.com/statquest
...or...
YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
https://statquest.org/statquest-store/

...or just donating to StatQuest!
https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
https://twitter.com/joshuastarmer

0:00 Awesome song and introduction
1:26 Word Embedding
7:30 Positional Encoding
12:53 Self-Attention
23:37 Encoder and Decoder defined
23:53 Decoder Word Embedding
25:08 Decoder Positional Encoding
25:50 Transformers were designed for parallel computing
27:13 Decoder Self-Attention
27:59 Encoder-Decoder Attention
31:19 Decoding numbers into words
32:23 Decoding the second token
34:13 Extra stuff you can add to a Transformer

#StatQuest #Transformer #ChatGPT