Building GPT-2 from Scratch: Day 3 - Demystifying Embeddings & Self-Attention (MLX Series)

Опубликовано: 13 Октябрь 2024
на канале: Ray Fernando

104

Coding GPT-2 with MLX.
Day 3 of Ray Fernando's GPT-2 from scratch series! Join this late-night coding session as Ray simplifies complex concepts for busy learners and AI enthusiasts using MLX.

🔗 Resources:
https://www.rayfernando.ai/day-3--cod...
https://tiktokenizer.vercel.app/
https://github.com/pranavjad/mlx-gpt2
   • Let's build GPT: from scratch, in cod...
   • Let's build the GPT Tokenizer

🤝 Connect with Ray:
Twitter:   / rayfernando1337
TikTok:   / rayfernando1337
Instagram:   / rayfernandojr

📚 VIDEO CHAPTERS
0:00 - Introduction and recap
5:37 - Diving into input embeddings
13:45 - Explaining tokens, vectors, and dimensions using the "magic backpack" analogy
20:58 - Positional embeddings explained
30:31 - Code implementation of embeddings
36:21 - Brief look at self-attention
37:48 - The importance of learning GPT architecture
42:14 - Preview of upcoming topics and daily commitment
47:35 - Bonus: Introduction to PAMIR's e-ink display device for edge computing

Don't forget to like, subscribe, and hit the notification bell to stay updated on this GPT from scratch series.

#GPTFromScratch #MachineLearning #AITutorial #MLX