Road to GPT Documentation
About Me¶
I am Muhammed Muzzammil Shah, currently working as a Junior Software Engineer at a startup. My primary focus is on implementing Artificial Intelligence and Machine Learning solutions.
If you would like to learn more about me, please feel free to visit my official website!
Purpose of this site¶
I created this platform to serve as a repository for all my notes and practical implementations related to everything I have learned on my journey toward implementing the GPT model.
My Learning path follows Andrej Karpathy's Neural Networks: Zero to Hero series.
Documentation structure¶
I have divided my learning resources into two sections: Set-1 and Set-2.
Set-1 will focus on Language Modelling Framework
| Path | Implementation of | Link | Lecture Status | Documentation Status |
|---|---|---|---|---|
| Backpropagation | Micrograd | Video Lecture | Done | Done |
| Language Model-1 | Makemore (Part 1) | Video Lecture | Done | Done |
| Language Model-2 | Makemore (Part 2) | Video Lecture | Done | Done |
| Language Model-3 | Makemore (Part 3) | Video Lecture | Done | Done |
| Language Model-4 | Makemore (Part 4) | Video Lecture | Ongoing | - |
| Language Model-5 | Makemore (Part 5) | Video Lecture | - | - |
Set-2 will focus on the Transformer Neural Network
| Path | Implementation of | Link | Lecture Status | Documentation Status |
|---|---|---|---|---|
| Transformer Model-1 (GPT) | Build GPT | Video Lecture | - | - |
| LLMs | Large Language Models | Video Lecture | - | - |
| Tokenizers | Build GPT Tokenizer | Video Lecture | - | - |
| Transformer Model-2 (GPT) | Reproducing GPT-2 | Video Lecture | - | - |
Reflections and Usage Guidelines¶
Note from the author
-
This site serves as a personal knowledge base that I can return to anytime I need to revise, understand, demo, or explain anything.
-
Feel free to use this site as a reference during your own learning journey!
-
If you find it helpful or notice any corrections that need to be made, don't hesitate to reach out.
Warning
Please refrain from directly copying the contents of this site for any commercial purposes. Although the primary lectures it draws from are open-source and freely available online (thanks to Andrej Karpathy), doing so would not be fair or ethical.