For Educational Purposes only
In this repository I will be replicating ML/DL/AI Papers and Model Architecture from Scratch using PyTorch (I may use Jax sometimes)
I will keep this file updated on how many models implemented so far.
I will be following a simple pipeline - Implement Block -> Test Block -> Repeat till model is implented -> Write Notes on understanding on why the block works and the intuition and maths behind it -> Test Train
TODO: I will be creating script files that will be accomodated in a final script so that the models can be accessed from a single file
So a package will be createdCOntemplating if I want to create a package or restructure this repo a bit
- Maxout
- RMSNorm (Jax implementation as well)
- RoPE
- [DDPM] - implementation of DDIM and DDPM is the same however only the noising process of the image is different. DDIM is faster.
- Classifier free guidance
- [cGAN]
- [CLiP ViT]
- [Vision Transfomer]
- [Transformer] - TODO: fix the error in the decoder cross attention mask
- [Sparsely Gated MoE]
- [xLSTM]
- [Mamba] - State Space Model
- [Titans: Memory as Context Implemented, MAL and MAG require SlidingWindow attention and so have been left]
- [DeepSeek V3]
- [Llama 3]
- cGAN
- ViT
Let us give a code snippet that follows the same structure for each model implemented:
To do ...