Skip to content

About

Replicating Deep Learning Papers (Model Architectures)

Resources

Stars

3 stars

Watchers

1 watching

Forks

Repository files navigation

Deep Learning Model Paper-Replications

For Educational Purposes only

In this repository I will be replicating ML/DL/AI Papers and Model Architecture from Scratch using PyTorch (I may use Jax sometimes)

I will keep this file updated on how many models implemented so far.

I will be following a simple pipeline - Implement Block -> Test Block -> Repeat till model is implented -> Write Notes on understanding on why the block works and the intuition and maths behind it -> Test Train

TODO: I will be creating script files that will be accomodated in a final script so that the models can be accessed from a single file

So a package will be created COntemplating if I want to create a package or restructure this repo a bit

Activation Functions and Embedding:

  • Maxout
  • RMSNorm (Jax implementation as well)
  • RoPE

Vision Models:

  • [DDPM] - implementation of DDIM and DDPM is the same however only the noising process of the image is different. DDIM is faster.
    • Classifier free guidance
  • [cGAN]
  • [CLiP ViT]
  • [Vision Transfomer]

Text Models and LLMs:

  • [Transformer] - TODO: fix the error in the decoder cross attention mask
  • [Sparsely Gated MoE]
  • [xLSTM]
  • [Mamba] - State Space Model
  • [Titans: Memory as Context Implemented, MAL and MAG require SlidingWindow attention and so have been left]
  • [DeepSeek V3]
  • [Llama 3]

Models Trained and Tested So far:

  • cGAN
  • ViT

How to use?

Let us give a code snippet that follows the same structure for each model implemented:

To do ...

Authors:

About

Replicating Deep Learning Papers (Model Architectures)

Resources

Stars

3 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages