# PyTorch-Architectures

### Note: This is a personal experimentation repo.

-----------------------------------------------------------------------------

| Name | Paper Link | Link to notebook |
| ---- | ---------- | ---------------- |
| LeNet | [Paper](http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_LeNet/test_sample_LeNet.ipynb) |
| AlexNet | [Paper](https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_AlexNet/test_sample_AlexNet.ipynb) |
| VGG16 | [Paper](https://arxiv.org/pdf/1409.1556.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_VGG16/VGG16-CIFAR10.ipynb) |
| DenseNet121 | [Paper](https://arxiv.org/pdf/1608.06993) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_DenseNet121/test_sample_DenseNet121.ipynb) |
| ResNet18 | [Paper](https://arxiv.org/pdf/1512.03385) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_ResNet/test_sample_ResNet18.ipynb) |
| ResNet34 | [Paper](https://arxiv.org/pdf/1512.03385)| [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_ResNet/test_sample_ResNet34.ipynb) |
| ResNet50 | [Paper](https://arxiv.org/pdf/1512.03385) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_ResNet/test_sample_ResNet50.ipynb?flush_cache=true) |
| ResNet101 | [Paper](https://arxiv.org/pdf/1512.03385) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_ResNet/test_sample_ResNet101.ipynb) |
| ResNet152 | [Paper](https://arxiv.org/pdf/1512.03385) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_ResNet/test_sample_ResNet152.ipynb?flush_cache=true) |
| NiN: Network in Network | [Paper](https://arxiv.org/pdf/1312.4400) | [Notebook](https://github.com/vishal-burman/PyTorch-Architectures/blob/master/modeling_NiN/test_sample_NiN.ipynb) |
| FRN: Filter Response Normalization | [Paper](https://arxiv.org/pdf/1911.09737) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_FRN/test_sample_FRN.ipynb) |
| MobileNetV1 | [Paper](https://arxiv.org/pdf/1704.04861) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_MobileNetV1/test_sample_MobileNetV1.ipynb) | 
| MobileNetV2 | [Paper](https://arxiv.org/pdf/1801.04381) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_MobileNetV2/test_sample_MobileNetV2.ipynb) |
| ShuffleNet | [Paper](https://arxiv.org/pdf/1707.01083) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_ShuffleNet/test_sample_ShuffleNet.ipynb) |
| ViT: Transformer for Image Recognition | [Paper](https://arxiv.org/pdf/2010.11929) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_ViT/test_sample_ViT.ipynb) |
| MLPMixer | [Paper](https://arxiv.org/pdf/2105.01601) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_MLPMixer/test_sample_MLPMixer.ipynb) |
| NNLM: Neural Language Model | [Paper](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_NNLM/test_sample_NNLM.ipynb) |
| Word2Vec | [Paper](https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_Word2Vec/test_sample_Word2Vec.ipynb) |
| FastText | [Paper](https://arxiv.org/pdf/1607.01759) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_FastText/test_sample_FastText.ipynb) |
| Seq2Seq | [Paper](https://arxiv.org/pdf/1409.3215) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_Seq2Seq/test_sample_Seq2Seq.ipynb) |
| Learning Phrase Representations using Seq2Seq | [Paper](https://arxiv.org/pdf/1406.1078) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_Seq2SeqPR/test_sample_Seq2SeqPR.ipynb) |
| TextRNN | [Paper](https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog1402_1) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_TextRNN/test_sample_TextRNN.ipynb) |
| TextCNN | [Paper](https://arxiv.org/pdf/1408.5882) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_TextCNN/test_sample_TextCNN.ipynb) |
| TextBiLSTM | [Paper](https://www.bioinf.jku.at/publications/older/2604.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_TextBiLSTM/test_sample_TextBiLSTM.ipynb) |
| BERT | [Paper](https://arxiv.org/pdf/1810.04805.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_BERT/test_sample_BERT.ipynb) |
| MLM: Masked Language Modeling(DistilBERT) | [Paper](https://arxiv.org/pdf/1810.04805.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/research/modeling_MLM/test_sample_MLM.ipynb) |
| OpenAI GPT | [Paper](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_OpenAI/test_sample_OpenAI.ipynb) |
| OpenAI GPT2 | [Paper](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_GPT2/test_sample_GPT2.ipynb) |
| Transformer XLM | [Paper](https://arxiv.org/pdf/1901.07291.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_XLM/test_sample_XLM.ipynb) |
| XLNet | [Paper](https://arxiv.org/pdf/1906.08237) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_XLNet/test_sample_XLNet.ipynb) |
| FNet | [Paper](https://arxiv.org/pdf/2105.03824) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_FNet/test_sample_FNet.ipynb) |
| Structured Self-Attentive Sentence Embedding | [Paper](https://arxiv.org/pdf/1703.03130) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_SSE/test_sample_SSE.ipynb) |
| TabTransformer | [Paper](https://arxiv.org/pdf/2012.06678) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_TabTransformer/test_sample_TabTransformer.ipynb) |
| BPR: Bayesian Personalized Ranking from Implicit Feedback | [Paper](https://arxiv.org/ftp/arxiv/papers/1205/1205.2618.pdf) | [Notebook](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/modeling_BPR/test_sample_BPR.ipynb) |

------------------------------------------------------------------------------------

## Miscellaneous:

| Name | Notebook Link |
| ---- | ------------- |
| Knowledge Distillation | [Notebook Link](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/misc/Knowledge_Distillation.ipynb) |
| T5 Add-Sub | [Notebook Link](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/misc/T5_Math_Add_Sub.ipynb) |
| T5 Unjumble | [Notebook Link](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/misc/T5Unjumble.ipynb) |
| T5 AutoComplete | [Notebook Link](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/misc/test_sample_T5AutoComplete.ipynb?flush_cache=true) |
| DistilBERT PT | [Notebook Link](https://github.com/vishal-burman/PyTorch-Architectures/blob/master/misc/test_sample_DistilBERT_PT.ipynb) |
| Image-Search(Dual Encoder) | [Notebook Link](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/misc/Image_Search%28Dual_Encoder%29.ipynb) |
| Metric Learning | [Notebook Link](https://github.com/vishal-burman/PyTorch-Architectures/blob/master/misc/Metric_Learning.ipynb) |
| FluentQ | [Notebook Link](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/misc/t5_fluentQA.ipynb) |
| Transformer-TimeSeries | [Notebook Link](https://nbviewer.jupyter.org/github/vishal-burman/PyTorch-Architectures/blob/master/misc/Transformer_TimeSeries.ipynb) |
