

## Efficient Architecture of LLM
| Title & Authors | Introduction | Links |
|:--|  :----: | :---:|
|[Efficient Hybrid Inference for LLMs: Reward-Based Token Modelling with Selective Cloud Assistance](https://arxiv.org/abs/2409.13757) <br> Adarsh MS, Jithin VG, Ditto PS @Bud Ecosystem |<img width="1002" alt="image" src="figures/selective-cloud-llm-assistance.png"> |[Paper](https://arxiv.org/pdf/2409.13757)|
|[![Star](https://img.shields.io/github/stars/YuchuanTian/RethinkTinyLM.svg?style=social&label=Star)](https://github.com/YuchuanTian/RethinkTinyLM)<br>[Rethinking Optimization and Architecture for Tiny Language Models](https://arxiv.org/abs/2402.02791) <br> Yehui Tang, Fangcheng Liu, Yunsheng Ni, Yuchuan Tian, Zheyuan Bai, Yi-Qi Hu, Sichao Liu, Shangling Jui, Kai Han, Yunhe Wang |<img width="1002" alt="image" src="https://github.com/YuchuanTian/RethinkTinyLM/blob/master/fig/improve.png"> |[Github](https://github.com/YuchuanTian/RethinkTinyLM) <br> [Paper](https://arxiv.org/abs/2402.02791)|
|[Tandem Transformers for Inference Efficient LLMs](https://arxiv.org/abs/2402.08644) <br> Aishwarya P S, Pranav Ajit Nair, Yashas Samaga, Toby Boyd, Sanjiv Kumar, Prateek Jain, Praneeth Netrapalli |<img width="1002" alt="image" src="figures/Tandem.png"> |[Paper](https://arxiv.org/abs/2402.08644)|
|[Scaling Efficient LLMs](https://arxiv.org/abs/2402.14746) <br> B.N. Kausik |<img width="1002" alt="image" src="figures/ScalingEfficientLLM.png"> |[Paper](https://arxiv.org/abs/2402.14746)|
|[MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases](https://arxiv.org/abs/2402.14905) <br> Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, Vikas Chandra |<img width="1002" alt="image" src="figures/MobileLLM.png"> |[Paper](https://arxiv.org/abs/2402.14905)|
|[Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding](https://arxiv.org/abs/2402.16844) <br> Benjamin Bergner, Andrii Skliar, Amelie Royer, Tijmen Blankevoort, Yuki Asano, Babak Ehteshami Bejnordi |<img width="1002" alt="image" src="https://arxiv.org/html/2402.16844v1/x1.png"> |[Paper](https://arxiv.org/abs/2402.16844)|
|[![Star](https://img.shields.io/github/stars/mbzuai-oryx/MobiLlama.svg?style=social&label=Star)](https://github.com/mbzuai-oryx/MobiLlama)<br>[MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT](https://arxiv.org/abs/2402.16840) <br> Omkar Thawakar, Ashmal Vayani, Salman Khan, Hisham Cholakal, Rao M. Anwer, Michael Felsberg, Tim Baldwin, Eric P. Xing, Fahad Shahbaz Khan |<img width="402" alt="image" src="https://github.com/mbzuai-oryx/MobiLlama/raw/main/images/mobillama_generation.gif"> |[Github](https://github.com/mbzuai-oryx/MobiLlama) <br> [Paper](https://arxiv.org/abs/2402.16840) <br>[Model](https://huggingface.co/MBZUAI/MobiLlama-05B) |
|[Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models](https://arxiv.org/abs/2402.19427) <br> Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando De Freitas, Caglar Gulcehre |<img width="1002" alt="image" src="https://arxiv.org/html/2402.19427v1/x3.png"> |[Paper](https://arxiv.org/abs/2402.19427)|
|[![Star](https://img.shields.io/github/stars/YuchuanTian/DiJiang.svg?style=social&label=Star)](https://github.com/YuchuanTian/DiJiang)<br>[DiJiang: Efficient Large Language Models through Compact Kernelization](https://arxiv.org/abs/2403.19928) <br> Hanting Chen, Zhicheng Liu, Xutao Wang, Yuchuan Tian, Yunhe Wang |<img width="1002" alt="image" src="https://github.com/YuchuanTian/DiJiang/raw/main/imgs/scheme.png"> |[Github](https://github.com/YuchuanTian/DiJiang) <br> [Paper](https://arxiv.org/abs/2403.19928)|
|[![Star](https://img.shields.io/github/stars/XuezheMax/megalodon.svg?style=social&label=Star)](https://github.com/XuezheMax/megalodon)<br>[Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length](https://arxiv.org/abs/2404.08801) <br> Xuezhe Ma, Xiaomeng Yang, Wenhan Xiong, Beidi Chen, Lili Yu, Hao Zhang, Jonathan May, Luke Zettlemoyer, Omer Levy, Chunting Zhou |<img width="1002" alt="image" src="figures/megalodon.png"> |[Github](https://github.com/XuezheMax/megalodon) <br> [Paper](https://arxiv.org/abs/2404.08801)|
|[![Star](https://img.shields.io/github/stars/alinlab/HOMER.svg?style=social&label=Star)](https://github.com/alinlab/HOMER)[![Publish](https://img.shields.io/badge/Conference-ICLR'24-blue)]()<br>[Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs](https://arxiv.org/abs/2404.10308) <br> Woomin Song, Seunghyuk Oh, Sangwoo Mo, Jaehyung Kim, Sukmin Yun, Jung-Woo Ha, Jinwoo Shin |<img width="1002" alt="image" src="figures/homer.png"> |[Github](https://github.com/alinlab/HOMER) <br> [Paper](https://arxiv.org/abs/2404.10308)|
|[![Star](https://img.shields.io/github/stars/itsnamgyu/block-transformer.svg?style=social&label=Star)](https://github.com/itsnamgyu/block-transformer)<br>[Block Transformer: Global-to-Local Language Modeling for Fast Inference](https://arxiv.org/abs/2406.02657) <br> Namgyu Ho, Sangmin Bae, Taehyeon Kim, Hyunjik Jo, Yireun Kim, Tal Schuster, Adam Fisch, James Thorne, Se-Young Yun |<img width="1002" alt="image" src="https://arxiv.org/html/2406.02657v1/x1.png"> |[Github](https://github.com/itsnamgyu/block-transformer) <br> [Paper](https://arxiv.org/abs/2406.02657)|[//]: #06/12
|[![Star](https://img.shields.io/github/stars/metacarbon/shareAtt.svg?style=social&label=Star)](https://github.com/metacarbon/shareAtt)<br>[Beyond KV Caching: Shared Attention for Efficient LLMs](https://arxiv.org/abs/2407.12866) <br> Bingli Liao, Danilo Vasconcellos Vargas |<img width="1002" alt="image" src="https://arxiv.org/html/2407.12866v1/x1.png"> |[Github](https://github.com/metacarbon/shareAtt) <br> [Paper](https://arxiv.org/abs/2407.12866)|[//]: #07/21
|[![Star](https://img.shields.io/github/stars/linxihui/dkernel.svg?style=social&label=Star)](https://github.com/linxihui/dkernel)<br>[Efficient LLM Training and Serving with Heterogeneous Context Sharding among Attention Heads](https://arxiv.org/abs/2407.17678) <br> Xihui Lin, Yunan Zhang, Suyu Ge, Barun Patra, Vishrav Chaudhary, Xia Song |<img width="1002" alt="image" src="https://github.com/linxihui/dkernel/raw/main/assets/localstride.png"> |[Github](https://github.com/linxihui/dkernel) <br> [Paper](https://arxiv.org/abs/2407.17678)|[//]: #07/26
|[SentenceVAE: Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context](https://arxiv.org/abs/2408.00655) <br> Hongjun An, Yifan Chen, Zhe Sun, Xuelong Li |<img width="1002" alt="image" src="https://arxiv.org/html/2408.00655v4/x2.png"> |[Paper](https://arxiv.org/abs/2408.00655)|[//]: #08/08
|[Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions](https://arxiv.org/abs/2410.06577) <br> Zhihao He, Hang Yu, Zi Gong, Shizhan Liu, Jianguo Li, Weiyao Lin |<img width="1002" alt="image" src="https://arxiv.org/html/2410.06577v1/x3.png"> |[Paper](https://arxiv.org/abs/2410.06577)|[//]: #10/14
|[![Star](https://img.shields.io/github/stars/TUDa-HWAI/Basis_Sharing.svg?style=social&label=Star)](https://github.com/TUDa-HWAI/Basis_Sharing)<br>[Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression](https://arxiv.org/abs/2410.03765) <br> Jingcun Wang, Yu-Guang Chen, Ing-Chao Lin, Bing Li, Grace Li Zhang |<img width="1002" alt="image" src="https://arxiv.org/html/2410.03765v1/x1.png"> |[Github](https://github.com/TUDa-HWAI/Basis_Sharing) <br> [Paper](https://arxiv.org/abs/2410.03765)|[//]: #10/14
|[![Star](https://img.shields.io/github/stars/microsoft/SeerAttention.svg?style=social&label=Star)](https://github.com/microsoft/SeerAttention)<br>[SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs](https://arxiv.org/abs/2410.13276) <br> Yizhao Gao, Zhichen Zeng, Dayou Du, Shijie Cao, Hayden Kwok-Hay So, Ting Cao, Fan Yang, Mao Yang |<img width="202" alt="image" src="https://arxiv.org/html/2410.13276v1/x4.png"> |[Github](https://github.com/microsoft/SeerAttention) <br> [Paper](https://arxiv.org/abs/2410.13276)|[//]: #10/21
|[Taipan: Efficient and Expressive State Space Language Models with Selective Attention](https://arxiv.org/abs/2410.18572) <br> Chien Van Nguyen, Huy Huu Nguyen, Thang M. Pham, Ruiyi Zhang, Hanieh Deilamsalehy, Puneet Mathur, Ryan A. Rossi, Trung Bui, Viet Dac Lai, Franck Dernoncourt, Thien Huu Nguyen |<img width="1002" alt="image" src="https://arxiv.org/html/2410.18572v1/x2.png"> |[Paper](https://arxiv.org/abs/2410.18572)|[//]: #10/29