
## Efficient Training
| Title & Authors | Introduction | Links |
|:--|  :----: | :---:|
|[![Star](https://img.shields.io/github/stars/TsinghuaC3I/LPA.svg?style=social&label=Star)](https://github.com/TsinghuaC3I/LPA)[![Publish](https://img.shields.io/badge/Conference-EMNLP'24-blue)]()<br>[Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention](https://arxiv.org/abs/2411.02063) <br> Xingtai Lv, Ning Ding, Kaiyan Zhang, Ermo Hua, Ganqu Cui, Bowen Zhou |<img width="1002" alt="image" src="https://arxiv.org/html/2411.02063v1/x1.png"> |[Github](https://github.com/TsinghuaC3I/LPA) <br> [Paper](https://arxiv.org/abs/2411.02063)|[//]: #11/18
|[![Star](https://img.shields.io/github/stars/NVlabs/COAT.svg?style=social&label=Star)](https://github.com/NVlabs/COAT)<br>[COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training](https://arxiv.org/abs/2410.19313) <br> Haocheng Xi, Han Cai, Ligeng Zhu, Yao Lu, Kurt Keutzer, Jianfei Chen, Song Han |<img width="1002" alt="image" src="https://github.com/NVlabs/COAT/blob/main/docs/figs/FP8PrecisionFlow.png"> |[Github](https://github.com/NVlabs/COAT) <br> [Paper](https://arxiv.org/abs/2410.19313)|[//]: #11/17
|[![Star](https://img.shields.io/github/stars/wuhouming/BitPipe.svg?style=social&label=Star)](https://github.com/wuhouming/BitPipe)<br>[BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training](https://arxiv.org/abs/2410.19367) <br> Houming Wu, Ling Chen, Wenjie Yu |<img width="1002" alt="image" src="https://github.com/wuhouming/BitPipe/raw/main/docs/BitPipe_images/BitPipe-v.svg"> |[Github](https://github.com/wuhouming/BitPipe) <br> [Paper](https://arxiv.org/abs/2410.19367)|[//]: #11/17
|[![Star](https://img.shields.io/github/stars/neiterman21/LDB.svg?style=social&label=Star)](https://github.com/neiterman21/LDB)<br>[LayerDropBack: A Universally Applicable Approach for Accelerating Training of Deep Networks](https://arxiv.org/abs/2412.18027) <br> Evgeny Hershkovitch Neiterman, Gil Ben-Artzi |<img width="1002" alt="image" src="https://arxiv.org/html/2412.18027v1/x1.png"> |[Github](https://github.com/neiterman21/LDB) <br> [Paper](https://arxiv.org/abs/2412.18027)|[//]: #12/30