About Me
Hi, I am Shenggui Li, you can call me Frank Lee as well. I am a first-year PhD student at the Nanyang Technological University in Singapore, advised by Prof. Tianwei Zhang and Prof. Ivor Tsang. I am also affliated with Singapore A*STAR Center for Frontier AI Research (CFAR). Prior to that, I obtained my 🤓 B.Eng in Computer Science in Nanyang Technological University in 2021 advised by Prof. Chen Change Loy. I also worked with Prof. Yang You from 2021 to 2024 as a research asssistant in the HPC-AI Lab and a co-founder of HPC-AI Technology.
My interest lies at the intersection of 🤖 machine learning and ⚡️ high performance computing. I first proposed the concept of sequence parallelism in 2021, which has attracted follow-ups and is widely adopted for large-scale model training. I am the author of Ring Self-Attention, Colossal-AI and Open-Sora. During my undergraduate study, I had the privilege to spend a wonderful year on the NTU HPC Team and we won the overall 🥈 2nd place in the student cluster competition of ISC21.
You can find my 📄 CV here.
I am involved in several interesting projects, feel free to email me for collaboration.
- 🎥 Open-Sora: Core contributor
- ⚡️ SwiftInfer: Core contributor
- ⚡️ ColossalAI: Top 1 contributor as of Aug 2024
News
- [2024-08] 🏫 Started my PhD journey at NTU CCSD, under the supervision of Prof Tianwei Zhang.
- [2024-06] 🎥 Released Open-Sora V1.2 with features such as 3D-VAE, rectified flow, and score conditio and better video generation quality!
- [2024-05] 🎉 One paper accepted for ICML 2024, congratulations to Cunxiao Du!
- [2024-04] 🎉 Awarded the prestigious A*STAR Computing & Information Science Scholarship, which will support my PhD studies at NTU.
- [2024-03] 🎥 Released Open-Sora V1.0 , congratulations to Zangwei Zheng, Xiangyu Peng, Hongxin Liu and other collaborators and contributors!
- [2024-03] 🎉 Admitted into the 2024 Fall PhD programme of NTU under the supervision of Prof Tianwei Zhang.
- [2024-01] 🚀 Released SwiftInfer , a TensorRT implementation of StreamingLLM.
Selected Publications
- Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training
Shenggui Li, Hongxin Liu, Zhengda Bian, Jiarui Fang, Haichen Huang, Yuliang Liu, Boxiang Wang, Yang You,
Proceedings of the 2023 International Conference for Parallel Processing (ICPP 2023)
- Sequence Parallelism: Long Sequence Training from System Perspective
Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, Yang You,
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)
- Parallel Training of Pre-Trained Models via Chunk-Based Dynamic Memory Management
Jiarui Fang, Zilin Zhu, Shenggui Li, Hui Su, Yang Yu, Jie Zhou, Yang You,
IEEE Transactions on Parallel and Distributed Systems (TPDS 2023)
- Glide with a cape: A low-hassle method to accelerate speculative decoding
Cunxiao Du, Jing Jiang, Xu Yuanchen, Jiawei Wu, Sicheng Yu, Yongqi Li, Shenggui Li, Kai Xu, Liqiang Nie, Zhaopeng Tu, Yang You,
Proceedings of the 2024 International Conference on Machine Learning (ICML 2024)
- Online evolutionary batch size orchestration for scheduling deep learning workloads in GPU clusters
Zhengda Bian, Shenggui Li, Wei Wang, Yang You,
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC 2021)
Honours and Awards
- [2024] A*STAR Computing & Information Science Scholarship
- [2021] Overall 2nd Place, ISC21 Student Cluster Competition
- [2020] Merit Award, APAC HPC-AI Competition
- [2018] NTU President Research Scholar
- [2017-2021] NTU Science & Technology Undergraduate Scholarship
- [2012-2016] Singapore SM1 Scholarship