Undergraduate student interested in EfficientML (i.e., memory-efficient ML systems):
- Distributed training : Distributed Data Parallel (DDP), Tensor Parallel (TP)
- TinyML : Pruning, Neural Architecture Searching (NAS)
- Optimization algorithms : Preconditioned Stochastic Tensor (Shampoo), ZeRO
For deeper understanding, I am currently focusing on the following list:
- OSS Contribution : shampoo, maintained by Meta Research.
- Cohort of Pseudo-Lab : Learning TinyML techniques and reviewing papers with prof. Song Han's lecture, MIT-6.5940
- Paper Reading Group hosted by SqueezeBits: A collaborative study focused on Efficient Generative AI.
p.s. Since I love exploring GitHub OSS, many useful tools are archived in starred list. Please check DevOps and Linux!