MegaScale

library
ByteDance
System for scaling LLM training to over 10,000 GPUs.
Paper (arXiv)

Paper

Citations 24
arXiv HTML
infrastructuretrainingscaling