Towards Greater Leverage: Scaling Laws for Efficient MoE

MoE scaling laws paper exploring efficient mixture-of-experts language models.