A transparent end-to-end MoE research platform from the FLAME Center (Kang, Xiong): seven fully open mixture-of-experts language models with complete training artifacts, built as the community's controlled substrate for MoE science — CMU's in-house answer to the fully-open-pretraining wave.

Paper

moeopen-weightpretraining