Model Merging in Pre-training of LLMs

Research on model merging techniques during pre-training of large language models, exploring how independently trained model branches can be merged to improve training efficiency and final model quality.