Muhan Zhang's group converts existing GQA checkpoints to DeepSeek-style Multi-head Latent Attention (~490★): it shows MLA is strictly more expressive than GQA at the same KV-cache budget and migrates released models with minimal quality loss, letting the open ecosystem inherit DeepSeek's KV-cache compression without retraining from scratch.

Paper

architectureattentionefficiency

Related