Llama Scope

256 open sparse autoencoders trained on every layer and sublayer of Llama-3.1-8B — the anchor of OpenMOSS's mechanistic-interpretability program (Xipeng Qiu), shipped with the Language-Model-SAEs framework. Extended by Lorsa (low-rank sparse attention, arXiv:2504.20938, ICLR 2026) and the crosscoder study of concept evolution across pretraining (arXiv:2509.17196, ICLR 2026), both of which add SII affiliations as the group migrated.