Aligner

PKU-Alignment's residual-correction alignment module (NeurIPS 2024 Oral): a small model stacked on any upstream LLM learns the residual between preferred and dispreferred answers and rewrites outputs at inference — alignment as a plug-in that transfers across API-only and open models without touching their weights.