FlashInfer

A customizable JIT attention-kernel engine (MLSys 2025 Best Paper) that now powers the major LLM serving stacks — vLLM, SGLang, TensorRT-LLM, TGI, and MLC-LLM (~6K★, NVIDIA-backed). Led by Zihao Ye in UW's Allen School, a UW + NVIDIA + CMU collaboration; the engine generates and dispatches specialized attention kernels for varied batching, prefix-sharing, and quantization regimes at serving time.