Open omni-modal LLM with personalized, long-horizon speech: an "omni-centric" training pipeline scaling speech understanding and generation on top of the Mini-Gemini lineage (arXiv 2403.18814, the lab's 2024 first-in-series multimodal model), targeting long-form and voice-cloned speech interaction alongside image/video understanding.

From Jiaya Jia's lab — the group moved CUHK→HKUST around 2023–24 (GitHub org renamed dvlab-researchJIA-Lab-research), so authorship is mixed CUHK/HKUST; filed under HKUST per the group's current home. Siblings in the same lab: DreamOmni2 instruction-based image editing (2K★) and the VisionThink/Seg-Zero RL visual-reasoning line (NeurIPS 2025).

Paper

multimodalaudioopen-weight