IPADS's MoE family "natively trained for local deployment" — from-scratch pretraining co-designed with the on-device inference stack (fine-grained sparsity, pre-attention router for compute/IO overlap) rather than compressing a cloud model after the fact. Successor to the viral SmallThinker-3B-Preview (415 HF likes); weights now ship under the team's Tiiny-AI HF org.

Model Details

Architecture MOE
Parameters 21B
Active params 3B

Variants

Name Parameters Notes
SmallThinker-4B-A0.6B 4B
SmallThinker-21B-A3B 21B

Paper

efficiencymoeopen-weightreasoning