SmallThinker
modelIPADS's MoE family "natively trained for local deployment" — from-scratch pretraining co-designed with the on-device inference stack (fine-grained sparsity, pre-attention router for compute/IO overlap) rather than compressing a cloud model after the fact. Successor to the viral SmallThinker-3B-Preview (415 HF likes); weights now ship under the team's Tiiny-AI HF org.
Model Details
Architecture MOE
Parameters 21B
Active params 3B
Variants
| Name | Parameters | Notes |
|---|---|---|
| SmallThinker-4B-A0.6B | 4B | — |
| SmallThinker-21B-A3B | 21B | — |