SB Intuitions

corporate

japan · Founded 2023-08 · Part of SoftBank Corp.

SB Intuitions is SoftBank Corp.'s wholly-owned subsidiary for Japanese-specialized large language models, launched in August 2023 and framed publicly as "an OpenAI for Japan." President & CEO Hironobu Tamba concurrently runs SoftBank's Data Platform Strategy Division. The company operates the Sarashina Lab research division and has grown to 150+ employees, actively recruiting AI researchers from Japan and abroad.

The flagship Sarashina model family progressed from the Sarashina2 dense series (7B/13B/70B, trained from scratch on 2.1T tokens, MIT license, August 2024) to Sarashina2-8x70B, a ~465B-total MoE built via sparse upcycling and released for academic and industrial research (November 2024). The smaller, more compute-efficient Sarashina2.2 family (0.5B/1B/3B, March 2025) added instruction-tuning with strong Japanese scores — the 3B-Instruct beats Qwen2.5-3B-Instruct on Elyza-tasks-100 (3.75 vs 2.99). Multimodal variants followed (Sarashina2.2-Vision-3B, Sarashina2.2-OCR) along with embedding and encoder models (Sarashina-Embedding-v2-1B, ModernBERT-Ja). The commercial Sarashina mini API (70B) launched November 28, 2025 for SoftBank enterprise customers.

Sarashina Lab has produced a strong cluster of foundational training-stability and scaling papers at top venues: Spike No More (COLM 2025, architectural stability), Large Vocabulary Size Improves LLMs (ACL Findings 2025), Scaling Laws for Upcycling MoE (ICML 2025 — the basis for Sarashina2-8x70B itself), and Pre-training LLM without Learning Rate Decay (ICLR 2026). Training compute runs on SoftBank's NVIDIA Blackwell-based DGX SuperPOD, announced in 2025 as the world's largest Blackwell cluster for LLM development.

Website HuggingFace
frontiermultilingualjapanesemoe

People

News

Outputs (8)

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning

paper

Proposes Warmup-Stable-Only (WSO): a pretraining learning-rate schedule that holds the learning rate constant after warmup, with no decay phase. Altho…

Sarashina2.2 3

model

Compute-efficient Sarashina iteration focused on small, high-quality Japanese models. Dense Llama-style Transformers at 0.5B / 1B / 3B with both base…

Scaling Laws for Upcycling Mixture-of-Experts Language Models

paper

Derives empirical scaling laws for sparse upcycling — the practice of converting a pretrained dense LLM into a Mixture-of-Experts model by dupli…

ModernBERT-Ja

model

Japanese adaptation of the ModernBERT encoder (Warner et al., 2024) in three sizes: 30M, 130M, and 310M. Released February–May 2025 under the MI…

Sarashina2-8x70B

model

~465B-total-parameter Mixture-of-Experts Japanese LLM, built by sparse upcycling (Komatsuzaki et al., 2022) from Sarashina2-70B: 8 experts of 70B para…

Sarashina2

model

SB Intuitions' first frontier Japanese LLM series and the foundation of all later Sarashina releases. 7B, 13B, and 70B dense Llama-2-style Transformer…

Large Vocabulary Size Improves Large Language Models

paper

Empirical scaling study showing that larger subword vocabularies reliably improve LLM quality at fixed parameter budgets — a finding that ran co…

Spike No More: Stabilizing the Pre-training of Large Language Models

paper

Analyzes loss spikes during LLM pre-training — the sudden, training-destabilizing jumps that have plagued frontier-scale training runs — t…