Taobao MNN Team Open-Sources 0.3B Mobile Model
On November 6, Taobao's MNN team open-sourced MNN-Pocket-0.3B, an on-device model with approximately 300 million parameters. It can run completely offline on the iPhone 18 Pro, achieving an average decoding speed of about 350 tokens/s, with its INT4 deployment file being just 173 MB. The team aims to reduce peak memory usage to under 300 MB and increase decoding speed to over 200 tokens/s.
Taobao's MNN Team Open-Sources 0.3B Mobile Model
On November 6, Taobao's MNN team open-sourced MNN-Pocket-0.3B, an on-device model with approximately 300 million parameters. It runs completely offline on the iPhone 18 Pro, achieving an average decoding speed of about 350 tokens/s, with its INT4 deployment file being just 173 MB.
The team's goal was to achieve a peak memory usage below 300 MB and a decoding speed exceeding 200 tokens/s. The final design adopts a 28-layer structure, with the vocabulary reduced from 151K to 64K. Multiple versions have been open-sourced for pre-training, post-training, and on-device deployment.
Source: IT Home
