Baidu Open-Sources Base Model ERNIE 5.5 Flash Base
Baidu has officially open-sourced its base model, ERNIE 5.5 Flash Base, which features a MoE architecture with a total of 280 billion parameters and 14 billion activated per token. It was pre-trained on 33.5 trillion tokens, with a maximum pre-training context length of 128K, and supports up to 1 million tokens for fine-tuning and inference. The model has been systematically optimized for efficiency in areas such as the attention mechanism, expert routing, and training objectives.
Baidu Open-Sources Foundation Model ERNIE 5.5 Flash Base
Baidu has officially open-sourced its latest foundation model, ERNIE 5.5 Flash Base. The model adopts an MoE architecture, with a total of 280 billion parameters and 14 billion parameters activated per token. It was pre-trained on 33.5 trillion tokens, with a maximum pre-training context length of 128K. For fine-tuning and inference, it supports a context length of up to 1 million tokens.
Centered on the theme of "efficiency," the model has undergone systematic optimizations in areas such as the attention mechanism, expert routing, and training objectives to improve resource utilization efficiency.
Xinhua News Agency
https://www.news.cn/info/20261105/f3bbdb6854f5441ca9fb34062f213875/c.html