On January 15, Shanghai-based AI startup MiniMax announced the release and open-sourcing of its next-generation MiniMax-01 series models. This release includes the foundational language model MiniMax-Text-01 and the visual multimodal model MiniMax-VL-01.
The MiniMax-01 series introduces an innovative hybrid structure. For every 8 layers, 7 use a linear Lightning Attention mechanism, while 1 retains the traditional SoftMax Attention. This unique approach enables near-linear complexity, allowing the model to efficiently process ultra-long inputs and outputs.
Since Google introduced the Transformer architecture in 2017, it has become the dominant paradigm for large models. However, since 2023, the natural language processing field has seen a wave of innovation, with increasing demand for architectural breakthroughs.
SEE ALSO: iQIYI Sues Minimax for Unauthorized Use of Content in Model Training
The Linear Attention mechanism represents a step forward, optimizing algorithms to transform the quadratic growth in input length and computational complexity into a linear relationship.
With 456 billion parameters and 45.9 billion activated per inference, the MiniMax-01 models can handle up to 4 million tokens of context. This surpasses GPT-4o by 32 times and Claude-3.5-Sonnet by 20 times, making it the first commercially viable model to adopt Lightning Attention at this scale.
The open-source MiniMax-01 series models have already been integrated into the global AI application Hailuo AI. Enterprises and developers can access these capabilities via the MiniMax API platform. Pricing starts at ¥1 per million input tokens and ¥8 per million output tokens.
Sign up today for 5 free articles monthly!
