Mistral AI has unveiled Mistral Large 2, the latest iteration of its flagship language model, featuring significant advancements in code generation, mathematics, and multilingual capabilities. This new model, with 123 billion parameters and a 128,000-token context window, aims to compete with industry leaders in both performance and efficiency.
Mistral Large 2 has demonstrated strong performance across various benchmarks. In code generation tasks, such as HumanEval and MultiPL-E, it surpasses the newly released Llama 3.1 405B by Meta, and performs just below GPT-4. For mathematics, particularly on the MATH benchmark (zero-shot, without chain-of-thought reasoning), it ranks second only to GPT-4o.
The model’s multilingual capabilities have also been notably enhanced. On the Multilingual MMLU benchmark, Mistral Large 2 outperforms Llama 3.1 70B by an average of 6.3% across nine languages and matches the performance of Llama 3 405B.
Designed for single-node inference, Mistral Large 2 emphasizes throughput for long-context applications. It is available on Mistral AI’s platform, la Plateforme, and the weights for the instruct model have been released on HuggingFace for research.
Arthur Mensch, CEO of Mistral AI, stated, “Mistral Large 2 sets a new standard in performance-to-cost ratio on evaluation metrics.” The pretrained model achieves an 84.0% accuracy on MMLU, establishing a new benchmark in the performance/cost ratio for open models.
The model builds on Mistral AI’s experience with previous code-focused models, delivering performance comparable to leading models such as GPT-4, Claude 3 Opus, and Llama 3 405B in coding tasks. Efforts to enhance reasoning capabilities and reduce hallucinations have also been successful, with notable improvements in mathematical benchmarks.
Mistral Large 2 excels in instruction-following and conversational tasks, showcasing advancements in handling precise instructions and extended, multi-turn conversations.
The release of Mistral Large 2, coming shortly after Llama 3.1, underscores the growing competition in the AI language model sector. Its strengths in specialized areas, coupled with robust multilingual support, position it as a key contender for both research and commercial applications.
