Mistral Large 4 Goes Open Source: A Full Breakdown of Its Architectural Differences from DeepSeek

European AI company Mistral AI has released the Large 4 series of open-source large models, taking direct aim at China's DeepSeek. According to a Hugging Face technical blog post, this model borrows heavily from DeepSeek's MLA and DSA techniques in its architecture, but takes a different path in parameter scale, activation strategy, and deployment compatibility.

Parameter Scale: 1T Total Parameters vs. 763B, with 49B Activated

Mistral Large 4 uses a MoE architecture with 1024B (1T) total parameters and 49B activated parameters. By comparison, DeepSeek^Ad V4.1 Flash has 552B backbone parameters, with total parameters of around 763B after including other components. In terms of sheer size, Mistral Large 4 is indeed larger, but that also means higher inference costs.

In terms of context window, Mistral Large 4 supports 1 million tokens, making it suitable for handling ultra-long documents and complex multimodal tasks. The model is offered in two versions: FP8 and NVFP4. FP8 is compatible with GPUs such as H100, H200, and RTX 4090, while NVFP4 targets newer chips like RTX 5090 and B200 (Blackwell architecture), offering advantages in compute utilization.

MoE (Mixture of Experts) architecture is the key to understanding the difference between "total parameters" and "activated parameters." A MoE model consists of multiple parallel feed-forward "expert" subnetworks plus a routing gate. When each token passes through, the gate selects only a few experts to participate in computation. Therefore, although the model has 1024B total parameters, a single token actually activates only 49B of them. The inference compute overhead roughly corresponds to a dense model of 49B scale, while the model capacity (knowledge storage and expressiveness) approaches the 1T level.

This also explains the article's judgment that it is "larger in size but has higher inference costs": MoE memory requirements are still calculated based on total parameters (all experts must be loaded into VRAM), so 1T total parameters means an extremely high VRAM threshold, which directly corresponds to the later deployment requirement of "at least 8 H100/H200 GPUs." Activated parameters determine the computation per token; 49B activated is higher than DeepSeek's 16B during the decoding phase, which is an important reason for its slower inference speed.

Architecture Borrowing: MLA and DSA Become Open-Source Standards

Mistral Large 4's core architecture is clearly influenced by DeepSeek. It adopts the MLA (Multi-head Latent Attention) and DSA (DeepSeek Sparse Attention) mechanisms released by DeepSeek, which are also the mainstream choices for current open-source large models. From this perspective, DeepSeek is leading the architectural direction of the open-source AI ecosystem.

However, the backbone designs of the two differ. Mistral Large 4 uses a 72-layer Decoder-only architecture, with a dedicated image encoder connected to the backbone to achieve full-modality support. DeepSeek V4.1 Flash adopts an asymmetric encoder-decoder architecture: 20 encoder layers plus 20 decoder layers, with the encoder mainly used during the prefill phase and the decoder during the decoding phase. This design allows DeepSeek to activate only a small number of parameters (about 8B or 16B) during prefill and 16B during decoding, thereby compressing compute costs.

Mistral Large 4 does not adopt this asymmetric strategy. With 72 decoding layers plus 49B activated parameters, its inference speed is likely slower than DeepSeek V4.1 Flash. DeepSeek's architectural choices clearly lean toward extreme cost-effectiveness, while Mistral seems to focus more on model capacity and multilingual capabilities.

MLA (Multi-head Latent Attention) was first proposed by DeepSeek in V2. Its core idea is to first project the Key/Value cache in traditional multi-head attention into a low-dimensional "latent vector" before performing attention computation, thereby compressing the VRAM usage of the KV Cache by several times. For long-context and MoE large models, the KV Cache is often the main bottleneck in inference VRAM. Therefore, MLA can support longer contexts and larger batches under the same VRAM, which is the fundamental reason it has been adopted by many open-source models.

DSA (DeepSeek Sparse Attention) introduces sparsity into attention computation, allowing each token to attend to only some positions rather than the full context, thereby reducing the computational complexity of attention from quadratic growth with sequence length to approximately linear. One saves VRAM and the other saves compute, and they are often used together. Mistral Large 4's simultaneous adoption of MLA and DSA means that at the architectural level it directly inherits DeepSeek's entire set of ideas for optimizing long-context inference costs, which is also the technical basis for the article's claim of "open-source architecture convergence."

Ecosystem and Deployment: Open Source and Free for Commercial Use, but High Hardware Threshold

Mistral Large 4 uses an open-source, free-for-commercial-use license, with solid ecosystem support. vLLM and SGLang completed adaptation on the day of release, and the llama.cpp team and Hugging Face also provided GGUF format. For speculative decoding, both companies use the Eagle algorithm released by DeepSeek.

However, the deployment threshold cannot be ignored. The official recommendation is that using the NVFP4 version requires at least 8 H100 or H200 GPUs to run; 8 D300 GPUs can be used for FP8 fine-tuning training (not full-parameter training). Even with GGUF format, running this 1T model on a personal computer remains very demanding.

vLLM, SGLang, llama.cpp, and GGUF form the current toolchain for deploying open-source models. vLLM and SGLang are server-oriented high-throughput inference frameworks, featuring optimizations such as PagedAttention and continuous batching for VRAM and scheduling, suitable for multi-user concurrent deployment; same-day adaptation means users can immediately run the model as an API service. llama.cpp is a lightweight inference engine for local and edge devices, and GGUF is its accompanying quantized model file format, which can compress weights to 4 bits or even lower for running on consumer-grade hardware.

But a complete toolchain does not mean the threshold disappears: GGUF quantization mainly compresses VRAM usage. A 1T total-parameter MoE model still requires enormous VRAM even after quantization, and activated parameters are not reduced by quantization, so running it on a personal computer remains difficult. Speculative Decoding and the Eagle algorithm mentioned in the article are acceleration methods: a small model first "drafts" several tokens, and then the large model verifies them in parallel at once, improving throughput in the generation-length direction without changing the output distribution. This is also why both companies use Eagle.

Performance Positioning: Strongest in Europe, but Hard to Match DeepSeek's Cost-Effectiveness

Based on available information, Mistral Large 4 may be inferior to DeepSeek V4.1 Flash in overall performance and cost-effectiveness. Its advantage lies in the European language system—it performs better in multilingual scenarios such as Italian, French, and Spanish, making it suitable for multimodal applications such as cross-border e-commerce. However, its compute requirements are clearly heavier.

For developers, the choice depends on the scenario: for extreme cost-effectiveness and inference speed, DeepSeek V4.1 Flash remains the first choice; for European language support or larger parameter capacity, Mistral Large 4 offers a new option. The architectural convergence of open-source models is also worth noting—MLA and DSA are becoming de facto standards, and it is difficult for latecomers to avoid DeepSeek's technical path.

FP8 and NVFP4 are two low-precision numerical formats used to compress model weights and computation during inference. FP8 is 8-bit floating point, with native tensor core support on the Hopper architecture (H100/H200). It is currently the mainstream low-precision format for large model inference and training, with small precision loss and a mature ecosystem. NVFP4 is a 4-bit floating point format introduced by NVIDIA for the Blackwell architecture (B200, RTX 5090). With a smaller bit width, it can theoretically halve VRAM usage and bandwidth requirements again, but it requires new hardware for efficient execution.

By offering both versions, Mistral Large 4 effectively paves both paths: "compatibility with the existing GPU ecosystem" and "squeezing compute out of the new generation of hardware." Users with older cards go with FP8, while users with newer cards go with NVFP4 for higher throughput. This also means that the actual deployment cost of the model depends heavily on the hardware generation—if you only have pre-Hopper cards, the compute advantages of NVFP4 cannot be realized, and you can only fall back to FP8 and bear greater VRAM pressure.

One-line Summary

Mistral Large 4 takes on DeepSeek with 1T total parameters, borrowing MLA and DSA architectures but trailing in cost-effectiveness and deployment threshold.

Paragraph Summary

Mistral AI has released the open-source large model Large 4, with 1024B total parameters and 49B activated, surpassing DeepSeek V4.1 Flash's 763B total parameters in scale. Its core architecture adopts the MLA and DSA attention mechanisms proposed by DeepSeek, showing that open-source large model architectures are converging. However, the backbone designs differ: Mistral uses a 72-layer Decoder-only architecture with an image encoder for full modality, while DeepSeek uses an asymmetric encoder-decoder to compress prefill costs and save compute during inference. Mistral offers FP8 and NVFP4 versions, with rapid adaptation by vLLM, SGLang, and llama.cpp, but deployment requires at least 8 H100/H200 GPUs. Its advantage lies in European multilingual scenarios, while its overall cost-effectiveness and inference speed may fall short of DeepSeek.