Llama 4 Scout

How it is built

Llama 4 Scout has 16 experts and 109B parameters. Each token uses 17B of them, about 15.6%.¹¹1Meta, Llama 4 Scout model card, Hugging Face, 5 April 2025. MMLU Pro 74.3, GPQA Diamond 57.2, LiveCodeBench 32.8 pass@1, DocVQA 94.4 ANLS, all 0-shot. https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct. Arithmetic: 17 / 109 = 15.6%.

The card does not say how many experts a token visits, or whether one of them is shared.¹¹ Meta's reference code for Llama 4 builds a shared expert into every mixture of experts layer and sends each token to one routed expert by default.²²2Meta, Llama 4 reference implementation in llama-models: moe.py creates a shared_expert in every MoE layer, and args.py sets top_k to 1 by default. Scout's own params file sits behind the Hugging Face gate. https://github.com/meta-llama/llama-models/tree/main/models/llama4 Scout's own configuration sits behind the Hugging Face gate.²²

Attention is iRoPE, as in Llama 4 Maverick. Some layers carry no positional embeddings, and the attention temperature is scaled at inference time.³³3Meta, "The Llama 4 herd", 5 April 2025: Scout "is both pre-trained and post-trained with a 256K context length, which empowers the base model with advanced length generalization capability". https://ai.meta.com/blog/llama-4-multimodal-intelligence/

Images and text go through the same backbone, a design Meta calls early fusion.³³

What is new

Scout reads up to 10M tokens of context, ten times Maverick's 1M.¹¹⁴⁴4Meta, Llama 4 Maverick model card, Hugging Face. https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct. Arithmetic: 10M / 1M = 10; 40 / 22 = 1.8. Meta pre-trained and post-trained it at 256K, and credits that training for its length generalisation.³³

Pre-training used about 40 trillion tokens, nearly twice Maverick's 22 trillion.¹¹⁴⁴ Training took 5.0 million H100 GPU hours. Meta puts the location-based emissions at 1,354 tons of CO2 equivalent.¹¹

Instruction-tuned, Scout scores 74.3 on MMLU Pro and 57.2 on GPQA Diamond.¹¹ LiveCodeBench, over problems from October 2024 to February 2025, gives 32.8 pass@1. On DocVQA its 94.4 ANLS equals Maverick's.¹¹⁴⁴ All four scores are 0-shot.¹¹ The knowledge cutoff is August 2024.¹¹

Running it

The weights are released in BF16.⁵⁵5Meta, Llama 4 model card in llama-models: "The Llama 4 Scout model is released as BF16 weights". https://github.com/meta-llama/llama-models/blob/main/models/llama4/MODEL_CARD.md At 2 bytes per parameter that is about 218 GB (109B x 2 bytes).⁶⁶6Arithmetic, weights only: 109B x 2 bytes = 218 GB in BF16; 109B x 0.5 byte = 54.5 GB at 4 bits, before quantization scales and the KV cache. The card gives training hardware as H100-80GB. https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct

The card says Scout "can fit within a single H100 GPU with on-the-fly int4 quantization".¹¹ At half a byte per parameter the weights shrink to about 55 GB (109B x 0.5 byte), which leaves part of an 80 GB H100 for the KV cache.⁶⁶

Access to the weights on Hugging Face is gated. Companies above 700 million monthly active users need a separate licence from Meta.⁷⁷7Meta, Llama 4 Community License Agreement, effective 5 April 2025. https://github.com/meta-llama/llama-models/blob/main/models/llama4/LICENSE The acceptable use policy grants no rights to Llama 4's multimodal models to people domiciled in the EU, or to companies based there. End users of products built on them are exempt.⁸⁸8Meta, Llama 4 Acceptable Use Policy. https://github.com/meta-llama/llama-models/blob/main/models/llama4/USE_POLICY.md

No Llama model appears in the Meta Model API.⁹⁹9Meta, Meta Model API documentation, models list, read 29 September 2026. https://dev.meta.ai/docs/models Meta's newest open weights belong to Muse Glimmer 30B, a dense model from August 2026.¹⁰¹⁰10Meta Superintelligence Labs, Muse Glimmer model card, August 2026. https://huggingface.co/meta-models/Muse-Glimmer-30B