Gemma 4 31B

How it is built

Every token goes through all 30.7B parameters of Gemma 4 31B, spread over 60 layers.¹¹1Google, Gemma 4 model card: 31B dense, 30.7B parameters, 60 layers, sliding window 1,024 tokens, context 256K, vocabulary 262K, vision encoder about 550M parameters; hybrid attention "ensuring the final layer is always global". https://ai.google.dev/gemma/docs/core/model_card_4

Most layers see only the last 1,024 tokens, through a sliding window. The rest attend to the whole context.¹¹

The published config makes every sixth layer global, 10 in all, with 50 local layers between them.²²2Google, config.json of gemma-4-31B-it on Hugging Face: 60 layers, full attention at layers 6, 12, 18 and so on up to 60, 32 attention heads, 16 key-value heads, 4 global key-value heads, dtype bfloat16. https://huggingface.co/google/gemma-4-31B-it/blob/main/config.json. Arithmetic: 60 / 6 = 10 global layers; 60 - 10 = 50 local layers. The card adds that the final layer is always global.¹¹

The local layers pair 32 query heads with 16 key and value heads. The global layers cut that to 4.²² Images go through a vision encoder of about 550M parameters, and the vocabulary has 262K entries.¹¹

What is new

Gemma 4 came out in April 2026 in four sizes.³³3Google, Gemma releases page: "Release of Gemma 4" in E2B, E4B, 31B and 26B A4B sizes, dated 31 March 2026; Gemma 4 MTP drafters for the same sizes, 16 April 2026. Google's launch post and the Gemini API release notes date the launch 2 April 2026, the day of the first commit to the Hugging Face repository. https://ai.google.dev/gemma/docs/releases

The licence changed. Gemma 3 27B was released under Google's own Gemma licence, with gated weights on Hugging Face.⁴⁴4Hugging Face, google/gemma-3-27b-it repository: licence "gemma", access gated. https://huggingface.co/google/gemma-3-27b-it Gemma 4 uses Apache 2.0, and nobody has to ask for access.⁵⁵5Google, gemma-4-31B-it model card on Hugging Face: licence Apache 2.0, not gated. Instruction-tuned results: MMLU Pro 85.2% against 67.6% for Gemma 3 27B; AIME 2026 no tools 89.2% against 20.8%; LiveCodeBench v6 80.0% against 29.1%; Codeforces ELO 2150 against 110; the Gemma 3 27B column is headed "no think". Thinking is enabled by a token at the start of the system prompt. https://huggingface.co/google/gemma-4-31B-it

The context window doubled, from 128K tokens in Gemma 3 27B to 256K.⁶⁶6Google, Gemma 3 model card: "Total input context of 128K tokens for the 4B, 12B, and 27B sizes". https://ai.google.dev/gemma/docs/core/model_card_3. Arithmetic: 256K / 128K = 2. ¹¹

Against Gemma 3 27B the benchmark gaps are wide, though the older model's column was run without thinking.⁵⁵ MMLU Pro rose from 67.6% to 85.2%. AIME 2026 without tools went from 20.8% to 89.2%, and LiveCodeBench v6 from 29.1% to 80.0%.⁵⁵ The older model's Codeforces Elo is 110. Gemma 4 31B reaches 2150.⁵⁵

Google's launch post put it third among open models on the Arena AI text leaderboard.⁷⁷7Google, "Gemma 4: Byte for byte, the most capable open models", 2 April 2026: the 31B ranked "#3 open model in the world" on the Arena AI text leaderboard at launch; weights on Hugging Face, Kaggle and Ollama. https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

Running it

The weights are stored in BF16, where Google's memory table puts them at 69.9 GB.²²⁸⁸8Google, Gemma 4 overview, memory table: 31B at 69.9 GB in BF16, 34.9 GB in SFP8 and 17.5 GB in Q4_0; 26B A4B at 57.7 GB in BF16. The figures "only account for the memory required to load the static model weights". https://ai.google.dev/gemma/docs/core. Arithmetic: 69.9 / 2 = 34.95. SFP8 halves that to 34.9 GB, and Q4_0 brings it to 17.5 GB.⁸⁸ These figures cover the weights alone. The context and the serving software come on top.⁸⁸

Gemma 4 26B A4B, the mixture of experts released with it, needs 57.7 GB in BF16.⁸⁸

Thinking stays off unless the system prompt starts with the <|think|> token.⁵⁵ Draft models for speculative decoding arrived on 16 April 2026.³³

The weights are on Hugging Face and Kaggle, and the Gemini API serves the model as gemma-4-31b-it.⁷⁷⁹⁹9Google, Gemini API release notes, 2 April 2026: "Released gemma-4-26b-a4b-it and gemma-4-31b-it". https://ai.google.dev/gemini-api/docs/changelog