Kimi K3.1 Pro VRAM Requirements

A 2.8 trillion parameter model is only open to people who have somewhere to put it, and most of the usual ways to shrink a big model assume its weights start out in 16-bit. Kimi K3.1 Pro's official checkpoint is 1,561.3 GB, and because its experts already ship in 4-bit MXFP4, quantizing again buys little: the smallest GGUF made for llama.cpp is still 1,009.3 GB.

That puts the floor for the full model at one 8-GPU Blackwell Ultra or AMD MI355X node. An 8x B200 node is 121 GB short before the cache, and no Mac holds the full model. The numbers below come from Hugging Face file listings on November 7, 2026, the vLLM recipe and the model's own config.json. One example runs through all of it: a platform team that wants to serve coding agents to 20 engineers, each session holding up to 256K tokens.

Why the Weights Are Already as Small as They Get

Kimi K3.1 Pro is Moonshot AI's Mixture-of-Experts model with 2.8 trillion total parameters and 104 billion active per token, a 1,048,576-token context window and 93 layers, according to its model card. Each layer after the first routes every token to 16 of 896 experts plus two shared experts. Moonshot published the weights on Hugging Face in early November 2026; the repository's initial commit is dated November 2.

The checkpoint stores the routed experts in MXFP4, 4-bit values with one shared scale per block of 32, and everything else in BF16. The repository's safetensors metadata counts 57.36 billion BF16 parameters, which is 114.7 GB for everything except the routed experts, including attention, the vision encoder and the embeddings. The other 1,446.5 GB is routed experts. All 896 experts per layer have to stay loaded because the router can pick any of them for the next token, so active parameters set the compute cost of a token, not the memory bill.

Every Checkpoint and What It Weighs

Sizes are each repository's weight files summed from the Hugging Face API, in decimal gigabytes. Every row but the first is a community build.

The 4-bit GGUF saves only 52 GB over the original, because the experts are already 4-bit. llama.cpp merged support for the text model on November 6, 2026 in pull request #31779, and the first release containing it is build b12180. DevQuasar's files declare the same kimi-k31 architecture as the merged code, but its card points to a fork's kimi-k31 branch, so test them on b12180 before you plan around them.

The vllm-project pruned build drops 75% of the routed experts, and its card warns to "expect a large quality regression": 43.7% on GSM8K, against 96.2% for the full model in the vLLM recipe (possibly a different harness). It exists to test vLLM setups on fewer GPUs.

What the Cache Adds

Kimi K3.1 Pro mixes two kinds of attention layers, and only one of them grows with context. Its 24 full-attention layers use multi-head latent attention with DeepSeek-style sparse attention (DSA): each caches one compressed latent per token, plus a 128-value indexer key that picks the 2,048 most relevant earlier tokens. The other 69 layers use Kimi Delta Attention (KDA), a linear attention that keeps a fixed-size state per sequence.

The vLLM recipe gives the latent's size as 576 bytes per token per layer in FP8, the format its capacity figures use. vLLM's source (kda_state_dtype in mamba_utils.py) keeps the KDA recurrent state in FP32 unless you set --mamba-ssm-cache-dtype. Applying both to the config:

Python
Copy
import json, urllib.request

url = "https://huggingface.co/moonshotai/Kimi-K3.1-Pro/resolve/main/config.json"
t = json.load(urllib.request.urlopen(url))["text_config"]
la, dsa = t["linear_attn_config"], t["dsa_config"]

mla = len(la["full_attn_layers"])   # cache grows with every token
kda = len(la["kda_layers"])         # fixed-size state per sequence

# FP8 latent (576 bytes, per the vLLM recipe) + FP8 indexer key with one FP32 scale
per_token = mla * (576 + dsa["index_head_dim"] + 4)

h, d, k = la["num_heads"], la["head_dim"], la["short_conv_kernel_size"]
per_seq = kda * (h * d * d * 4                # FP32 recurrent state
                 + 3 * h * d * (k - 1) * 2)   # BF16 short-conv state for q, k, v

print(f"{mla} MLA layers, {kda} KDA layers")
print(f"{per_token:,} bytes per token + {per_seq / 1e6:.1f} MB per sequence")
for ctx in (32_768, 131_072, 262_144, 1_048_576):
    print(f"{ctx:>9,} tokens: {(per_token * ctx + per_seq) / 1e9:5.2f} GB")

text
Copy
24 MLA layers, 69 KDA layers
16,992 bytes per token + 449.4 MB per sequence
   32,768 tokens:  1.01 GB
  131,072 tokens:  2.68 GB
  262,144 tokens:  4.90 GB
1,048,576 tokens: 18.27 GB

So one of the platform team's 256K-token sessions costs 4.90 GB, and 0.45 GB of that is KDA state that does not grow. A full 1M-token session is 18.27 GB, small next to 1.56 TB of weights. For scale, the recipe reports room for about 2.6 million cached tokens on 8x B300 at tensor parallel 8, which at 16,992 bytes each is 44.2 GB of cache on every GPU.

The catch for 20 engineers is where that cache lives. Under plain tensor parallelism every GPU holds a full copy of the latent cache, so 2.6 million tokens is the whole node's budget: about nine sessions at 256K. vLLM's decode context parallelism (DCP8) shards the latent and indexer keys across the eight GPUs instead and raises capacity to about 18.5 million tokens, or 70 full sessions, at the cost of decode steps that are 7% to 25% slower. The KDA state is already split by attention head across the GPUs, so 20 sessions add only about 1.1 GB per GPU. An NVFP4 latent cache (352 bytes instead of 576) holds about 1.5 times more tokens and matched FP8 on GSM8K (96.1% vs 96.2%), but its long-context accuracy has not been measured yet.

Which Data Center GPU Nodes Fit

The vLLM recipe's baseline is one tensor-parallel group of eight GPUs: a single 8x B300 node, two 4-GPU GB300 trays, or one 8x AMD Instinct MI355X node, where only a tuned single-node agentic configuration has run so far. It also covers Google TPU v7x, Zhenwu PPUs and a VR200 preview. NVIDIA lists 2.1 TB of total GPU memory for a DGX B300, and AMD lists 288 GB per MI355X, so both nodes clear 1.56 TB with room for the cache.

H100, H200 and B200 are not on the list, and the arithmetic says why. A DGX B200 has 1,440 GB of GPU memory in total, 121 GB less than the weights alone, and eight 141 GB H200s come to 1,128 GB. Both would need a second node, which the recipe does not cover, and Hopper GPUs have no native FP4 support for the MXFP4 experts.

Software is the other constraint. The recipe requires vLLM 0.34.0, which it describes as not released yet; PyPI's latest was 0.33.0 on November 7, 2026. So you use a nightly wheel or the vllm/vllm-openai:kimi-k31 image, and SGLang users need its lmsysorg/sglang:dev-kimi-k31-pro image. This is the recipe's single-node B300 command, not run here:

Bash
Copy
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  vllm/vllm-openai:kimi-k31 moonshotai/Kimi-K3.1-Pro \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --load-format fastsafetensors \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

For the 20-engineer case, add the recipe's decode context parallelism option.

Can You Run It on a Mac or a Workstation?

Not the full model in memory. The largest Mac Studio is an M5 Ultra with 512 GB of unified memory, which is 549.8 GB in the decimal units used here, and the smallest llama.cpp GGUF is 1,009.3 GB.

What fits a single 512 GB Mac is a pruned copy. pipenetwork's REAP80 builds keep 179 of the 896 experts in each MoE layer, picked with a code-heavy calibration set, and weigh 321.9 GB to 350.0 GB in MLX format. The model card of the 336.0 GB q6 build reports about 6 tokens per second on a 512 GB M3 Ultra, using mlx-lm with a loader file from the repository after raising iogpu.wired_limit_mb. The same card warns of real degradation versus the full model, especially outside code.

The full set of experts runs on a Mac only by streaming them from disk. The inferencerlabs MLX build keeps the 114.7 GB of non-expert weights in memory and offloads the 1,446.5 GB of experts to the SSD. Its publisher reports about 0.37 tokens per second on a 512 GB M3 Ultra at 10,000 tokens of context, using Inferencer 2.5.2, so a 500-token answer takes about 22 minutes.

On Linux, llama.cpp splits a GGUF between GPU memory and system RAM, so the two together must exceed the file size with room to spare: well over 1 TB for the Q2_K build, which takes server or high-end workstation hardware. The author of the llama.cpp pull request reports testing the MXFP4 weights and a mixed 2-bit and 3-bit quant without issues. The midagedev IQ1_S build needs less, two NVIDIA cards plus 256 GB of RAM and an NVMe drive in its author's test, but only its own bloomery engine loads it, and context is capped at 2,048 tokens for now. We have not measured speed or quality on any of these setups.

Or Pay per Token Instead

Moonshot serves the same model as kimi-k3.1-pro at $2.00 per million input tokens, $0.20 per million cached input tokens and $10.00 per million output tokens, with cache writes at $2.00 per million for the default 5-minute lifetime and $4.00 for 1 hour, according to the Kimi API pricing page on November 7, 2026. Prices are flat across the whole 1M-token window, and batch jobs bill at 60% of the standard rate.

For the platform team, an agent turn that reuses a cached 240K-token prefix, adds 16K new tokens and writes 4K back costs $0.048 + $0.032 + $0.040, about 12 cents. Twenty engineers running 150 turns per workday for 21 workdays is 63,000 turns, or about $7,560 a month.

The Kimi K3.1 Pro License is a modified MIT license that allows commercial use with two conditions. A company running a "Model as a Service" business with more than $20 million in revenue over 12 months needs a separate agreement with Moonshot. And a product with more than 100 million monthly users or $20 million in monthly revenue must display "Kimi K3.1-Pro" in its interface. Internal use is exempt from both.

Which Setup to Pick

• Already running 8x B300, GB300 or MI355X: the official checkpoint in the kimi-k31 vLLM image at tensor parallel 8, with DCP8 once sessions get long or numerous.
• On H200 or B200: use the API; two-node layouts for this model are not validated.
• A 512 GB Mac Studio: the REAP80 q6 build at about 6 tokens per second, if a code-focused subset of the experts is enough for you.
• A Linux server with 1.2 TB or more of RAM: llama.cpp b12180 or later with a 2-bit GGUF is possible, but definately test its output on your own tasks first, since no quality numbers exist for those files yet.
• Building a product on it: read the Model as a Service and attribution clauses first, then price the API against your own hardware.

Conclusion

Kimi K3.1 Pro needs memory for all 2.8 trillion parameters, and because Moonshot already ships the experts in 4-bit, the 1,561.3 GB checkpoint is close to the smallest full-quality form of the model. Its hybrid attention keeps the cache unusually small, 16,992 bytes per token plus 449.4 MB per session, so the weights decide the hardware: one 8-GPU B300, GB300 or MI355X node, running a vLLM build that has not reached a stable release yet.

For the platform team serving 20 engineers, our pick is the API unless that node is already in the building. About $7,560 a month covers 63,000 agent turns with no queue to manage and no nightly-wheel enviroment to maintain. If you do self-host, budget for decode context parallelism from day one, because without it a single node runs out of cache at nine full-length sessions.

Related DevToolLab Tools

• Download Time Calculator - see what 1,561 GB of weights means on your link: about 3.5 hours at 1 Gbps, about 21 minutes at 10 Gbps.
• GGUF Viewer - read a community GGUF shard's quant types from its header before you pull a terabyte.
• Safetensors Viewer - read an official shard's header to see the U8-packed MXFP4 experts beside the BF16 attention weights.
• Prompt Caching Calculator - enter Kimi's $2.00, $0.20 and $10.00 rates in custom mode to see what cached prefixes save.

Related Guides

• Mistral Large 4 VRAM Requirements does the same math for a 1 trillion parameter model that fits a 512 GB Mac at 2-bit.
• Local LLM VRAM Requirements explains the general formula for models that fit on one GPU.
• Best GPU Cloud Providers for AI Workloads in 2026 covers where to rent the Blackwell nodes this model needs.
• Local LLM Hardware: GPU vs Mac in 2026 compares unified memory with discrete VRAM for local inference.