Deploying Qwen3.8-2.4T-A95B with vLLM: Verified GPU Pods, Quants, and Serving Recipes
Qwen3.8-2.4T-A95B is a 2.4-trillion-parameter Mixture-of-Experts model with roughly 95B parameters active for each token. If you're planning to self-host it, the first thing to know is that this is a genuinely large distributed model: even the low-precision checkpoints are measured in terabytes. The official open checkpoint is: Qwen/Qwen3.8-2.4T-A95B The model has 512 routed experts and selects 10 of them per token alongside one shared expert. Its 92-layer backbone mixes 69 Gated DeltaNet linear-attention layers with 23 full-attention layers, with full attention appearing every fourth layer. Native context is 262,144 tokens , with an extended configuration available up to roughly 1.01 million tokens . The open checkpoint is text-only and always uses reasoning. This is different from Qwen's hosted Qwen3.8-Max service, which adds features such as vision input and non-thinking mode. For GPU deployment, the main decision is not whether 2.4T parameters will somehow fit. It is which precision format gives you a documented configuration on the hardware you actually have . Start with the checkpoint that matches your GPUs The practical options today are: Your GPUs Checkpoint Documented setup 8× B300 Inferact/Qwen3.8-2.4T-A95B-NVFP4 TP8 8× GB300 Inferact/Qwen3.8-2.4T-A95B-NVFP4 TP8 across two NVL4 trays 16× B300 Qwen/Qwen3.8-2.4T-A95B-FP8 TP16 16× GB300 Qwen/Qwen3.8-2.4T-A95B-FP8 TP16 12× GB300 Qwen/Qwen3.8-2.4T-A95B-FP8 TP4 × PP3 8× MI355X Inferact/Qwen3.8-2.4T-A95B-MXFP4 TP8 The full BF16 checkpoint is roughly 4.45 TiB . The official FP8 version is around 2.27 TiB , while the NVFP4 checkpoint used in the NVIDIA eight-GPU recipe is around 1.32 TiB . That is why NVFP4 is the most approachable NVIDIA deployment if your goal is simply to get Qwen3.8 running without moving immediately to a 16-GPU cluster. H100, H200, A100, B200 and smaller GPU configurations are not included here. Current vLLM material contains sizing information for some of those GPUs, but not equivalent end-to