Run Qwen 3.8 27B Locally: Real GGUF Sizes, the KV Cache Trick, and the Template Trap

Qwen 3.8 arrived as two different releases with two different licences, and only one of them is something you can put on a card you own. The 2.4 trillion parameter A95B opened up on 12 August under Alibaba's own qwen3.8-max terms. The one that matters for local work is Qwen 3.8 27B , whose safetensors went up on 13 August at 08:23 UTC with an Apache 2.0 LICENSE file following the next morning. Both dates are off the Hugging Face commit log, not a launch post. Here is the practical picture: what it needs, why its long context is unusually cheap, and the one setting that makes people think they downloaded a broken quant. The shape of the model decides everything 27B dense parameters across 64 layers, hidden size 5120. The interesting part is in config.json , where layer_types reads 48 linear attention layers and 16 full attention layers , alternating three to one ( full_attention_interval: 4 ). Only those 16 layers keep a KV cache. The rest of the shape: 24 attention heads with head_dim 256 and 4 KV heads , a 248,320 token vocabulary, and max_position_embeddings of 262,144 . It is a native vision language model, so images and video go in without a wrapper, and the ggml-org pack also ships a multi token prediction head as a separate file. The numbers Sizes below are the file sizes Hugging Face reports for unsloth/Qwen3.8-27B-GGUF , read on 14 August 2026. Packs differ by a few hundred megabytes, so check the repo you actually pull from. lmstudio-community has Q4_K_M at 16.8 GB and ggml-org at 19.0 GB for the same nominal quant. Quant Size on disk Realistic home UD-IQ2_XXS 9.0 GB 12 GB cards, visible quality cost UD-Q2_K_XL 10.7 GB 12 GB cards, almost no context left UD-Q3_K_XL 13.4 GB 16 GB cards Q3_K_M 13.8 GB 16 GB cards IQ4_XS 15.7 GB largest quant that stays whole on 16 GB Q4_K_M (sweet spot) 17.1 GB 24 GB cards Q5_K_M 19.8 GB 24 GB, less context headroom Q6_K 22.9 GB 24 GB barely, or 32 GB Q8_0 29.0 GB 32 GB or a two card split BF16 (from ggml-org ) 53.8 GB server