Gemma 4 12B local setup thread — what's your hardware, quant, and use case? [D]

ok so the model's been up on HF now (apache 2.0, ~12B BF16, any-to-any multimodal). community has already shipped a pile of quants: - GGUF: unsloth, bartowski, ggml-org, lmstudio-community - MLX: mlx-community has 4bit / 8bit / bf16 / nvfp4 - official: google/gemma-4-12B-it (BF16) and the -assistant variant still trying to figure out which combo is actually worth downloading. the "12B runs on your laptop" hype is loud but i haven't seen many concrete numbers. if you've got it running, drop: - hardware (chip / RAM / GPU) - which quant + which repo (e.g. unsloth Q4_K_M, mlx-community 4bit, etc.) - runtime (llama.cpp / ollama / lm studio / mlx-lm / vllm / transformers …) - tokens/sec - context length you've actually used in practice - what you're using it for (chat / code / OCR / vision / agent …) - one thing it does well + one thing it falls apart on genuinely curious where the floor is — does it actually work on 16gb or only 32gb+? is mlx noticeably faster than gguf on apple silicon in 2026? anyone using the multimodal side seriously, or is it text-mostly in practice? submitted by /u/Individual_Soil4641 [link] [留言]