Why DDR5 Bandwidth Kills Dual-LLM Inference on APUs (Benchmarks Inside)

Did you know that a 35-billion-parameter model can generate tokens at the same compute cost as a 4B model? That single fact made me abandon a multi-model agent architecture I'd spent a weekend building. But I had to run the benchmarks first to understand why. Here's the full breakdown, with commands, numbers, and the architectural reason it all falls apart on shared-memory hardware. The Discovery That Changed Everything I'd been running qwen3.6:35b on my Minisforum UM790Pro for weeks as my daily coding assistant. 17.8 tokens/second -- genuinely usable for interactive work. But I kept wondering: could I run a lightweight sidecar model alongside it for quick classification and tool-calling in an agent pipeline? Before I even started benchmarking, I dug into what qwen3.6:35b actually is under the hood. It's a Mixture of Experts model: 256 total experts with only 8 activated per token. The architecture also incorporates SSM (State Space Model) components alongside traditional attention -- Mamba-style layers that handle certain sequence patterns more efficiently than pure transformers. The math hit me: 8 out of 256 experts means each token only touches roughly 4-5B parameters worth of compute. The model carries 36 billion parameters of knowledge , but its per-token cost is comparable to a small dense model. I was planning to run a separate 4B model for "fast tasks" next to a model that already operates at 4B-class speed. But I had to prove it with numbers. Hardware and Ollama Setup The UM790Pro specs that matter for this experiment: CPU: AMD Ryzen 9 7940HS (Zen 4, 8C/16T) iGPU: AMD Radeon 780M (12 RDNA 3 compute units) RAM: 96 GB DDR5-5600 (~80 GB/s bandwidth) GPU memory pool: 2 GB dedicated VRAM + 46 GB GTT = 48 GB GPU-accessible That 48 GB GPU pool sounds enormous until you realize it's carved from the same DDR5 that the CPU also uses. There is no separate GDDR6 bus. Everything -- CPU inference, GPU inference, KV caches, OS operations -- flows through one 80 GB/s pipe.