I Benchmarked Two Local LLMs on Real Dev Work — Qwopus 27B vs Muse Glimmer 30B

I Benchmarked Two Local LLMs on Real Dev Work — Qwopus 27B vs Muse Glimmer 30B Two open-weight models, one 20 GB GPU, two real development tasks, and a third model as the referee. Here is what actually happened when I made Qwopus 3.6 27B and Meta's Muse Glimmer 30B implement a bug fix and then a full feature in my own project. The setup Both models ran fully local on an AMD Radeon RX 7900 XT (20 GB VRAM) via a llama.cpp multi-model router (one OpenAI-compatible endpoint, GGUF models, load-mode=dio — more on why below). Each model was driven by the pi CLI in non-interactive mode with --thinking high . A third model — Codex, through a disciplined stdin wrapper — reviewed both outputs and gave the verdict. The fairness method was simple but strict: One task , described in a markdown spec, copied byte-identical into two isolated git clones of my project. Each model worked in its own clone, its own branch , never seeing the other's work. Objective verification by script: existing test suite + new tests + production build. Cross-review by Codex , examining both branches against the same criteria. The test project: Jeu de Cochons (a "Pass the Pigs" dice game, vanilla JS PWA on Vite + Vitest) — real code, real tests, no toy repo. Qwopus 3.6 27B Muse Glimmer 30B Source Community fine-tune of Qwen 3.6 Meta (distilled from Muse Spark) Size 27B 29.6B Quant IQ4_XS (~15 GB) UD-Q4_K_XL (~14.8 GB) Round 1 — fixing a regression (short task) The project had a broken PWA: a commit that added a /jeu-de-cochons/ base path for GitHub Pages had broken 3 service-worker tests (manifest, precache, offline navigation fallback). Task: fix the regression without touching the tests , keep the other 84 green. Qwopus Muse PWA tests (11) 11/11 ✅ 11/11 ✅ Full suite (87) 87/87 ✅ 87/87 ✅ Files touched 2 2 Diff size +4/−4 +4/−4 Wall time ~8.5 min ~21 min Leftover artifacts none one .bak file The remarkable result: both models produced a byte-identical diff. Same diagnosis (a lost capture group in the a