标签：#homelab

This is the technical, reproducible version of a fix I shipped on my own homelab. If you want the narrative version, that's on Medium. This one is the recipe: the measurements, the math, the Modelfile, and the exact prompt I gave Claude Code to generate it. Copy-paste friendly. Repo for the dashboard used throughout: https://github.com/SikamikanikoBG/homelab-monitor TL;DR One 24GB RTX 3090, two GPU services: WhisperX large-v3 (STT, 7.7GB peak) and a Devstral Small 24B email-triage LLM (Q4_K_M, ~18.3GB). 18.3 + 7.7 = 26GB → CUDA OOM whenever they overlapped. The LLM was loaded with a 40k context window but the triage job never needed more than ~5–8k tokens. Capped num_ctx to 8192 → KV cache drops from ~6.1GB to ~1.25GB → model footprint ~18.3GB → ~14.2GB . 14.2 + 7.7 = 21.9GB → both resident, zero OOM, no quality loss. The setup Host : openSUSE, Xeon (56 threads), 125GB RAM, 1x RTX 3090 (24GB) GPU svc : WhisperX large-v3 (speech-to-text) GPU svc : Ollama -> devstral-small-2 (24B, Q4_K_M) for background email triage Both services run all the time. The OOM only happened when I dictated to my assistant (WhisperX) while the triage loop was active. Step 1 — Make the contention measurable nvidia-smi shows instantaneous VRAM. It can't show you which service spiked or when two of them overlapped — and an intermittent OOM is a timing problem. You need per-service VRAM history. I use my own dashboard (homelab-monitor) for this. The relevant view is "AI Models", which attributes VRAM per model server and per loaded model, over a time range, with OOM markers and a capacity ceiling line. What the history showed at the overlap window: Service Peak VRAM Devstral 24B (triage) ~18.3 GB WhisperX large-v3 7.7 GB Total ~26 GB on a 24 GB card If you want to reproduce the measurement, the dashboard runs as a single container: git clone https://github.com/SikamikanikoBG/homelab-monitor cd homelab-monitor docker compose up -d --build # open http://<host>:9800 -> AI Models / GPU views (NVIDI

A homelab box that never sleeps runs 8,760 hours a year. So the spec that decides what it costs you is not the one on the box. It is the one nobody prints: how many watts it pulls sitting at the login prompt doing nothing. I kept hitting this while shopping for a Proxmox node, so I put the measured numbers in one place. More on that at the end. First, why the spec sheet lies to you. TDP is a thermal budget, not a power reading TDP is the heat the cooler has to handle at full tilt. It is a design target for the heatsink, not a measurement of what the chip draws, and it says almost nothing about idle. Your homelab box spends 95%+ of its life idle, so the number that runs up the meter is idle wall power, and that number is never on the product page. The arithmetic is unforgiving. One watt running continuously is 8.76 kWh a year. So the gap between a 7 W box and a 35 W box is not 28 watts, it is about 245 kWh a year, every year, for as long as the box is on. Plug in your own rate to get the dollars; the point is the gap compounds. Where TDP actively misleads you A few measured results from the dataset I'll link below, all from third-party wall-meter readings, not vendor claims: The new N100 wave is genuinely low. A Minisforum UN100C measures 5 to 7 W at idle. Beelink, GMKtec and Trigkey N100 boxes land in the 6 to 10 W range. For a Pi-hole, a few containers and some light VMs, this tier is hard to beat on running cost. AMD mini PCs idle far higher than their marketing suggests. A Minisforum UM790 Pro measures 25 to 45 W at idle. A Beelink SER6 Pro lands at 20 to 35 W. These are fast little machines, but if you picked one expecting "small box, small draw," the meter disagrees, and over a year that delta is real money. Newer and higher-TDP is not lower-idle. A Dell OptiPlex 7060 Micro idles just over 18 W on its 65 W-TDP desktop chip. The older 7070 with a six-core part sits around 13 W, and the low-power "T" SKUs lower still. The CPU's TDP class predicted idle better tha

You buy a NAS for silent, always-on storage. It sits in a corner, humming quietly, doing its thing. Then you install Docker. Then Syncthing. Then a few containers. Suddenly the HDDs never stop. Seeking, spinning, clicking. Not occasionally - constantly. At 2am you can hear it from the next room. This is what happened, why it happens, and how to make it stop. What's Actually Causing the Noise Mechanical HDDs make noise when the read/write head moves. The more random the I/O - small reads and writes scattered across the disk - the more seeking, the more noise. Sequential writes to a single file are quiet. Random I/O across thousands of small files is loud. Docker and Syncthing are both pathological for HDDs in different ways. Docker overlay2 Docker's default storage driver is overlay2. Every container runs on top of layered filesystems - the image layers are stacked, and a thin writable layer sits on top for each running container. Every file operation inside a container that touches a file from a lower layer triggers a copy-on-write : the entire file gets copied up to the writable layer before the write happens. On an SSD this is fast and silent. On spinning HDDs with mechanical heads, every copy-on-write is a seek, a read, and a write - often scattered across the disk. And it's not just copy-on-write. Docker's overlay2 metadata lives in small files across a deep directory tree. Container startup reads dozens of these. Log rotation writes to them. Health checks touch them. Any container doing anything at all generates constant scattered I/O. Syncthing's 60-Second Heartbeat Syncthing uses BoltDB as its internal database - it stores file indexes, sync state, and peer information there. BoltDB flushes to disk every 60 seconds regardless of whether anything changed. On a busy sync folder with thousands of files, this flush isn't a single sequential write. It rewrites pages across the B-tree structure, which means multiple seeks across the database file. Quiet when it's a