Running Local Private AI Models – How And Why
Originally published at dragosroua.com . Last week, Anthropic released Fable 5. Three days later, the US government ordered them to shut it down — for people outside US. Anthropic said they couldn’t filter users by nationality fast enough, so they pulled the plug on the whole thing. Like any good ol’ miracle, it lasted only 3 days. That was a very much needed cold shower. When you realize someone can take away your workforce just like that, running local, private AI models, suddenly becomes the number one priority. Why You Should Run Your Own Local AI Models In no particular order (because all of them count): No one can take it away. Local AI models on your machine don’t care about export controls, government directives, or provider board decisions. No usage limits. No rate limits, no subscription tiers, no “you’ve used your monthly tokens.” Play as much as you want. Nothing leaves your machine. Your code, your documents, your client data — none of it hits a third-party server. Local AI models are private by default. Fixed cost. You pay for hardware plus electricity. No surprise price hikes mid-year. No API dependency. Your workflow doesn’t break when a provider has an outage, deprecates a model, or gets a compliance letter. You can modify it. Fine-tune, quantize, run on your own data. Build something they can’t sell you. Make your own local, private AI model factory. What It Actually Costs I hear you: but I don’t have the money to build a data center in my basement. Fair play. But here’s the thing: you don’t have to. Here are four realistic options, as of June 2026 money: MacBook Pro M4 Max (~$3,000–4,500) : 546 GB/s memory bandwidth. Runs 70B models at around 70 tokens/second with 4-bit quantization. Fast enough to feel snappy. This is the “you might already own this” option. Mac Studio M3 Ultra (~$5,000–10,000) : 800 GB/s, up to 512 GB unified memory. Runs DeepSeek R1 — a 671-billion-parameter model — at 17–18 tokens/second. That’s a model that costs real money p