Local-First LLM Routing: A Decision Table for Latency, Secrets, and Offline Mode

A field-service team learns the hard way A field-service team built a support chatbot that sent every message to a cloud LLM endpoint. The design held until a technician drove through a tunnel, and the request queue grew into an eleven-minute backlog. The same week, a support ticket containing a customer's account number appeared in a third-party log because the payload was never classified. The fix was not a bigger cloud budget but a local-first router that decides where each request runs. Why cloud-first fails in three specific ways Latency is the first failure mode, because a round trip to a hosted endpoint adds network time on top of model time. Autocomplete-style features feel broken when every keystroke waits for a distant server instead of a local process. Secrets are the second failure, because any payload sent to a third party can leak into logs or vendor systems. Offline is the third, because a tablet in a tunnel simply has no route to the cloud. The decision table that replaces the either-or debate Local inference and cloud APIs are two legs of a routing policy, not a binary choice. Each request deserves an evaluation against the same conditions, and the table below captures those conditions. The router implementation in the next section turns that table into executable logic with a small Python module. The recent wave of free and cheap model announcements makes this decision more urgent, because every new endpoint adds another leg to the routing table. Condition Local model Cloud free server Payload contains PII Always Never Network unreachable Always Never Latency budget under 300 ms Prefer Avoid Task requires strong reasoning Avoid Prefer Local queue deeper than three Avoid Prefer Token budget nearly exhausted Prefer Avoid The table encodes a simple principle: privacy and availability win over capability. Capability wins only when the network is healthy and the payload is safe. The table also exposes the hidden assumption that a local model is always a