How to Integrate the OpenAI API into a Production Express App

Last year I helped a startup integrate the OpenAI API into their product. It was a chat feature — users could ask questions about their data and get natural language answers. The integration took about a day. Three days after launch, the founder messaged me: "Hey, something's wrong. Our AWS bill just showed an unexpected charge." It was $340. For three days. They had 60 users. The issue wasn't a bug — it was that production API usage looks nothing like a tutorial. The tutorial shows you openai.chat.completions.create() and returns a response. The tutorial doesn't show you what happens when users send 500-token messages, when they open 15 browser tabs each maintaining their own chat context, or when one user fires requests 30 times per minute because they think it's broken. This guide covers what the tutorials skip: rate limiting, token counting, cost guards, streaming, error handling with retries, and model selection. These aren't optional additions — they're what separates a demo from a production feature. Why Production Is Different Here's the gap between tutorial code and production code, stated plainly: Concern Tutorial Code Production Code Cost control Not mentioned Token counting, spending limits, model selection by task Rate limiting Not mentioned Per-user and per-IP limits to prevent abuse Error handling try/catch that logs to console Typed errors, retries with backoff, user-facing messages Response delivery Wait for full completion, return at once Streaming via SSE — response appears as it generates Context management Each request is independent Conversation history managed, truncated at token limit Secrets management API key hardcoded or in .env (no rotation) Rotation strategy, usage monitoring, per-feature keys Let's build a production-grade Express API that addresses all of this. We'll go layer by layer. The Architecture ┌─────────────────────────────────────────────────────────┐ │ CLIENT (Browser / Mobile) │ │ POST /api/chat { messages: [...] } │ │ GET