标签：#backendarchitecture

webmcp agentic web: Why Backend Engineers Must Rethink Their Architecture Quick Answer webmcp agentic web: Agentic web workloads over MCP require stateless gateways, distributed context stores, prompt caching, and fine‑grained telemetry to keep latency below 350 ms and cost under control. Latency and State in Multi‑Agent LLMs When a Multi‑Agent System talks to an LLM over the Model Context Protocol (MCP) , the assumptions that hold for CRUD REST APIs break apart. A 200‑ms timeout that covers a simple GET request now collapses into a 2‑second latency spike because each tool call injects a new sub‑prompt, inflates the token budget, and forces the backend to stitch together dozens of partial contexts. In the field, the LLM behaves like a stateful, high‑throughput service that must be orchestrated, not a stateless function. Real‑World Example Consider a U.S. e‑commerce platform that needs to serve 12 k concurrent shopping sessions. Each session spawns up to five agents (pricing, inventory, recommendation, fraud, checkout). The platform’s existing micro‑service stack was built for single‑shot CRUD calls; when the agentic layer was added, the following issues surfaced: Context drift: stale prompts silently degraded recommendation quality. Token explosion: every tool call added 200–300 tokens, pushing the total payload past 8 k tokens. Throughput hit: the MCP service was throttled by Azure OpenAI’s per‑deployment request rate limits. After re‑architecting to a stateless MCP gateway backed by a distributed context store, the platform maintained 99th‑percentile latency under 350 ms even during a Black Friday surge. Trade‑Offs Aspect Option A Option B When to choose Context Storage Redis Cluster (in‑memory, low latency) Cosmos DB (strong consistency, global replication) Redis for ultra‑low latency, Cosmos for compliance or multi‑region writes Prompt Caching Enable KV‑cache on Azure OpenAI Re‑send system prompt on every request Enable when prompt size >20% of total token budge