Keenable: Agent-First Search API Architecture and the 100B-Page Index Trade-Off

Agents don't search like humans. They issue hundreds of queries per session, need structured extraction over snippet relevance, and care more about p95 latency than the perfect top result. Keenable built a search API around those constraints with a 100B+ page proprietary index, SQL-like query interface, and continuous benchmarking against agent-like workloads. The founders (Amazon AGI web grounding, Yandex search lead) are betting that wrapping existing search APIs won't cut it when agents become the primary consumers of web data. The architecture reveals what changes when you optimize for machine callers instead of human eyeballs. Why Agent Search Needs Different Plumbing Human search optimizes for the first three results and tolerates 500ms variance. Agent search runs in tight loops where every query blocks downstream tool calls. The contract shifts: Query volume : Agents issue 10-100x more queries per task than humans per session Latency budget : p95 matters because agents serialize tool calls; tail latency compounds across multi-step workflows Result consumption : Agents parse structured data, not blue links; relevance scoring for human click-through doesn't align with extraction success Query patterns : Agents use precise filters (date ranges, domain constraints, schema hints) that humans rarely specify Traditional search APIs built for human traffic handle agent workloads poorly. Rate limits assume sporadic queries. Pricing tiers penalize high-volume programmatic access. Relevance models optimize for engagement metrics that don't exist in agent contexts. The 100B-Page Index Decision Keenable maintains its own crawl and index instead of wrapping Google, Bing, or Brave. This is expensive but unlocks control over: Crawl strategy : Agents need fresh data on niche domains that human-centric crawlers deprioritize. A proprietary crawl can target high-churn sources (job boards, pricing pages, event listings) and re-crawl on agent-driven schedules rather than PageRank-