今日已更新 328 条资讯 | 累计 20300 条内容
关于我们

I Benchmarked Lynkr Against LiteLLM on the Same Backends.

Vishal VeeraReddy 2026年06月06日 08:14 5 次阅读 来源:Dev.to

I Benchmarked Lynkr Against LiteLLM on the Same Backends. Lynkr Was Cheaper for Tool-Heavy Workloads Founder disclosure: I built Lynkr, so take this as a technical benchmark write-up, not a neutral industry report. The numbers below come from the same backend providers on both gateways. If you're routing AI coding traffic through a gateway, just switching providers is not enough. The real savings come from reducing the tokens that ever reach the model in the first place. I ran Lynkr and LiteLLM against the same backends — Ollama locally, Moonshot, and Azure OpenAI — across 9 scenarios. On the scenarios that actually look like agentic coding work, Lynkr was cheaper because it does three things before forwarding the request upstream: smart tool selection, TOON compression, and semantic caching. The short version Lynkr was measurably better on the cost-sensitive parts of the workload: Smart tool selection: 53% fewer input tokens, 52% lower cost TOON JSON compression: 87.6% fewer billed tokens on a large tool result, 50% lower cost Semantic cache: 171ms cache-hit response vs 3,282ms on the repeat query path Tier routing: escalated hard prompts to stronger models instead of blindly sending everything to the cheapest route Area Lynkr result Why it mattered Tool selection 53% fewer tokens Removes irrelevant tool schemas TOON compression 87.6% fewer tokens Shrinks large JSON tool outputs Semantic cache 171ms cache hit Avoids repeat model calls Tier routing Escalates hard prompts Doesn’t over-optimize for cheapest path This matters if you're running Claude Code, Codex, Cursor, or similar agent workflows where tools, file reads, grep output, and repeated context dominate your token bill. Setup Same benchmark inputs, same providers, same request shape. Machine: macOS on Apple Silicon Lynkr: v9.3.2 on Node 20 LiteLLM: v1.87.1 on Python 3.12 Backends used: Ollama local, Moonshot, Azure OpenAI Scenarios: 9 total across simple prompts, tools, history, cache, and routing Each scena

本文内容来源于互联网,版权归原作者所有
查看原文