今日已更新 412 条资讯 | 累计 19972 条内容
关于我们

737x faster LangGraph checkpoints, and the case where Rust lost

Dipankar Sarkar 2026年07月11日 23:24 2 次阅读 来源:Dev.to

Run a LangGraph agent long enough and the model call stops being your bottleneck. The plumbing takes over. Every step, the graph serializes its state to a checkpoint so you can resume, replay, or recover. LangGraph does that with Python's deepcopy . For a small dict that is fine. For a 250KB agent state with nested messages, tool outputs, and accumulated context, deepcopy is brutally slow, and you pay it on every single step of a long run. So I built fast-langgraph : a set of Rust accelerators for the hot paths in LangGraph, packaged as drop-in components that keep full API compatibility. Lead with the numbers, including the ones that hurt Here is what the Rust paths actually buy you, measured against the Python equivalents: Operation Speedup Where Complex checkpoint (250KB) 737x faster than deepcopy Large agent state Complex checkpoint (35KB) 178x faster Medium state Sustained state updates 13-46x Long-running graphs, many steps LLM response caching 10x at 90% hit rate Repeated prompts, RAG End-to-end graph execution 2-3x Production workloads with checkpointing And the automatic mode, the one that needs zero code changes, lands around 2.8x for a typical invocation. Now the honest part. These are not "Rust is faster at everything" numbers. The checkpoint speedup scales with state size. It is a serialization story. For a small, flat dict, Python's built-in dict is implemented in C and already fast. Rust does not win there, and the README says so plainly. The 737x is a large complex-state number, not a headline you get on a toy graph. The core idea: reimplement the critical paths, keep the API LangGraph is good. I did not want to fork it or replace it. I wanted to swap out the three operations that dominate a real workload: Checkpoint serialization. deepcopy on complex nested state is the single biggest cost in a long run. Rust does a structured serialize instead. State management at scale. High-frequency updates accumulate overhead. A Rust merge path handles append-h

本文内容来源于互联网,版权归原作者所有
查看原文