今日已更新 299 条资讯 | 累计 41519 条内容
关于我们

We rewrote LF Edge eKuiper in Rust: 425k eps, 8MB RAM, 13ms boot (vs Flink, Go eKuiper & Benthos)

Ankur Kumar Pandey 2026年09月10日 11:30 0 次阅读 来源:Dev.to

Over at I-Dacs Labs , we run high-throughput telemetry pipelines on edge devices (Raspberry Pis, Advantech gateways, and embedded x86/ARM boxes). We've been using LF Edge eKuiper for local stream processing (SQL filtering, sliding windows, and MQTT/Kafka sinks), but kept hitting the classic edge computing wall: JVM engines (Apache Flink) : Incredible throughput, but require >1 GB RAM and take 20+ seconds to boot. Unusable on small industrial hardware. Go engines (Upstream eKuiper, Benthos, Telegraf) : Much lighter, but continuous Stop-The-World GC sweeps introduced tail latency jitter. Worse, under burst sensor loads (10k–100k events/sec), Go channel buffer saturation led to silent packet loss. We decided to rewrite the entire engine in pure Rust: rekuiper (v0.421-beta, dual licensed under MIT / Apache-2.0). --- ### What We Built Core Architecture : Lock-free stream bus ( StreamBus ), Tokio async actors for rule execution, and bounded actor queues for sinks with zero runtime GC pauses. 100% Drop-In Parity : Fully compatible with the existing eKuiper Manager Web UI, OpenAPI 3.0 schemas, and standard streaming SQL. Zero scaffolded stubs across all 98 REST endpoints. Footprint : 9.60 MB stripped static binary, ~6 – 8.2 MB idle RAM consumption. - Sub-15ms Cold Boot : 12.5 – 14.5 ms internal daemon bootstrap; 123 ms end-to-end process-spawn-to-ready. Empirical Head-to-Head Benchmarks (500,000 Records) Rather than hand-waving estimates, we ran all engines head-to-head on the exact same Linux machine (WSL2 / Ubuntu x86_64) using an identical 500,000-record telemetry workload: Pipeline: Parse 500k JSON events → Compute formula ( temp * 1.8 + 32 ) → Filter ( temp > 20.0 ) → Project ( id, temp_f ) → Sink Engine Runtime 500k Elapsed Throughput Data Drops Memory rekuiper (0.421) Pure Rust 1.176 s 425,308 eps 0 (0.0%) ~8 MB Apache Flink Java / JVM 2.144 s 233,209 eps 0 (0.0%) ~1,022 MB Telegraf Go 8.194 s 61,019 eps 0 (0.0%) ~50 MB Upstream Go eKuiper Go 11.290 s 44,287 eps 72,9

本文内容来源于互联网,版权归原作者所有
查看原文