标签：#r

Local LLM inference has an expensive habit: It recomputes prefixes it has already seen. A system prompt. A reused RAG document. A few-shot block. A long static context. If the prefix is identical, why pay the prefill cost again? That's the problem I explored with EdgeSync-LLM. The idea The mechanism is simple: Prompt = shared prefix + new suffix On the first request, EdgeSync prefills the prefix and captures its KV cache state. On the next request sharing that exact prefix, it restores the state and decodes only the new suffix. No llama.cpp fork. No patch. The current validated path uses the public: llama_state_seq_get_data and llama_state_seq_set_data APIs. Measured on a real Android ARM64 phone Model: Qwen2.5-0.5B-Instruct Q4_K_M Shared prefix: 123 tokens 40 requests. 4 threads. Release build. Path Mean TTFT p50 p95 Cold 4828 ms 4752 ms 5297 ms KV state reuse 486 ms 476 ms 569 ms 9.9× lower TTFT on cache hits. The warm path was approximately: 363 ms to decode the 10-token suffix 123 ms to restore the state blob Fragment size: 1.64 MB I also measured the same mechanism on x86-64. Cold mean TTFT: 1395 ms Warm mean TTFT: 185 ms That's 7.5× on cache hits. But I almost published a fake 8.8× speedup This was the most important part of the project. My first implementation directly copied raw K/V tensors. It was fast. Very fast. The benchmark reported an 8.8× speedup. There was one problem. It was wrong. llama.cpp tracks more than the K/V tensor values. Cache cells also have position and sequence metadata used to construct the attention mask. Copying tensor values without restoring that bookkeeping produced an inert fragment. The model skipped prefix computation... ...but attention could not actually see the restored prefix. 14 of 24 cache hits reproduced, token for token, the output of a generation with no prefix at all. The “speedup” was dropped context. So I discarded it. Timing is not enough A broken cache can be fast. That's why EdgeSync now runs two correctness chec

I've been building apps with IndexedDB for years. The local part works fine — store data, query it, show it on screen. The hard part is keeping that data in sync with a server when the network comes and goes. Most tutorials show you how to build an offline app with a framework. Firebase, RxDB, WatermelonDB. Those work, but they bring their own abstractions, their own sync protocols, their own opinions. I wanted something simpler. A database with a sync API that doesn't dictate how my backend works. Here's the setup I landed on. npm: npm install ctrodb Docs: ctrodb.vercel.app/docs/sync/overview What We're Building A notes app that works offline. Create and edit notes on the train, in a tunnel, on a plane. When the network comes back, everything syncs automatically. The database is ctrodb (zero-dependency, browser-based). The backend is anything that speaks HTTP. Step 1: Database Setup import { Database , syncPlugin , HttpTransport } from " ctrodb " const db = new Database ({ name : " notes-app " , schema : { version : 1 , collections : { notes : { fields : { title : { type : " string " , required : true }, body : { type : " string " }, updatedAt : { type : " string " , default : () => new Date (). toISOString () }, }, indexes : [{ field : " updatedAt " }], }, }, }, }) await db . connect () Every collection you want to sync needs a timestamp field. The sync engine uses it to order changes and detect conflicts. Plugins are passed in the Database constructor via plugins array: const transport = new HttpTransport ({ url : " https://api.myapp.com/sync " , }) const db = new Database ({ name : " notes-app " , schema : { ... }, plugins : [ syncPlugin ({ transport })], }) await db . connect () The transport takes a single base URL and appends /push and /pull automatically. The sync plugin hooks into every write operation and records it in the change log. The plugin exposes devtools that take the database instance as their first argument: import { inspectSyncQueue , retryFaile

If you've ever bid on a public sector contract, you know the PQQ drill. Someone sends you a Word document with 47 questions spread across 6 sections. Company info. Technical capability. Financial standing. Health & safety. References. Maybe something about modern slavery or carbon reporting because it's 2026 and everything has to check everything. You have to: Read every question Figure out what category it falls under Decide which ones are easy and which will take a week Dig up the right evidence for each one Track word limits And you're doing this at 10pm because the submission deadline is Friday. I got tired of doing this manually, so I built a free tool that does it in one click. What it does PQQCheck takes any PQQ document — pasted raw, formatting and all — and runs it through an LLM that understands procurement documents. It returns: Every question extracted — no more re-reading the document to check you didn't miss one Category tags — Technical, Financial, H&S, Insurance, etc. Difficulty ratings — Easy / Medium / Hard at a glance so you know where to start Suggested evidence — what to prepare for each question Word limits — pulled straight from the document Here's what the output looks like: | Question | Category | Difficulty | Suggested Evidence | Limit | |-----------------------------------|-------------|------------|----------------------------|-------| | Provide your registered name & no | Company | Easy | Certificate of Incorporation | 50 | | Describe IT managed services exp | Technical | Hard | 3 case studies + CVs | 500 | | Provide H&S policy | H&S | Easy | Current policy document | — | | ISO 27001 certification details | Technical | Medium | Certificate + scope doc | 200 | Why this matters for procurement teams Most PQQ response planning is reactive. You read the document, start answering, and discover mid-way that a question needs a certificate you don't have or a reference you can't get in time. PQQCheck flips that. You know before you start writing

AI Doesn’t Replace Agile. It Makes Good Agile More Important. The discussion around AI replacing Agile is becoming increasingly common. The argument usually goes something like this: Information is now instantly accessible. Code can be generated in hours instead of weeks. Documentation is no longer expensive to produce. Communication overhead is dramatically reduced. If all of that is true, do we still need Agile? I believe the answer is yes—but perhaps not in the way we practice it today. The mistake is assuming Agile is defined by stand-ups, sprint planning, retrospectives, or two-week iterations. Those are practices, not principles. The real purpose of Agile has always been much simpler: Deliver customer value incrementally while maintaining enough structure to ensure quality, accountability, and continuous learning. That objective hasn’t disappeared because AI became faster. AI Changes Execution, Not Responsibility Large language models can generate code, documentation, tests, infrastructure, and even architecture proposals. What they don’t generate is accountability. In enterprise environments—especially regulated industries—the question is rarely “Who wrote this code?” The real questions are: Who owns this decision? Why was this solution selected? Can we trace how we arrived here? Can we audit the process? Who is responsible when something fails? Without clear ownership and controlled handoffs, AI can produce enormous amounts of output that become increasingly difficult to understand, validate, or maintain. Speed without governance simply creates technical debt faster. Coordination Isn’t Going Away Many people assume AI eliminates the need for coordination. I would argue the opposite. As AI agents begin collaborating with humans—and eventually with other AI agents—the need for explicit coordination actually increases. Someone still needs to define: objectives, responsibilities, interfaces, quality gates, acceptance criteria, governance, and success metrics. Th

After setting the music gear corner of the internet on fire back in 2023 with the first glimpse at the Nopia, creators Martin Grieco and Rocío Gal are almost ready to bring it to market. The duo brought it to the MusicRadar offices for an in-depth first look and revealed that it will be launching […]

Hello HN! I'm the author of Ant, a JavaScript ecosystem built around a runtime with its own JavaScript engine. Ant also includes a package manager, the ants.land package registry, a platform for deploying and hosting applications, and Ant Desktop for building native desktop apps with web technologies, similar to Electron. The goal is for these pieces to work as one coherent platform while remaining compatible with the wider JavaScript ecosystem. It's still early, and I'd appreciate any feedback

GitHub热门项目 | HTML is the new markdown. Lavish is the new editor for your HTML artifacts. | Stars: 1,811 | 265 stars this week | 语言: JavaScript

Ever wondered what would happen if the world's leading Large Language Models settled their benchmark disputes in a 2D cybercity arena? It's easy to look at model performance on standardized benchmarks (like MMLU, MATH, or HumanEval). It is much more fun to visualize their underlying architectures, parameter scales, and hardware constraints as a retro-cyber fighting game. So, we built Model Kombat (Mixture of Experts Edition)! 🕹️ Play Directly Here 🎮 Launch Game in Full Screen 🧬 Playable ML Concepts Explained This isn't just a basic stick-figure fighting game. Every mechanic—from rendering complexity to the speed at which characters recover—is a direct, playable representation of real-world Large Language Model engineering. 1. 📐 Parameter Scaling vs. Render Tiers A model's representation capacity (intelligence) scales with its parameter count. In Model Kombat, a fighter's visual complexity, joint detail, and rendering fidelity directly reflect its real-world parameter size: Tier 1 (< 5B Parameters - Gemma 2B, Llama 3.2 3B) - Primitive Capsules : Drawn as simple, single-color flat limbs with low joint segmentation. This visualizes the limited representation capacity and coarse output resolution of small edge models. Tier 2 (7B - 14B Parameters - Mistral 7B, Claude Haiku) - Simple Vectors : Structured as thin skeletal wireframe vectors. Tier 3 (14B - 35B Parameters - Gemini Flash, Mixtral) - Two-Tone Vectors : Rendered as dual-color, layered vector limbs. Tier 4 (35B - 100B Parameters - Llama 8B, Claude Sonnet) - Cyborg Shading : Rendered as detailed vector cylinders with dynamic code particle streams flowing along their limbs. Tier 5 (> 100B Parameters - o3, GPT-4o, Claude Opus) - Quantum Vectors : Rendered as glowing vector limbs with digital matrix code particles, soft drop-shadow depth buffers, and real-time afterimage motion trails. 2. ⚡ Reasoning Tokens & KV-Cache Overcharging Instead of arbitrary "mana" or "stamina," fighters charge a Ki bar representing interna