标签：#ORM

A slow AI feature rarely fails all at once. It starts with a longer prompt, then a bigger retrieval result, then one more tool call, then a retry path nobody measured. The demo still works, but users feel the delay before your dashboard explains it. That is why small AI product teams need an LLM latency budget before they start optimizing. Not a vague goal like “make it faster.” A budget says how much time each stage is allowed to spend, what happens when it exceeds that limit, and which user experience is still acceptable when the model, retrieval layer, or tool chain slows down. The payoff is practical: you stop guessing where the delay lives, stop overpaying for wasted work, and make AI workflows feel reliable even when traffic, context, and providers are messy. Why latency budgets matter now Recent AI platform news points in one direction: AI workflows are becoming longer, more tool-heavy, and more expensive to run without discipline. A current news scan showed several signals builders should notice: Production LLM cost and latency guidance is shifting from “add more compute” to “remove wasted work.” Agent environments are being designed for long-running background tasks, persistent state, and cheaper idle time. New model releases emphasize tool use, computer use, multimodal context, subagents, and larger context windows. AI gateways and enterprise platforms are adding cost controls, routing, caching, audit trails, and usage limits. Developers are asking more practical questions about why AI coding and agent workflows interrupt flow with repeated prompt-wait-evaluate loops. For AI SaaS builders, this means latency is no longer just a model selection problem. It is a workflow design problem. A simple chat completion might have one bottleneck. A real AI workflow may include: request queueing auth and tenant checks prompt assembly memory lookup vector search reranking model routing tool calls browser or API actions structured output validation fallback attempts str

The dashboard says forty instances, up from twelve this morning. The autoscaler did its job: it saw latency climb and threw hardware at it. And latency got worse. Not flat. Worse. You're paying for three times the compute to serve a slower product. Somewhere under all forty of those boxes is a single thing they're all waiting in line for, and every instance you add makes the line longer. Horizontal scaling multiplies work that doesn't have to coordinate. The instant the work does have to coordinate, more instances make it slower. Amdahl wrote this down in 1967: the serial fraction of a job sets a hard ceiling on how much faster you can go, no matter how much hardware you throw at the parallel part. Neil Gunther's Universal Scalability Law goes further: past a certain point, the cost of nodes coordinating with each other bends the curve back down. Add capacity, get less throughput. That ceiling was not set by the autoscaler, and it will not be moved by the autoscaler. It was set a long time before this morning, in a room, by whoever decided where the state lives and who has to touch it at the same instant. Now hand the service to a fleet of agents. It writes you something that looks built to scale: stateless handlers, a tidy repo, green tests, a canary that bakes fine at 1% traffic. Every gate you trust says ship it. And the bottleneck is sitting right there in the design, invisible to all of it, because the mistake isn't in the lines, it's in the shape. You cannot catch a shape problem by reading a diff. Name the hot state before you pick a framework. Where does the contended state live, and which requests touch it at the same instant? Answer that out loud, before anyone opens an editor. The tool is downstream of that answer, every time. Originally published at https://imacto.com/writing/scale-is-a-design-not-a-dial . Written with Claude Opus 4.8.

Introduction When you build games with Unity, you eventually run into the problem of managing static game data—often called master data in Japanese game development. At first, ScriptableObject may be more than enough. If your project has a few dozen items, a few dozen enemies, and only a small number of stage definitions, ScriptableObject is convenient because you can inspect and edit everything directly in the Unity Editor. As the project grows, however, the situation changes. You may end up with tables for items, characters, skills, quests, rewards, shops, gacha pools, stages, enemy placements, progression curves, and localization text. The data is no longer edited only by programmers. Planners and game designers may need to work with it in Excel or Google Sheets. At that point, the problem is no longer just choosing a file format. You need to think about questions such as: How do you load a large amount of data quickly? How do you write ID lookups and composite-key queries safely? Should CSV or JSON be parsed directly at runtime? Is it reasonable to create a large number of Dictionaries? How do you validate references between tables? How do you debug data after converting it to binary? How do you connect the source data edited by planners to the data loaded by Unity? For the runtime loading and lookup part of that problem, one strong option is Cysharp's MasterMemory . The official README describes MasterMemory as a “Source Generator based Embedded Typed Readonly In-Memory Document Database” for .NET and Unity. In practical terms, you define your schema as C# types, a Source Generator creates a typed read-only in-memory database API, and the application loads MessagePack binary data that can be queried through type-safe methods. The official README highlights performance compared with SQLite, low allocation during queries, a small database size, and generated database structures that are type-safe and IDE-friendly. Cygames Engineers' Blog also has useful articles

Microsoft Corp. today released software updates to plug at least 570 security holes in its Windows operating systems and other software, almost triple the number of vulnerabilities the software giant fixed in its record-smashing Patch Tuesday release last month. Microsoft attributed the burgeoning patch counts to vulnerability discoveries aided by artificial intelligence.

“On-device AI saves battery” and “cloud AI is more efficient” can both sound plausible. Neither is a measurement. The placement decision crosses at least four different budgets: user wait + network transfer + provider spend + device energy Do not collapse them into one vague “cost” number. Measure each with its own unit and evidence boundary. Start by identifying the actual execution path I reviewed MonkeyCode mobile code at commit c58bcd4 . The task stream opens a server-supported WebSocket. The speech-to-text hook also participates in a server-supported streaming path. That reviewed path is not evidence of on-device model inference. So a fair current study would measure a mobile client using remote task and voice services. An on-device alternative would be a separate prototype with its model, runtime, and packaging declared. Record a measurement envelope The included CSV template begins with these fields: sample_id,sample_kind,placement,device,os,framework,model,network,input_tokens,output_tokens,latency_ms,bytes_up,bytes_down,energy_joules,cost_usd Why so many? device , os , and framework make thermal and runtime results interpretable; model and token counts keep workload size visible; network separates offline, Wi-Fi, and cellular behavior; latency is milliseconds, transfer is bytes, energy is joules, and provider spend is currency; sample_kind prevents synthetic examples from masquerading as device measurements. Battery percentage is too coarse for short runs. It is affected by display, radio, background work, battery health, temperature, and OS estimation. If you cannot collect energy with an appropriate platform profiler or external power measurement, leave energy_joules empty. Use matched user flows Compare the same tasks, not unrelated model demos: Flow Cloud case On-device case Short prompt Same input and output cap Same semantic task and cap Voice turn Same audio fixture Same audio fixture Offline Expected failure or queued action Local completion if supp

Rendering a big image on iOS is one of those things that looks trivial until your app gets killed by the OS mid-export. CGContext , draw, makeImage() , done — except the moment the output gets large, that innocent-looking pipeline quietly asks for gigabytes of RAM and iOS terminates you. I hit this wall building Mozary , an iOS app that packs 100+ photos into a single giant picture (a photo mosaic). In v1.1.0 I finally killed the "high-resolution export crashes with out-of-memory" bug for good. The fix: stop putting the canvas in RAM at all. Put it in a memory-mapped file and let the OS page it to disk. RAM usage dropped from "4.8 GB, please die" to a few dozen MB, flat, regardless of output size. This post is the walkthrough — with the actual Swift. If you've ever seen Core Graphics blow up on a big image (mosaics, collages, stitched panoramas, high-res rendering — same trap), this is for you. TL;DR A non-compressed bitmap costs 4 bytes/pixel . A 1.2-gigapixel image = ~4.8 GB of RAM just for the canvas. CGContext(data: nil, ...) allocates that in RAM. context.makeImage() then copies it again . Double death. Back the canvas with a memory-mapped file ( mmap ). Writes transparently page out to disk and don't count against your app's memory footprint . Wrap that same mapping in a CGImage via CGDataProvider — zero copy — and stream it straight to a JPEG on disk. Never call makeImage() . Decode source tiles at their draw size , not full size. Because you now spend disk instead of RAM: add a free-space pre-flight check and clean up temp files after a crash. Let's dig in. The problem: "compressed file size" is a lie about memory Mozary lays photos out on a grid. A typical high-res export is a 200 × 267 grid with each tile drawn at 150px : width: 200 × 150 = 30,000 px height: 267 × 150 = 40,050 px That's ~1.2 gigapixels . Here's the part people underestimate: the final JPEG is only a few hundred MB to ~2 GB because JPEG is compressed . But while you're drawing, the canvas i

DoorDash details the architecture behind Ask DoorDash, its AI-powered conversational shopping assistant, combining LLMs, specialized AI agents, MCP-based tooling, and an intelligence layer with persistent consumer memory and live backend data. Early results show up to 24% higher checkout conversion, 17% larger baskets, and improved intent accuracy using memory-backed sessions. By Leela Kumili

Davide de Paolis discusses the realities of rolling out cloud infrastructure compliance without fracturing developer relations. Drawing from a real-world platform team reboot at Sevdesk, he explains how to implement "minimum viable governance" on AWS, utilize event-driven Slack alerting to automate policy feedback, and shift from rigid enforcement to high-empathy, data-driven collaboration. By Davide de Paolis

The web has changed . Applications are no longer simple HTTP servers. Today we build real-time dashboards, AI-powered services, multiplayer systems, APIs, microservices, and applications that need to handle thousands of connections with minimal overhead. But our frameworks are still mostly designed for yesterday's problems. So we asked a simple question: What if a * Go * framework was built from the ground up for modern workloads? Meet Breeze . A high-performance Go framework designed around one idea: Performance should not come at the cost of developer experience. Why Breeze ? Go already gives us incredible performance. But the framework layer often becomes the bottleneck. Too much abstraction. Too many allocations. Too much hidden complexity. Breeze takes a different approach: ⚡ High-performance networking powered by gnet 🔥 Real-time WebSocket architecture built in 🧩 Modular middleware system 📚 Automatic Swagger/OpenAPI generation 🎨 Built-in SPA template engine 🚀 Optimized worker pool architecture 🗄️ BreezeORM for efficient database operations Everything you need to build production-grade applications — without assembling dozens of unrelated tools. The Future Is Real-Time Modern applications are moving toward instant experiences: Live collaboration Trading platforms AI assistants Gaming backends Monitoring systems Real-time analytics Breeze is designed for this world. Instead of adding real-time capabilities later, Breeze treats them as a first-class citizen. Less Glue Code. More Building. A common problem in backend development: You start with a simple API... Then suddenly you need: Authentication Documentation WebSockets Background workers Database optimization Frontend integration Your stack becomes a collection of disconnected pieces. Breeze tries to bring these pieces together into one coherent ecosystem. Built With Go Philosophy Go was created around simplicity, performance, and reliability. Breeze follows the same principles: Simple APIs. Predictable behavi

Videos are one of the heaviest assets you can add to a web page. Loading videos too early can significantly impact your application's performance. The good news is that modern browsers are starting to support lazy loading for video elements , allowing you to defer loading until users are likely to watch them. However, there's one important thing to know: 👉 This feature is not yet part of the Baseline web platform , so browser support is still limited. At the time of writing, lazy loading for <video> elements is supported in Chromium-based browsers such as Google Chrome , Microsoft Edge , and Opera , while browsers like Firefox and Safari do not yet support it natively. In this article, we'll explore: What lazy loading videos is Why it's important for web performance How to implement it Browser support considerations Best practices for optimizing video loading Let's dive in. 🤔 What Is Lazy Loading for Videos? Lazy loading means delaying the loading of a resource until it's actually needed. Instead of downloading every video immediately during page load, the browser waits until the video is close to entering the viewport. This helps reduce: initial network requests bandwidth usage page load time memory consumption Especially on pages with multiple videos, the difference can be significant. 🟢 What Problem Does It Solve? Imagine an e-commerce page with several product videos. Without lazy loading: every video starts downloading immediately bandwidth is consumed even for videos users never watch page rendering may become slower This negatively impacts; Largest Contentful Paint (LCP), Time to Interactive (TTI), and overall user experience. Most visitors won't watch every video on the page. So why load them all? Lazy loading ensures videos are fetched only when they're actually needed. 🟢 How to Lazy Load a Video The easiest approach is using the loading="lazy" attribute. Example: <video controls loading= "lazy" poster= "/preview.jpg" > <source src= "/video.mp4" type= "vide

Throughout my career I have had to work with quite a few platform teams, and I was part of two for a couple of years. Some were bad, some were good and some should not have existed at all. I want to tell you my user experience, what I have seen work and what not and what you should definitely avoid doing. Be The Multiplier This is the main goal of a platform team. As a team, it needs to be a multiplier. If the platform team supports 10 teams, then each work it commits should multiply by 10. If the team member builds a new feature, it should be helpful for all the other teams. Otherwise, the platform team is an addition, and in most cases it is then better to split the platform team and add them to all other teams, instead of being a separate team. Because the amount of communication needed is in most cases quadratic in relation to the number of teams. Reducing Cognitive Load The platform should take away cognitive load for all the teams it supports. By doing so, they will have more time to implement business requirements. Let's say a platform team provides Gitlab runners or Azure Agents where people can run their CI/CD code on. They should not need to know how the runners are scaled, or how the agents are updated. This takes away the need for that skill set in all the teams. Build a Community A platform team has a unique position. It is building something that all other developers probably can build as well. Some could do it even better than the platform team itself. For some platform teams, their ego sometimes comes in to play or just straight up refuses their help because they are not the team. But it is not the job of the platform team to build a product, but a platform where everyone can thrive and/or build on. So onboard the community on the platform! The Law of Diffusion applies to almost all companies. You will have the innovators that want to build it themselves and the early adopters that will voice their opinion, but will not build it themselves. Those two

Headline: React Compiler — formerly React Forget — shipped stable with React 19 and automatically memoizes components, hooks, and callbacks by analyzing data flow at build time. No dependency arrays to write; the compiler infers them. Here is what it handles, when it opts out, and whether you should delete your useMemo calls. Key takeaways React Compiler inserts useMemo , useCallback , and React.memo automatically at build time — no dependency arrays to maintain. Enable it in Next.js 15/16 with experimental.reactCompiler: true in next.config.ts . The compiler is conservative: if it cannot prove memoization is safe, it emits the component unchanged. "use no memo" is the escape hatch for functions the compiler should not touch. Run npx react-compiler-healthcheck@latest before enabling to see coverage and violations. What does React Compiler actually do? React Compiler transforms component and hook code at build time to insert memoization automatically. Instead of useMemo(() => expensiveCalc(a, b), [a, b]) , the compiler analyzes data flow, determines which values are stable across renders, and emits equivalent memoized code. The compiled output uses React's memo infrastructure at runtime. The compiler is babel-plugin-react-compiler — it works with any Babel-based build pipeline. How do I enable it in Next.js? // next.config.ts const nextConfig = { experimental : { reactCompiler : true , }, }; export default nextConfig ; Before enabling, run the healthcheck: npx react-compiler-healthcheck@latest The healthcheck reports optimizable component count, files with violations, and blocking patterns. Fix violations first for more coverage on day one. What does the compiler memoize? Components — equivalent to React.memo ; re-renders only when props change. Values — equivalent to useMemo ; computed results, derived arrays, objects. Callbacks — equivalent to useCallback : event handlers, functions passed as props. Dependencies are inferred from escape analysis — n

Headline: Partial Prerendering (PPR) in Next.js serves a static HTML shell from the CDN edge instantly, then streams Suspense-wrapped dynamic children from the origin in the same HTTP response. No full-page ISR staleness, no full-page origin latency. I shipped it on two production routes — here is the model. Key takeaways PPR serves a static HTML shell from the CDN edge , then streams dynamic Suspense children from the origin in the same response. The static shell is built at build time — outside <Suspense> renders statically; inside renders dynamically per request. PPR replaces the ISR vs. dynamic tradeoff for pages that are mostly static with isolated personalized sections. No changes to Server Components or Suspense — just experimental.ppr: 'incremental' in config and export const experimental_ppr = true per route. PPR and use cache are complementary : CDN delivery for the shell, origin memoization for dynamic islands. What does PPR actually do? PPR splits a page into two rendering phases within the same HTTP response. At build time, Next.js freezes everything that does not read dynamic request data into a static HTML shell on the CDN edge. At request time, the CDN delivers the shell at edge latency while the origin streams each <Suspense> boundary's content into the same response. On a product page: navigation, title, and description arrive at CDN speed. The in-stock badge and personalized recommendations stream from the origin a fraction of a second later. The user sees a nearly-complete page immediately. How is PPR different from ISR and streaming Suspense? Strategy First byte Dynamic freshness Staleness ISR (revalidate: N) CDN edge Whole page up to N seconds stale Full page Dynamic rendering Origin 100% fresh; waits for slowest query None Streaming Suspense (no PPR) Origin Fresh; TTFB includes origin latency None PPR CDN edge Dynamic islands 100% fresh Static shell only How do I enable PPR? // next.config.ts export default { experimental : { ppr : ' inc

Local LLM inference has an expensive habit: It recomputes prefixes it has already seen. A system prompt. A reused RAG document. A few-shot block. A long static context. If the prefix is identical, why pay the prefill cost again? That's the problem I explored with EdgeSync-LLM. The idea The mechanism is simple: Prompt = shared prefix + new suffix On the first request, EdgeSync prefills the prefix and captures its KV cache state. On the next request sharing that exact prefix, it restores the state and decodes only the new suffix. No llama.cpp fork. No patch. The current validated path uses the public: llama_state_seq_get_data and llama_state_seq_set_data APIs. Measured on a real Android ARM64 phone Model: Qwen2.5-0.5B-Instruct Q4_K_M Shared prefix: 123 tokens 40 requests. 4 threads. Release build. Path Mean TTFT p50 p95 Cold 4828 ms 4752 ms 5297 ms KV state reuse 486 ms 476 ms 569 ms 9.9× lower TTFT on cache hits. The warm path was approximately: 363 ms to decode the 10-token suffix 123 ms to restore the state blob Fragment size: 1.64 MB I also measured the same mechanism on x86-64. Cold mean TTFT: 1395 ms Warm mean TTFT: 185 ms That's 7.5× on cache hits. But I almost published a fake 8.8× speedup This was the most important part of the project. My first implementation directly copied raw K/V tensors. It was fast. Very fast. The benchmark reported an 8.8× speedup. There was one problem. It was wrong. llama.cpp tracks more than the K/V tensor values. Cache cells also have position and sequence metadata used to construct the attention mask. Copying tensor values without restoring that bookkeeping produced an inert fragment. The model skipped prefix computation... ...but attention could not actually see the restored prefix. 14 of 24 cache hits reproduced, token for token, the output of a generation with no prefix at all. The “speedup” was dropped context. So I discarded it. Timing is not enough A broken cache can be fast. That's why EdgeSync now runs two correctness chec

Lately, I've been doing some deep personal reflection. I'm talking about myself, I hope no one misunderstands, on how pervasive the use of AI has become in my daily development workflow. Through a bit of self-analysis, I've discovered some interesting dynamics. Dependencies often arise from the desire to fill a void. But what kind of void does an experienced developer like me face? As a professional, I have the skills. Sure, AI helps me get things done faster, but the final product is always the translation of my vision; if I don't fully understand the solution, I discard it. I'm not looking for "magic," I'm looking for efficiency. Yet, I realize I've used AI to fill a specific void: the need for discussion. Software development is inherently solitary. The satisfaction of a successful "execution" after hours of discussions, refinements, and clashes over an architecture is an experience I miss today. The chat interface is always there, ready to respond. But there's a problem: it's a "yes-man." Even when I force it to be critical or provocative via the system's prompts, I know it's just reciting a script to please me. There's no conviction, no risk of error, none of the friction that arises when a colleague courageously defends their vision, perhaps one that conflicts with mine. We are part of a huge community, but debate often remains superficial. One might argue that posts and comments are enough, but anyone who has tried knows it doesn't work very well: a debate is truly alive only when there is no latency. In comments, the time between thinking, writing, and waiting for a response diminishes the energy of the exchange, turning it into a series of monologues rather than a dialogue. Why don't we try creating "virtual tables" where we can discuss projects, architectures, and technical choices with the natural rhythm of a conversation? Direct, real-time discussions, in person or remotely, where the exchange of ideas can spark sparks, without the filter (and delay) of

The incident that taught me to respect pinning looked like nothing. A service freshly migrated to virtual threads, a load test that plateaued at about 420 requests per second no matter how much traffic we threw at it, CPU sitting at 9%, zero errors, zero warnings, nothing in the logs. The machine had 8 cores, and the one downstream HTTP call in the hot path took about 19 ms. Do the arithmetic: 8 × (1000 / 19) ≈ 421. The service that was supposed to scale to millions of virtual threads was serving exactly one request per CPU core. Loom had quietly handed us back a bounded thread pool, and the code looked perfectly innocent. That failure mode has a name — pinning — and this is the field guide I wish I'd had that night: what it is, the two (and only two) things that cause it, what JDK 24 changed, and how to catch it before your throughput graph does. What pinning actually is A virtual thread doesn't own an OS thread. It runs on a small pool of platform threads called carrier threads — concretely, the workers of a dedicated ForkJoinPool living in a thread group named CarrierThreads , with default parallelism equal to Runtime.availableProcessors() . When a virtual thread blocks — on I/O, a lock, a queue — it normally unmounts : it saves its stack, steps off the carrier, and frees that carrier to run another virtual thread. That unmount is the entire trick that lets a handful of OS threads serve millions of virtual ones. Pinning is when the unmount can't happen. The virtual thread blocks but stays mounted, and its carrier sits there doing nothing useful for the whole duration. One pinned carrier is a rounding error. But the default carrier pool is only as big as your core count, so if a hot path pins routinely, you pin every carrier at once — and then no virtual thread anywhere makes progress. That's not a slowdown; it's scheduler starvation, and from the outside it looks a lot like a deadlock. You can raise the ceiling with -Djdk.virtualThreadScheduler.parallelism=N , bu

An interactive guide to the architecture behind modern language models. Instead of predicting the next word, this Transformer predicts the next move in a game of fading Tic-Tac-Toe—making every step of the model easy to visualize and understand. Play the game, inspect every matrix multiplication, and watch tokens flow through the network in real time. What's covered Tokenization and embeddings Learned positional encoding Self-attention (Q, K, V) Multi-head attention Causal masking and softmax Residual connections and layer normalization MLP (feed-forward network) Unembedding and sampling Model ablations (no positional encoding, no causal mask, no MLP, no residual stream) Includes interactive visualizations for every stage of the Transformer pipeline - from input tokens to the final prediction. https://sbondaryev.dev/articles/transformer

Now it's an arms race between OEMs locking down chips and tuners trying to crack them.

One common issue on WordPress websites with a large number of posts is that the Rank Math XML Sitemap can become slow to load. This happens because the sitemap is generated dynamically every time a visitor or search engine bot requests it. A simple solution is to use a static sitemap cache , allowing the web server to serve pre-generated XML files directly without executing PHP for every request. This significantly reduces server load and improves crawling performance. Benefits of Using a Static Sitemap Using a static sitemap cache provides several advantages: Faster sitemap loading times. Lower CPU and PHP worker usage. Improved crawling efficiency for Google and other search engines. Ideal for websites with thousands or even millions of URLs. Reduced server load when search engine bots frequently request sitemap files. 1. Setting RankMath Sitemap Cache The first step is to enable static sitemap generation using the Rank Math Sitemap Tweak plugin. The plugin automatically creates static copies of your XML sitemaps and stores them in the following directory: /wp-content/uploads/rank-math/ Instead of generating the sitemap dynamically through WordPress, your web server can serve these static files directly. 2. Configure Apache (.htaccess) If your website is running on Apache , add the following rules to your .htaccess file. # ========================== # XML cache # ========================== RewriteCond %{REQUEST_METHOD} GET RewriteCond %{QUERY_STRING} ^$ RewriteCond %{HTTP:Cookie} !wordpress_logged_in RewriteCond %{DOCUMENT_ROOT}/wp-content/uploads/rank-math/%{HTTP_HOST}%{REQUEST_URI} -f RewriteRule ^(.*)$ /wp-content/uploads/rank-math/%{HTTP_HOST}/$1 [L] These rules check whether a cached sitemap file exists. If it does, Apache serves the static file immediately without loading WordPress. 3. Configure Nginx If your server is using Nginx , add the following configuration inside your server block. # # Static cache # location / { try_files \ /wp-content/uploads/rank-

Adding cloud regions changes latency and cost in ways simple math can't capture. This article presents a framework from multiple launches: decompose your latency budget before committing to infrastructure, choose deployment patterns by consistency and traffic profile, and optimize before expanding. A phased approach cut latency 35% through routing alone, before a new region brought it under 60ms. By Uttara Asthana