标签：#orm

ASP.NET Core Output Caching: How to Make Web APIs Faster in .NET When an API receives the same request repeatedly, performing the same database query and rebuilding the same response every time can waste valuable resources. For example, imagine this endpoint: GET /api/products If thousands of users request the same product catalog, your application might repeatedly: HTTP Request ↓ Controller ↓ Database Query ↓ Business Logic ↓ JSON Response For data that doesn't change frequently, this can create unnecessary database load. ASP.NET Core provides Output Caching to help solve this problem. Instead of executing the complete request pipeline every time, the application can temporarily store the generated response and reuse it for subsequent requests. In this tutorial, we'll look at how Output Caching works, how to configure it, how to invalidate cached responses, and when you should avoid using it. What Is Output Caching? Output caching stores the generated response from an endpoint. For example: First request ↓ GET /api/products ↓ Execute controller ↓ Query database ↓ Generate response ↓ Store response in cache Later: Second request ↓ GET /api/products ↓ Cached response ↓ Return immediately The database doesn't need to be queried again while the cached response is valid. Output Caching vs Response Caching These two concepts are often confused. Response Caching Response caching mainly relies on HTTP caching semantics and headers. Output Caching Output caching is controlled by ASP.NET Core and allows your application to decide which responses should be cached and for how long. Output caching provides more control over server-side response caching. 1. Add Output Caching Start by registering the output-cache services. var builder = WebApplication.CreateBuilder(args); builder.Services.AddControllers(); builder.Services.AddOutputCache(); var app = builder.Build(); app.UseOutputCache(); app.MapControllers(); app.Run(); The important pieces are: AddOutputCache() ↓ Configure cac

Performance testing is essential for ensuring your applications can handle expected user loads without bottlenecks or failures. Apache JMeter remains one of the most popular open-source tools for load, stress, and performance testing. Here is a quick guide to getting your JMeter environment set up and executing your first load test. 1. Prerequisites JMeter requires Java to execute. Ensure you have JDK 11 or higher installed on your system. Verify your Java installation: java -version 2. Download and Installation Download the latest binary zip/tgz file from the Official Apache JMeter Site. Extract the archive into your preferred local directory. Launch JMeter from the bin directory: Windows: Double-click jmeter.bat macOS/Linux: Open terminal and run ./jmeter.sh 3. Install the Plugins Manager The Plugins Manager simplifies adding listeners, graph generators, and custom samplers. Download jmeter-plugins-manager.jar from JMeter Plugins. Move the file into your JMeter lib/ext directory. Restart JMeter. Access the Plugins Manager under Options > Plugins Manager. 4. Building Your First Test Plan Set up a basic HTTP test using the GUI interface: Thread Group: Right-click Test Plan > Add > Threads (Users) > Thread Group. Configure your target virtual users, ramp-up time, and loop count. HTTP Request Defaults: Right-click Thread Group > Add > Config Element > HTTP Request Defaults. Set your target server domain/IP and port. HTTP Sampler: Right-click Thread Group > Add > Sampler > HTTP Request. Define the API path and request method. Listeners: Right-click Thread Group > Add > Listener > View Results Tree or Summary Report (use these GUI listeners primarily for test script validation). 5. Running Tests in Non-GUI Mode Never run actual heavy load tests through the JMeter GUI as it consumes significant local system resources. Use CLI mode for accuracy: jmeter -n -t /path/to/testplan.jmx -l /path/to/results.jtl -e -o /path/to/html-report-folder -n: Non-GUI execution -t: Path to y

I spent a while building browser-based hardware diagnostics and came away with a much clearer sense of where the web platform is genuinely capable and where it quietly lies to you. Notes below, with live demos for each API so you can poke at them yourself. Refresh rate: requestAnimationFrame is the only signal you get There's no screen.refreshRate . The only approach is timing requestAnimationFrame callbacks and inferring the rate from the median frame delta: const deltas = []; let last = performance . now (); function tick ( now ) { deltas . push ( now - last ); last = now ; if ( deltas . length < 180 ) requestAnimationFrame ( tick ); else { const sorted = deltas . slice (). sort (( a , b ) => a - b ); console . log ( Math . round ( 1000 / sorted [ sorted . length >> 1 ])); } } requestAnimationFrame ( tick ); Two gotchas that cost me time. Use the median , not the mean — a single dropped frame wrecks an average. And browsers throttle rAF in background tabs, so the measurement is meaningless unless the tab is visible; gate it on document.visibilityState . ( live version ) Screen dimensions: four different answers, all "correct" screen.width , window.innerWidth , window.devicePixelRatio and screen.availWidth measure genuinely different things, and the one people usually want — actual native panel resolution — is screen.width * devicePixelRatio . Except that's still CSS-pixel derived, so on a scaled display it can disagree with what the panel physically is. The browser simply does not expose true hardware resolution. ( demo ) Keyboard: event.code vs event.key , and the keys you never receive event.key is layout-dependent, event.code is physical position — for a hardware tester you want code . The real limitation is that some keys never reach JS at all: PrintScreen often doesn't fire keydown , Meta combinations get swallowed by the OS, and Fn isn't a browser-visible key on most laptops. N-key rollover testing works surprisingly well though, since you just track the siz

Watch a developer start an agent session on real enterprise work and you will see a ritual. Before the first useful prompt, they gather. They paste the deployment standard, link the runbook, and explain what the criticality tiers mean. Then they correct the agent's first confident guess about a naming convention the team retired two years ago. Tomorrow they will do it all again, because the agent will not remember. We have quietly decided that this gathering is the developer's job. Every guide to working with AI repeats some version of the same advice: give the model good context. So developers hunt for it, one session at a time, across systems that were never designed to answer an agent's questions. I think that framing is backwards, and I think fixing it is platform work. In Your Platform Has a New User: The Agent , I argued that internal platforms now serve two personas: the developer and the developer's agent. Near the end, I wrote that context is becoming part of the platform. I called it one of the most important developer experience problems of the next few years. That idea got four paragraphs. It deserves an essay, so here is the longer version. The gathering is the tax Agents can remember more than they used to. What they cannot reliably accumulate on their own is organizational truth. A new engineer pays the onboarding cost once, then amortizes it over years of context, hallway conversations, and scar tissue. An agent may retain instructions, memory, or project state. None of those automatically tell it which standard is authoritative, which exception still applies, or which decision was reversed six months ago. Whatever it needs to know about your organization still has to come from somewhere. Now multiply that across hundreds of engineers. People rediscover the same standards, fork the same repo, re-paste the same runbooks, and retype the same corrections, day after day. Quality varies too. Your strongest engineers assemble excellent context and get exce

If you're running Magento 2 with MSI (Multi-Source Inventory) enabled — and since Magento 2.4 it's the default — you have a silent performance killer lurking in your database. The inventory_reservation table grows without bound, and every single cart operation hits it. This post walks through why this table becomes a bottleneck, how to measure the impact, and concrete steps to fix it. How Inventory Reservations Work When a customer adds a product to their cart, Magento doesn't immediately decrement stock. Instead, it creates a reservation — a record in inventory_reservation that says "this quantity is tentatively reserved for this order." The actual stock deduction happens later, when the order is placed and the shipment is processed. The flow looks like this: Add to cart → placeReservation writes a negative reservation record Place order → reservation is linked to the order Ship order → inventory_source_item is decremented, reservation should be compensated Compensation reservation → a positive record that cancels out the original negative one In theory, reservations are transient. They exist to bridge the gap between cart and shipment. In practice, they accumulate forever. The Problem: Unbounded Growth Here's what happens in production: Orders that are canceled leave orphaned negative reservations Orders that fail during checkout leave reservations that are never compensated Partial shipments create partial compensation records Quote conversions that error out mid-process leave dangling reservations Re-indexing, re-stocking, and admin edits can create duplicate records After 6–12 months of moderate traffic, the inventory_reservation table routinely hits several million rows . I've seen tables with 10M+ rows on stores doing 200 orders/day. SELECT COUNT ( * ) FROM inventory_reservation ; -- 4,872,341 rows on a store running 8 months SELECT COUNT ( * ) FROM inventory_reservation WHERE created_at < DATE_SUB ( NOW (), INTERVAL 30 DAY ); -- 4,710,882 — 96.7% of rows are

For the past few months, I've been working on a project that translates online meetings in real time. At first, I thought the problem would mostly be about choosing the right speech recognition model and finding a good translation API. It turned out those were the easy parts. The real challenge was making everything feel instant. People don't wait for subtitles. If the translation appears two or three seconds after someone finishes speaking, the conversation has already moved on. Technically it works, but from a user's perspective, it feels broken. That completely changed how I approached the project. Instead of optimizing only for accuracy, I had to optimize the entire pipeline for latency: Capturing audio continuously Streaming audio to ASR Performing speech recognition incrementally Translating partial sentences Updating subtitles without flickering Handling corrections when the speech recognizer revised previous words Every stage might only take a few hundred milliseconds, but together they determine whether the experience feels "real-time." Another lesson surprised me even more. Translation quality isn't just about picking a better LLM. Spoken language is messy. People interrupt each other, change direction halfway through a sentence, use filler words, and rarely speak in complete grammatical sentences. A model that performs well on benchmarks can still struggle in a live conversation if the input arrives one fragment at a time. That forced me to rethink prompts, buffering strategies, and when to display or revise translated text. Building this project also gave me a new appreciation for streaming systems in general. Real-time applications are fundamentally different from batch processing. Instead of asking, "How accurate is the result?", you're constantly balancing three competing goals: Latency Stability Accuracy Improving one often makes another worse. I'm still learning every day, and there are plenty of problems left to solve. Over the next few weeks, I'd

If you’ve managed web servers or applications for any length of time, you’ve probably seen this happen: a new feature or campaign goes live, traffic suddenly spikes, and Website Load Testing becomes critical when your website starts returning 503 errors at exactly the moment you need it to perform. What happens next is usually a scramble, SSH into a server you haven’t checked in months, inspect running processes, restart services, and make infrastructure changes based on guesswork. Eventually, the traffic settles, the site recovers, and the immediate crisis is over. But that kind of incident is often preventable. Load testing helps you find your website’s limits before your users do. In this guide, we will cover what load testing is, why it matters at every scale, how to run your first test using loader.io (the most accessible free tool available), what your results actually mean, how to find and fix bottlenecks, and how to make load testing a normal part of how you ship software. TL;DR Load testing answers one critical question: how many concurrent users can your server handle before it falls over? Without it, you’re guessing about capacity, and guessing wrong right when it matters most loader.io is the simplest free tool to get started: no install, browser-based, generous free tier Your three essential numbers: concurrent user target, response time threshold, and peak traffic window Run load tests before every major deployment, not after your site goes down What Load Testing Actually Is Let me clear up some confusion first, because “load testing” gets thrown around interchangeably with a few related terms that mean different things. Load testing is specifically about simulating concurrent users hitting your site and measuring how your server behaves under a expected load. You’re asking: “When 500 people are on this site at the same time, what happens?” Stress testing pushes beyond that, you keep adding users until something breaks, then you figure out exactly wher

We all know the Big-O complexity of basic data structures. Arrays are O(n) for search. Hash maps are O(1). Linked lists are... well, complicated. But when I set out to build hashbrowns — a C++17 benchmarking suite comparing arrays, linked lists, and hash maps — I discovered that theory and practice are very different beasts. Here's what I learned building this project from scratch, and why you should probably benchmark before you optimize. 🎯 The Goal Was Simple (Ha!) I wanted a clean, educational project that would: Implement dynamic arrays, linked lists, and hash maps from scratch Benchmark insert, search, and remove operations Find the "crossover points" where one structure beats another Export everything to CSV for analysis Sounds straightforward, right? Four months later, I had written a custom memory tracker, implemented multiple hash map strategies, added statistical bootstrapping for confidence intervals, and learned more about CPU caches than I ever wanted to know. 📚 Lesson 1: Polymorphism Has a Price (But It's Worth It) My first architectural decision was creating a common DataStructure interface: class DataStructure { public: virtual void insert ( int key , const std :: string & value ) = 0 ; virtual bool search ( int key , std :: string & value ) const = 0 ; virtual bool remove ( int key ) = 0 ; virtual size_t memory_usage () const = 0 ; virtual std :: string type_name () const = 0 ; // ... }; This made benchmarking elegant — I could write generic code that tested any data structure: for ( auto & structure : structures ) { timer . start (); structure -> insert ( key , value ); timer . stop (); } But virtual function calls have overhead. In tight loops, that vtable lookup adds up. I spent a whole weekend convinced my hash map was slower than expected... until I realized I was measuring the cost of polymorphism, not the data structure itself. The fix? I kept the clean interface for the benchmarking harness but used templates internally where performance-cri

People with schizophrenia face a perfect storm of dangers on a hotter planet.

When a user closes a tab, submits a form, or clicks an external link, you often need to send one last...

Small business websites rarely fail because of one catastrophic bug. They fail from an accumulation of small, fixable problems — a slow hero image here, an unlabeled form field there, a broken tab order that quietly locks out keyboard users. None of it looks dramatic in a screenshot. All of it adds up to lost conversions. Working across client rebuilds and audits at Alynox, the same handful of issues show up repeatedly, regardless of industry. Here are ten of the most common, with the practical, mostly low-effort fixes that address them. Unoptimized Images Dragging Down Load Time The single most common performance killer on small business sites is still oversized images — a 4MB PNG hero banner exported straight from a design tool, served at full resolution to a phone screen 400px wide. Fix: html src="hero-800.webp" srcset="hero-400.webp 400w, hero-800.webp 800w, hero-1600.webp 1600w" sizes="(max-width: 600px) 400px, (max-width: 1200px) 800px, 1600px" alt="Interior of the workshop showing custom furniture in progress" loading="lazy" width="1600" height="900" /> Convert to WebP or AVIF, generate a handful of responsive sizes, lazy-load anything below the fold, and always set explicit width/height to reserve space and avoid layout shift. No Real Mobile-First Design A lot of "responsive" small business sites are really desktop layouts that get squeezed with media queries until they technically fit a phone screen. Buttons end up too small to tap accurately, text wraps awkwardly, and nav menus overlap content. Fix: Design and build mobile-first — base styles for small screens, then progressively enhance with min-width media queries for larger viewports: css .card { padding: 1rem; } @media (min-width: 768px) { .card { padding: 2rem; } } Tap targets should be at least 44×44px (per WCAG and Apple/Google HIG guidance), with enough spacing between interactive elements to prevent mis-taps on smaller screens. Accessibility Treated as an Afterthought Missing alt text, low-contras

Form Energy has landed Google and Crusoe as customers. Now, it has raised $750 million to expand manufacturing to deliver its massive, 100-hour batteries.

Hello I'm Marlene and I invite you to follow my journey developing ProForma.net. But since this is my first post about ProForma, I will give you an overview of what I'm trying to achieve. What ProForma.net is planned to be The main Goal is to develop an application shell for schema based Applications. You will have mainly to different types of UI schemes, first for the Window Layout, there you will tell which elements are contained in the different application sections, like what Buttons or Menus you will have in the window title bar, or what sidebar tabs you will provide for the Ribbons, what the content area is filled with (spoiler I'm going to use flexlayout-react https://github.com/caplin/FlexLayout ). As host application I will write a C# application using the WebView2 abstraction library Photino ( https://www.tryphotino.io/ ). What you can expect In this dev diary series I'll show what I was working on, I'll show you some code and will explain why did to choose the way I did it, or will share some thoughts about the project or the architecture. I also will show you how to write plugins for ProForma, because I plan to handle everything as a plugin so you can change the most aspects of the app. The journey begins: overcome the guard Ok, most of you will know it... parameter checking on top of a method... nearly endless 'if throw' constructs... they are ugly... if (! Directory . Exists ( physicalPath )) throw new DirectoryNotFoundException ( $"Could not find the given path ' { physicalPath } '." ); if ( _directories . ContainsKey ( urlPrefix )) throw new Exception ( $"Key ' { urlPrefix } ' already exists." ); if ( _directories . ContainsValue ( physicalPath )) throw new Exception ( $"Physical Path ' { physicalPath } ' already exists." ); I mean who wants to read that? I don't. So I wanted guards, and I've could used some 3rd Party library, but instead I came up with my own solution for the Guards, since I don't always want to throw the exception on a failed asser

If you have ever wanted to ship an AI-powered application without managing GPUs, model servers, or scaling infrastructure yourself, this guide is for you. Managed inference simply means letting a cloud provider run the AI model for you: you send a request, the platform handles the compute, and you get a response back. On Google Cloud, the cleanest way to do this today is to pair the Gemini Enterprise Agent Platform (formerly Vertex AI) with Google Cloud Run , dividing responsibilities between the two services. The Agent Platform serves as the orchestration and intelligence engine, while Cloud Run hosts your custom application logic, front-end UIs, or Model Context Protocol (MCP) servers. By the end of this article, you will be able to: Explain the hybrid architecture and why each layer exists Define an AI agent in code using the Agent Development Kit (ADK) Deploy your app layer to Cloud Run with a single command Choose between online and batch inference for your workload Secure and monitor the whole setup in production New to the underlying concept? Start with Google Cloud's primer: What is AI inference? Prerequisites To follow along hands-on, you will need: A Google Cloud project with billing enabled The gcloud CLI installed and authenticated Python 3.10+ and the ADK installed ( pip install google-adk ) You can also read this purely as an architecture walkthrough; every step is explained, not just shown. 1. The Architectural Blueprint This pattern splits your system into independent, auto-scaling tiers: [ Client / Web UI ] ──> [ Cloud Run Service ] (App Logic / Tool Front End) │ ▼ [ Gemini Enterprise Agent Platform — Agent Runtime ] (Orchestration, Intent Analysis, Memory) │ ▼ [ Managed Inference / Model Garden ] (Gemini 3.x Pro / Flash models) Why split it this way? Each tier scales independently and fails independently. Your web front end can handle a traffic spike without touching the model layer, and you can swap models without redeploying your application code

If you work with library data, you work with MARC21 — the length-prefixed binary record format catalogues have run on since the 1960s, complete with a directory of field offsets, subfield delimiters, and a pre-Unicode character encoding called MARC-8 that needs a lookup table with thousands of entries to decode. In Python that problem is solved: pymarc is mature, complete, and pleasant to use. In Go it wasn't. gomarc is a port of pymarc to Go. It covers the binary MARC21 transmission format, MARC-8 to Unicode conversion, MARCXML, and MARC-in-JSON — and on real catalogue exports it runs 4x to 11x faster than the library it was ported from. go get github.com/beyto1974/gomarc@v0.1.0 It reads like pymarc If you know pymarc, you already know this API. Iterate records, pull the fields you want: reader := marc . NewReader ( f ) for { record , err := reader . Next () if errors . Is ( err , io . EOF ) { break } if err != nil { log . Println ( err ) // permissive: bad records are skipped, not fatal continue } title , _ := record . Title () fmt . Println ( title ) } Title , Author , ISBN , ISSN , Subjects , Publisher , PubYear and more are there as methods. For anything else, go at the tag and subfield directly: value , ok := record . Get ( "245" ) . Subfield ( "a" ) for _ , f := range record . GetFields ( "650" ) { fmt . Println ( f ) } Build records, modify them, write them back: record . Get ( "245" ) . SetSubfield ( "a" , "The Zombie Programmer : " ) writer := marc . NewWriter ( out ) writer . Write ( record ) And convert to the formats the rest of your stack can actually read — both use UTF-8 throughout instead of MARC-8, so standard tooling works: s , err := record . AsJSON () // MARC-in-JSON records , err := marc . ParseXML ( r ) // MARCXML Large MARCXML files stream one record at a time via marc.NewXMLReader rather than loading into memory. The numbers Two real catalogue exports — 138,076 records, 166 MB. AMD Ryzen 5 3600, Go 1.25.12, CPython 3.13.5, gomarc v0.1.0, pym

The deal starts with the arrival of the GEN4 car and 2026/2027 season this December.

I spent a few days writing a satellite ground station daemon in Crystal, with an empty dependency list and a hard rule against third-party code. It works, it ships as one file, and it sits at 1.9 MB of memory at rest. This is what the language was like to use, and what it cost. The project is kozai : it reads orbital elements, propagates them with SGP4/SDP4, predicts passes over a ground station, serves a JSON API and an offline web interface, and drives a rotator and a radio through hamlib. About 9,000 lines of source and 6,400 lines of specs, on Crystal 1.21.0. None of that matters here except as the load under which the language was tested — this is a report on the tool, not on the satellites. What the language actually delivers The headline claim of a compiled language with a garbage collector is that you get Ruby's ergonomics and a binary at the end. In 2026 that claim holds, and the numbers are the part worth quoting: Docker image, FROM scratch 7.41 MB Static binary, musl, arm64 6.9 MB Dynamic binary, release 1.9 MB Memory at rest, 2 satellites 1.9 MB Memory at rest, 97 satellites 4.3 MB Memory after a day of serving, 97 satellites 19.3 MB, flat Build steps before crystal build none Runtime files outside the binary none The last two rows are the ones that changed how the project was built. There is no Node in this repository, no bundler, no asset pipeline, and no postinstall . The web interface — HTML, CSS, JavaScript, and a 66 KB SVG of the world's coastlines — is read at compile time by {{ read_file(...) }} and lives inside the executable ( src/assets.cr ). Deploying is scp . The standard library covered the whole surface of a network daemon with six imports: http/server , http/client , json , log , socket , option_parser . That list is not an aspiration; CI fails if a seventh appears. The type system earned its keep in the numerical core. Predicting a week of passes for a hundred satellites is on the order of ten million propagator calls, and the hot loop a

When I first connected my GitLab CI/CD pipelines to AWS, I used the simplest solution: an IAM user with an Access Key and Secret Access Key stored as GitLab CI/CD variables. It worked. But there was one problem: those credentials were permanent. They had to be stored, protected and eventually rotated. If they were accidentally exposed in logs or compromised, they could remain valid until manually revoked. I wanted a cleaner solution. So I replaced permanent AWS credentials with OIDC federation between GitLab and AWS . The result is simple: GitLab pipelines can access AWS without storing any permanent AWS credentials. In this post, I'll explain how I implemented it, how the authentication flow works, and one important issue I faced when using it with EKS and Terraform. The architecture The authentication flow looks like this: ┌──────────────┐ │ GitLab CI │ └──────┬───────┘ │ │ OIDC token ▼ ┌──────────────┐ │ AWS STS │ └──────┬───────┘ │ │ Temporary credentials ▼ ┌──────────────┐ │ IAM Role │ └──────┬───────┘ │ ├──────────► Terraform │ ├──────────► ECR │ └──────────► EKS Instead of GitLab storing an AWS Access Key, it proves its identity to AWS using a short-lived OIDC token. AWS verifies the token and returns temporary credentials. How does OIDC authentication work? The process can be summarized in five steps: GitLab creates an OIDC token for the CI/CD job. The pipeline sends this token to AWS. AWS verifies that the token really comes from GitLab. AWS checks that the project is allowed to assume the requested IAM role. AWS STS returns temporary credentials. These credentials expire automatically. So there is nothing permanent to store or rotate inside GitLab. Step 1 — Register GitLab as an OIDC provider AWS first needs to trust GitLab as an identity provider. I configured the OIDC provider using Terraform: data "tls_certificate" "gitlab" { url = "${var.gitlab_url}/.well-known/openid-configuration" } resource "aws_iam_openid_connect_provider" "gitlab" { url = var . gi

Evolusi Pemrosesan Data: Dari Batch ke Real-Time Transformasi infrastruktur data mendorong transisi dari pemrosesan batch statis ke arsitektur streaming. Integrasi LLM kini menjadi komponen inti sistem data terdistribusi, di mana kecepatan pemrosesan informasi menentukan relevansi dan akurasi output AI secara instan. Tantangan Latensi pada Integrasi LLM Langsung Menyematkan LLM dalam pipeline real-time memicu tantangan sinkronisasi state dan overhead komunikasi antar-node. Bottleneck utama biasanya terjadi pada transfer KV cache dan keterbatasan bandwidth memori, yang menghambat performa inferensi pada skala terdistribusi. Strategi Optimasi: TensorRT-LLM dan Arsitektur Asinkron Optimasi melalui TensorRT-LLM krusial untuk menekan latensi token-to-token melalui optimalisasi kernel CUDA dan manajemen memori yang lebih efisien. Selain itu, arsitektur asinkron seperti Pathways memungkinkan eksekusi grafik dataflow dinamis, meminimalkan idle time pada akselerator GPU/TPU. Kapan Memilih Pendekatan Batch Tradisional? Pendekatan batch tetap superior untuk tugas non-sensitif waktu. Efisiensi biaya (cost-efficiency) dan throughput tinggi menjadikan metode ini pilihan utama untuk pemrosesan dataset masif, seperti pelatihan ulang model (retraining) atau analisis historis skala besar. Masa Depan: Arsitektur Hibrida untuk Skala Besar Sistem masa depan akan mengadopsi model hibrida: inferensi kritis latensi dijalankan di edge atau buffer lokal, sementara pemrosesan berat tetap berada pada jalur batch. Strategi ini memaksimalkan data-locality dan mengoptimalkan alokasi sumber daya komputasi.

Meryem Arik discusses strategies for designing low-cost LLM inference architectures for high-volume, non-real-time workloads. She explains how software architects and engineering leaders can achieve order-of-magnitude cost reductions by making critical trade-offs across hardware, inference runtimes, speculative decoding, and smart queue reordering. By Meryem Arik