标签：#ast

TL;DR Traditional SaaS streaming requires exporting sensitive data to a vendor cloud, creating security risks and egress costs. BYOC reverses this model by running the data plane inside the customer’s cloud while the vendor manages the control plane. This keeps data within the enterprise perimeter while still providing a managed platform. Condense builds on this model with AI-driven automation, unified monitoring, and marketplace deployment, enabling private, compliant, and cost-efficient real-time data streaming. The enterprise data landscape is currently defined by a conflict between real-time AI data streaming utility and the strict requirements of data sovereignty . For years, the standard SaaS model forced a compromise. To access premium analytics, companies had to export sensitive telemetry to a vendor cloud. This created massive cloud egress costs and introduced significant security vulnerabilities. Bring Your Own Cloud (BYOC) for data streaming platforms has emerged as the professional solution to this dilemma. It allows a business to keep data within its own perimeter while benefiting from a fully managed, high-performance ecosystem. The BYOC Architecture: Privacy by Design An experienced analyst views BYOC as a clean separation of concerns. The architecture splits the environment into two distinct layers to ensure raw data never leaves the authorized environment. SaaS Control Plane: This is the management layer hosted by the provider. It handles the brain of the operation. It manages orchestration, user access, and pipeline configuration without ever seeing the actual data packets. Private Data Plane: This is the muscle. The managed Kafka clusters , Kubernetes (K8s) nodes, and storage engines like ClickHouse live inside the customer Virtual Private Cloud (VPC) . By keeping the data plane inside the customer perimeter, telemetry collection remains private. This architecture is the most direct path to satisfying internal security audits and global regulatory

I didn’t set out to diagnose hyperscalers. I wasn’t doing a grand industry analysis. I wasn’t mapping global architecture. I wasn’t trying to understand cloud strategy. I was just trying to use a popular software provider — and everything kept breaking. Every time something failed, I followed the thread. And every thread led to the same architectural gap. Eventually I realised I hadn’t been analysing hyperscalers at all. I’d accidentally mapped the substrate failure across the entire industry. Once you see the pattern, you can’t unsee it. Across Microsoft, AWS, Google, and Meta, the same structural drift appears: meaning drift identity drift trust drift state drift execution drift provenance drift agentic drift Different companies. Different stacks. Different histories. Same substrate gap. And it’s not just me. The world is waking up to these problems too. Vendor lock in isn’t just a technical nuisance anymore — it’s becoming a public conversation. People are asking why their money keeps disappearing into the same handful of providers. Organisations are asking why their systems collapse the moment they try to leave. Governments are asking why critical infrastructure depends on architectures they cannot inspect, cannot govern, and cannot reproduce. What started as a personal frustration with a popular software provider turns out to be the same structural issue everyone else is now discovering. And sovereignty is entering the conversation — not as a political slogan, but as an architectural question. When national systems depend on fragmented substrates owned by a tiny cluster of vendors, sovereignty becomes a structural issue. The question isn’t “who controls the cloud?” It’s “who controls the substrate the cloud is built on?” Follow the thread far enough and you reach a scenario nobody wants to think about: what happens in a moment of global stress when a hyperscaler’s fragmented substrate becomes a single point of failure? Not a political crisis — a structural one.

The Cybersecurity and Infrastructure Security Agency (CISA) has issued a postmortem on a data leak in which a contractor published dozens of internal CISA credentials -- including AWS Govcloud keys -- in a public GitHub repository for almost six months before being notified by KrebsOnSecurity. Experts say the gaps identified in the agency's initial response provide important lessons that all security teams should absorb.

Today, I’m talking with Xinzhou Wu, who is the head of automotive at Nvidia. Nvidia is obviously in the news constantly because of the AI boom — it’s one of the most valuable companies in the world, because the AI industry can’t get enough of the company’s GPUs. But Nvidia is also a key supplier […]

Experts explain how they work, what they can do, and what's still unsettled.

In this podcast, Michael Stiefel spoke to Sarah Wells about the relationship of governance to software architecture. Governance enables teams to work effectively by establishing procedures that minimize system complexity, improve security, and reduce repetitive tasks. Targeted checklists help engineers by reducing the stress over these procedures. By Sarah Wells

The state of the smart home can be frustrating, because it is just so obvious how things ought to work. You should be able to control everything from everywhere. Your spaces should adapt to what you're doing and how you're feeling. Making your home smart shouldn't require renovating, and the smarts should be mostly invisible. […]

I built Luxury Hotel Offers , a fully static site with 3,400+ listings that generates 4,400 HTML pages at build time. No SSR, no database at runtime. Here are the four hardest scaling problems I hit. 1. Googlebot's 2 MB HTML Limit With 3,400 hotels on one listing page, the naive approach (render all cards in HTML) produced a 9 MB page. Googlebot truncates at 2 MB and ignores the rest. The fix: cap the initial HTML at 400 cards. The remaining 2,500+ cards are generated as a separate HTML fragment file at a predictable URL ( /data/cards/{slug}-remaining/ ). A "Load More" button injects 48 cards at a time from the fragment. The first search or filter interaction loads the entire fragment so all cards are available for client-side filtering. This keeps every page under 2 MB for crawlers while giving users access to everything. 2. Content-Aware Lastmod with Cascading A site with 4,400 pages can't update every lastmod on every build. Search engines treat that as spam, and IndexNow has rate implications. Instead, the build hashes each hotel's SEO-relevant fields and compares against a persisted store. Only pages with actual content changes get their lastmod bumped. The interesting part is cascading: when a hotel in Paris changes, the Paris city page, France country page, and Europe region page all get their dates updated too, since their content changed (they list that hotel). Changed URLs feed into IndexNow so only genuinely modified pages get pushed to search engines. 3. DOM Filtering Breaks on Mobile at Scale The site started with pure DOM filtering: every card has data-* attributes for region, country, brand, and perks. JavaScript reads attributes and toggles visibility. Zero network requests, instant results. Great on desktop. On a mid-range phone with 2,500+ cards in the DOM, filtering took 2-3 seconds per interaction. textContent traversal across 20-40 nodes per card means ~60,000 DOM visits per keystroke. Layout thrashing with 10,000+ nodes made every show/hide cyc

It's unclear how the planet avoided its star's bloated red giant stage.

The probe sent back the first pictures of the asteroid Kamo’oalewa. Next step: landing on the surface and collecting samples to send back to Earth.

Netflix has shows and movies. And video games. And live sports. And podcasts. And also, apparently, YouTube videos? For a company that used to seem like the next big thing in TV, it all feels a little frenetic, and maybe a tad desperate. For a company that sees sleep as its primary competitor, it might […]

Bryan Oliver discusses the frontier of AI infrastructure: chaos engineering for large-scale GPU clusters. He shares how engineering leaders can handle complex topologies, network protocols like RDMA, and NUMA misalignments. Discover seven practical fault-injection strategies to maximize multi-million dollar hardware efficiency and build robust observability loops. By Bryan Oliver

"I have been waiting a long time to finally get up there..."

Searching billions of documents for a phrase and getting ranked results in tens of milliseconds looks like magic. It is not. It comes down to two ideas working together: an index that maps words to documents instead of scanning documents for words, and a way to spread that index across machines so each holds only a slice. Understand both and full-text search stops being mysterious. The core problem A database scans rows. If you ask a plain database to find every document containing a word, it reads documents and checks them, which is linear in the amount of data. That is fine for exact key lookups and hopeless for free-text search across huge corpora. You need the opposite mapping. Instead of "given a document, what words does it have", you want "given a word, which documents have it". That inversion is the whole trick. The second problem is size. One machine cannot hold the index for billions of documents, and one machine cannot serve the query load. So the index has to be split across nodes, and a query has to find the right nodes and combine their answers. Key design decisions Build an inverted index. At index time, each document is broken into tokens by an analyzer that lowercases, splits on word boundaries, and often strips or stems words. For every token, the engine keeps a posting list: the set of document ids that contain it, often with positions for phrase matching. A query for a word becomes a direct lookup of its posting list, not a scan. A multi-word query intersects or unions posting lists, which is fast because the lists are sorted. Store the index in immutable segments. New documents go into small new segments rather than editing existing ones. Segments are immutable, which makes them cache-friendly and safe to read without locks. A background process merges small segments into larger ones over time. A delete is just a marker; the document is removed for real during a later merge. Split an index into shards. An index is divided into shards, each a sel

In this podcast Shane Hastie, Lead Editor for Culture & Methods spoke to Gabriela Moreira about making formal methods accessible through the Quint specification language, how AI is dramatically lowering the barrier to entry for formal specification and model-based testing, and why defining correct system behaviour remains essential human work in an AI-driven world. By Gabriela Moreira

TL;DR Retailers process thousands of inventory transactions every second across physical stores, eCommerce platforms, warehouses, suppliers, and fulfillment centers. Yet many inventory systems still rely on scheduled synchronization, causing stock levels to become outdated within minutes. The result is overselling, delayed replenishment, inaccurate inventory visibility, and avoidable stockouts. Apache Kafka enables real-time inventory management by treating every inventory movement as an event that is streamed the moment it occurs. Sales, returns, warehouse transfers, supplier deliveries, and IoT sensor updates are continuously processed to maintain a consistent inventory view across all retail systems. This event-driven approach helps retailers improve inventory accuracy, automate replenishment, detect stockouts before they occur, and respond to changing demand in near real time. In this guide, you'll learn how Apache Kafka powers real-time inventory management, explore a production-ready reference architecture, understand how inventory events are processed across retail systems, and discover implementation best practices for building scalable, resilient inventory streaming applications. Introduction Retail inventory management has evolved far beyond tracking products on warehouse shelves. Today's retailers operate across physical stores, eCommerce platforms, online marketplaces, distribution centers, and supplier networks, where inventory levels change continuously throughout the day. Every sale, return, warehouse transfer, supplier delivery, and inventory adjustment impacts product availability, making accurate inventory visibility essential for delivering a seamless customer experience. However, many retailers still rely on scheduled synchronization between Point-of-Sale (POS) systems, Warehouse Management Systems (WMS), Enterprise Resource Planning (ERP) platforms, and online storefronts. While these systems perform different functions, they all depend on accur

Today on Uncanny Valley, we unpack the political debacle unfolding in Maine surrounding the campaign of Democratic candidate Graham Platner.

The company is taking a modular approach to designing these chips, anticipating that their needs will change as AI evolves rapidly by the time the chips are in production.

Today on Decoder, I’m talking with Peter Kafka, who is chief correspondent at Business Insider and host of Channels, a podcast about the media industry. And it’s a big week for the media industry — Comcast just announced that it’s splitting itself up, into the Comcast broadband company and the NBCUniversal entertainment company. That’s after […]

In version 1.11, HashiCorp introduced Terraform Ephemeral resources and write-only attributes to allow for root configs that do not store secrets in the Terraform statefile. But many users ask about how they can adopt ephemerals. This blog attempts to lay out the ways secrets can be stored in state and how you should update your configurations to remove those secrets. Note: For a primer on ephemerals ( see this blog post ). Scenarios to consider: Data sources that fetch a static secret Resources that receive a secret Resources that generate a dynamic a secret Resources that fetch generated secrets to store in another 3rd party system Scenario 1: Data sources with static secrets Ephemeral resources can often be a drop-in replacement for data sources pulling static values: data "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } ephemeral "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } However, using these values has 1 specific difference. The attributes on a ephemeral resource are considered ephemeral and can only be used as ephemeral arguments. That means 2 places: Provider blocks Provider blocks are considered ephemeral, so ephemeral resources may populate arguments: provider "example" { password = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } Write-only arguments Write-only arguments are special arguments that require the ephemeral taint for values: resource "aws_db_instance" "example" { ... password_wo = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } If the resource you wish to pass a value to does not have an available ephemeral, open an issue with that provider. You can reference: this blog post this agent skill Scenario 2: Resources that receive a static secret Without duplicating to the section above, write-only arguments are a way to get secrets out of state. Above has guidance if the secret value comes from a data source, but what if its from a variable?