标签：#feature

"The ISS radiators are expensive and heavy. We're focused on making them cheap and light."

Experts explain how they work, what they can do, and what's still unsettled.

Learning how to add evals to an LLM feature is the difference between shipping a demo and shipping a reliable product. When you embed an LLM into a real feature — a chatbot, a voice agent, a document summarizer — you’re not just calling a model. You’re betting your user’s experience on a non‑deterministic system that can silently break with every prompt tweak, model update, or edge case. That’s why we instrument every LLM feature we build with a purpose‑built eval suite. Here’s how we did it for an outbound AI calling agent and how you can do the same. Why Evals Are Not Optional LLMs are non‑deterministic: give them the same input twice, and you’ll get two different responses. That means unit tests that check for exact string matches are useless. As Pragmatic Engineer notes , you need evals to verify that the solution works well enough — because there’s no guarantee it will. When you’re building a feature that speaks to real customers, like the AI Calling Agent dashboard we built, a regression in tone or missed booking intent can cost revenue immediately. Evals turn that uncertainty into signal. How to Add Evals to an LLM Feature: A 4‑Step Workflow We’ll walk through the exact process we followed, from defining success to automating checks in CI, using the DeepEval framework as an example. You can swap in Evidently AI or build your own, but the pattern is the same. Step 1: Define Success for Your Feature Takeaway: Before you pick a metric, write down the one thing that makes the feature “done” — usually a business outcome, not a technical measure. For the AI Calling Agent, the core feature was an outbound call that books a meeting. The success criterion wasn’t “the LLM replied politely.” It was “the agent scheduled a meeting with the right time and date.” This is a reference‑based evaluation: you compare the output to a known ground truth. Evidently AI’s guide calls this pattern out as essential for regression testing and experimentation. From that criterion, we der

Now it's an arms race between OEMs locking down chips and tuners trying to crack them.

"The Starship Pez dispenser demonstrates very smart industrial design and scale."

Top robotics researchers and founders explain how robot autonomy is evolving.

Top robotics researchers and founders explain how robot autonomy is evolving.

This hybrid V8 has organic-looking 3D-printed components and shatters lap records.

It's difficult to pinpoint the moment in my life where America started to lose the plot.

Your comments on a dangerous rule putting politicals in charge of science can matter.

The storm developed quickly over west central Illinois on April 17th, first as a single high-intensity system called a supercell, and then later that evening transforming into a long squall line of thunderstorms. Tucked inside a wall of wind were several smaller, quick-forming tornadoes, one of which landed directly on Rivian’s electric vehicle factory on […]

A year in, National Design Studio delays plan to update government web standards.

NASA’s quiet supersonic flight tests could eventually go on a national tour.

It has 205 miles of bare-bones range.

Jos Benschop is climbing a ladder to get to the top of his newest machine. It’s a bit of a schlep. The contraption is the size of a double-decker bus—more than 150 tons of gleaming precision-milled aluminum covered in thousands of snaking tubes, colored cables, and pressurized tanks. From the ground, it looks like a…

"I don’t know of a bigger question we can answer as humans."

It’s cold, it’s very, very noisy, and—if I can be quite honest with you—I’m not feeling super relaxed. I’m currently around 300 meters, or 1,000 feet, beneath the North Sea, in a dark, dank cave. It smells weird. And I am increasingly aware of the pressure from millions of tons of seawater just above my…

Thinking about trying FtrIO? The new CLI makes it easy to start with just one feature flag If you've been curious about FtrIO (the .NET feature toggle library that replaces if (featureFlags.IsEnabled(...)) with a [Toggle] attribute woven directly into your compiled IL) but weren't sure where to start, the experimental release of FtrIO.onetwo just made that first step a lot smaller. You don't need to commit to a full migration. You don't need to rip out your existing flag library. You just need one method and 20 minutes. What FtrIO.onetwo does FtrIO.onetwo is a .NET CLI audit tool. Its default mode scans your source tree, finds every FtrIO toggle reference, and tells you exactly what's live right now: dotnet tool install --global FtrIO.onetwo --version 1.1.1-experimental ftrio.onetwo --source C: \P rojects \M yApp But in this experimental release it does two new things: ftrio.onetwo import : pulls your current flag state from LaunchDarkly, Flagsmith, flagd, environment variables, or an HTTP endpoint directly into appsettings.json . Your existing flag library keeps working unchanged. ftrio.onetwo migrate : scans your .cs files for LaunchDarkly or Flagsmith SDK call patterns using Roslyn, cross-references them against your live flag state, and generates a report showing exactly what each flag would look like in FtrIO and how to migrate it. The "try it on one flag" workflow The migrate report categorises every flag it finds: ✅ Ready to migrate : boolean flag, no targeting rules, straightforward [Toggle] replacement ⚠️ Needs review : targeting rules, number flags, needs a decision ❌ Cannot migrate : JSON flags, recommend moving to IConfiguration For every ready flag it shows the suggested refactor. Something like: new - checkout - flow → NewCheckoutFlow File : Services \ OrderService . cs : 42 Current code : if ( client . BoolVariation ( "new-checkout-flow" , user , false )) { ValidateCart (); ApplyDiscounts (); ProcessPayment (); } Suggested action : Extract the if bloc

The Counterintuitive Truth: Feature Flags Are Not Config Files Most engineers first encounter feature flags as a simple abstraction: a key-value lookup that returns true or false. That mental model works fine for a single service handling a few hundred requests per minute. It becomes actively dangerous at scale. A mature feature flag system isn't a config file with an API wrapper — it's a distributed control plane . The distinction matters architecturally. A control plane manages the real-time behavior of a running system across many nodes simultaneously, with its own consistency guarantees, failure semantics, and propagation latency. That's a fundamentally different design problem than reading a YAML file on startup. One constraint drives every downstream decision: user traffic must never block on a remote flag service call. If evaluation requires a synchronous RPC, you've coupled your request path to the availability and latency of an external system. Netflix's Archaius library enforces this by evaluating flags entirely in-process against a locally-cached configuration snapshot. A network round-trip per evaluation injects 10–50ms of tail latency at p99 — catastrophic when you're competing on streaming start times measured in hundreds of milliseconds. Google, Meta, and Netflix collectively evaluate flags against millions of requests per second with sub-millisecond overhead. That figure is only achievable through local evaluation backed by an async synchronization layer, not RPC. The other failure mode engineers underestimate is flag sprawl . Systems accumulate flags the way codebases accumulate dead functions — gradually, then all at once. I've seen services carrying thousands of flags where fewer than 10% were actively managed. The operational weight alone becomes a liability: which flags are safe to remove? Which ones are kill switches for production behavior that no one documented? Knight Capital's $440M loss in 45 minutes in 2012 remains the canonical cautionar

Researchers say these coral strongholds may help repopulate more degraded reefs.