标签：#t

Introduction Every post in this series has quietly touched a piece of the same problem. Building Agentic Workflows in Java said toolUse.input() is untrusted and must be validated before it reaches your code. Building Reliable LLM Applications in Java said the model will confidently invent facts, so ground it and get typed output instead of parsing prose. Neither post named the thing underneath both statements: anything that crosses from outside your code into the model, or from the model back into your code, is untrusted input — a request body from the network, not a trusted internal value. This post names that boundary directly and gathers the defenses in one place — prompt injection (direct and indirect), input validation, output validation, and PII redaction — with the SAFE pattern shown beside every unsafe one it replaces, since this is the security-forward capstone of the series. The Trust Boundary: Three Kinds of Untrusted Input An LLM application has three places where untrusted text enters: User input — anything a person types, uploads, or submits through an API. Retrieved content — Making RAG Accurate in Java built a pipeline that ranks and returns chunks from a document store; those chunks were written by whoever authored the source document, not by you, and a malicious or compromised document can carry text aimed at the model reading it, not at a human reader. Model output — untrusted the moment it's about to be used rather than displayed : passed to a tool, interpolated into a query, or fed into another LLM call as context. A model that just read attacker-controlled retrieved text can be manipulated into producing attacker-controlled output. The single rule under all three: text is data until your code has explicitly decided it's safe to use for anything more than display. Nothing below is executed against a live API — every snippet is illustrative, and none of it uses a real key or a real record. Direct Prompt Injection: Defending the System Prompt A di

Introduction Every post in this series has quietly touched a piece of the same problem. Building Agentic Workflows in Python said a tool's input is untrusted and must be validated before it reaches your code. Building Reliable LLM Applications in Python said the model will confidently invent facts, so ground it and get typed output instead of parsing prose. Neither post named the thing underneath both statements: anything that crosses from outside your code into the model, or from the model back into your code, is untrusted input — a request body from the network, not a trusted internal value. This post names that boundary directly and gathers the defenses in one place — prompt injection (direct and indirect), input validation, output validation, and PII redaction — with the SAFE pattern shown beside every unsafe one it replaces, since this is the security-forward capstone of the series. The Trust Boundary: Three Kinds of Untrusted Input An LLM application has three places where untrusted text enters: User input — anything a person types, uploads, or submits through an API. Retrieved content — Making RAG Accurate in Python built a pipeline that ranks and returns chunks from a document store; those chunks were written by whoever authored the source document, not by you, and a malicious or compromised document can carry text aimed at the model reading it, not at a human reader. Model output — untrusted the moment it's about to be used rather than displayed : passed to a tool, interpolated into a query, or fed into another LLM call as context. A model that just read attacker-controlled retrieved text can be manipulated into producing attacker-controlled output. The single rule under all three: text is data until your code has explicitly decided it's safe to use for anything more than display. Nothing below is executed against a live API — every snippet is illustrative, and none of it uses a real key or a real record. Direct Prompt Injection: Defending the System Prompt

Introduction https://pg-blogs.netlify.app/posts/10-building-reliable-llm-apps-in-python/ closed with a section on picking the right model per task and caching a shared prefix. That was the entry point into a bigger discipline: LLM spend is an engineering variable, not a fixed bill — one you can measure and reduce with the same rigor you'd apply to query latency or memory footprint. This post goes deeper on four levers: how input/output pricing actually works and why the prefix is usually where the money goes, the exact cache_control shape and how to prove a cache hit instead of assuming one, the Batches API for work that isn't latency-sensitive, and model routing — a cheap model triaging requests and escalating only the hard ones. The throughline is honest: measure before you optimize. Every lever here has its own cost; misapplied, it makes things slower or pricier, not cheaper. Token Economics: Why the Prefix Is the Bill LLM providers price input and output tokens separately, and output always costs more — generation is autoregressive (each token depends on every one before it), while input can be processed in parallel. Representative pricing from the current model catalog: Model Input Output Claude Opus 4.8 $5.00 / MTok $25.00 / MTok Claude Sonnet 5 $3.00 / MTok $15.00 / MTok Claude Haiku 4.5 $1.00 / MTok $5.00 / MTok Two things follow: A long system prompt, tool list, or RAG context is billed as input on every request , not written once. Send a 20K-token system prompt on 10,000 requests and that's 200M input tokens — at Opus 4.8 rates, $1,000 before the model has generated a single output token. The shared prefix , not the user's actual question, is usually the dominant cost. Verbose output costs twice — once directly (more output tokens billed at the higher rate), and again because the next turn's history carries that verbosity forward as input. Asking for concise output and setting a sane max_tokens is a cost control, not just a style choice. This is why the tw

Introduction Building Reliable LLM Applications in Python put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Python put the same discipline in pytest terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a pytest assertion either passes or fails against a fixed expected value; an LLM's output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assert . That harness has three parts: a golden dataset of representative cases with known-good expected behavior, scoring that turns each case into a pass/fail or a number, and regression testing that runs the harness on every change and fails the build when the score drops. Making RAG Accurate in Python already gave you half of this story — recall@k, precision@k, MRR, nDCG measure whether retrieval found the right chunks. This post measures the other half: whether the generated answer built from those chunks is actually good, which is a genuinely different question a retrieval metric can't answer on its own. Everything below is illustrative, non-executed Python, grounded in the same Anthropic SDK shapes as posts 10/11. The Golden Dataset: Curating Cases, Not Just Inputs A golden dataset is a small, hand-curated set of (input, expected behavior) pairs that represents the ways your application is actually used — not a random sample, and not just the cases that already work. Each case needs enough structure to be scored automatically later: from dataclasses import dataclass , field @dataclass class EvalCase : id : str category : str # "extraction", "qa", "summarization", ... input : str # the prompt/question sent to the system under test expected_exact : str | None = None # non-None only for cases scorable by exact match must_contain : list [ str ] = field ( default_f

Introduction Building Reliable LLM Applications in Java put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Java put the same discipline in JUnit terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a JUnit test either passes or fails against a fixed expected value; an LLM's output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assertEquals . That harness has three parts: a golden dataset of representative cases with known-good expected behavior, scoring that turns each case into a pass/fail or a number, and regression testing that runs the harness on every change and fails the build when the score drops. Making RAG Accurate in Java already gave you half of this story — recall@k, precision@k, MRR, nDCG measure whether retrieval found the right chunks. This post measures the other half: whether the generated answer built from those chunks is actually good, which is a genuinely different question a retrieval metric can't answer on its own. Everything below is illustrative, non-executed Java, grounded in the same Anthropic Java SDK shapes as posts 10/11. The Golden Dataset: Curating Cases, Not Just Inputs A golden dataset is a small, hand-curated set of (input, expected behavior) pairs that represents the ways your application is actually used — not a random sample, and not just the cases that already work. Each case needs enough structure to be scored automatically later: public record EvalCase ( String id , String category , // "extraction", "qa", "summarization", ... String input , // the prompt/question sent to the system under test String expectedExact , // non-null only for cases scorable by exact/programmatic match List < String > mustContain , // key facts a correct answer must mention (programma

Introduction Every agent needs tools, and every tool needs a way to reach the model. Building Agentic Workflows in Python built that connection by hand — a hand-written JSON schema, a loop that dispatches on block.name . LLM Frameworks vs. the Raw SDK in Python showed LangChain's @tool turning a plain function into that same schema via bind_tools . Both are still bespoke : the tool lives inside one process, wired to one agent, in one language. The Model Context Protocol (MCP) solves a different problem: it standardizes the wire format between an AI application and a tool server, so the server doesn't have to be rewritten per agent, per framework, or per language. This post covers what that buys you, builds a minimal MCP server and a client that consumes it — both on the official Python SDK — and gives an honest answer to when reaching for a protocol is worth it over a direct tool call. The Problem MCP Solves Without a shared protocol, every pairing of agent framework and tool needs its own glue code: a LangChain @tool wrapper, a hand-rolled schema for the raw SDK, a different wrapper again for whatever framework a teammate picks next — an integration per framework, per tool. That's an M×N problem. MCP flattens it to M+N. A server exposes tools, resources, and prompts once, over a standard JSON-RPC protocol. Any host application — Claude Code, Claude Desktop, VS Code, or your own agent — creates an MCP client that speaks that same protocol, regardless of which framework built the host. Write the server once; every MCP-aware host can use it without new integration code. The protocol itself is intentionally boring: JSON-RPC 2.0 messages for lifecycle negotiation, tool discovery, and tool execution. Discovery ( tools/list ) and execution ( tools/call ) are the two calls that matter for this post: // tools/list response (abbreviated) { "jsonrpc" : "2.0" , "id" : 2 , "result" : { "tools" : [ { "name" : "get_account_balance" , "description" : "Look up the balance for an ac

They were annoying to use and bad 3D movies didn't help.

There was a period in the early aughts when Definitive Jux (nee: Def Jux) seemed like it was going to be the future of hip hop. While the label featured plenty of experimental, boundary-pushing, and politically minded acts, Lif stood out as the most "conscious rapper" in the traditional sense. It was clear though, that […]

Hey devs 👋 OrinIDE is an AI-powered code editor that runs entirely in your browser — no...

These may be the last days of Amazon’s Mechanical Turk.

Anything released after 2028 will still be digital-only.

A tour of the stack behind mattstratton.com and speaking.mattstratton.com: a monorepo holding two Astro sites and a dev.to sync tool, twenty years of blog posts, and the pipeline that crossposts posts like this one.

Rockstar's long-awaited Grand Theft Auto VI is launching November 19th, 2026 for PlayStation 5 and Xbox Series S/X consoles. The game will be available digitally at launch, with physical cases containing codes, not discs, so your options for preordering are straightforward. The preloading phase for the game begins on November 12th, which should give you […]

I’m building Euro Toolhub , a German-first index of European software, SaaS, cloud and AI alternatives. The idea is simple: many companies, agencies, developers and privacy-conscious users want alternatives to common tools when they care about things like data residency, open source, self-hosting, European jurisdiction or reducing dependency on non-European providers. But most existing lists stop at listing tools. I want Euro Toolhub to go one step further. Each provider profile can include: country and jurisdiction category alternative-to mappings open source status self-hosting availability EU data residency DPA availability certifications where known target audience strengths and limitations a transparent sovereignty score embeddable badges for providers The project starts in German because the first target market is DACH, but the structure is prepared for more languages later. The long-term vision is to build a practical decision platform for digital sovereignty: not just “which European alternatives exist?”, but “which one fits my actual use case?” Current categories include web analytics, cloud and hosting, newsletter tools, CRM, email, password managers, AI APIs, project management, video conferencing and e-signatures. I’m looking for feedback from developers, SaaS founders, privacy people and self-hosting communities: Which European tools should be added? Which categories matter most? What would make a sovereignty score trustworthy? Should the provider dataset be opened through GitHub for corrections and submissions? Project: https://www.euro-toolhub.eu/de Provider submission: https://www.euro-toolhub.eu/de/anbieter-eintragen ``

Slides have an odd status in engineering: we make them constantly, but we treat them as disposable. They rarely live in a repo. They rarely go through review. The diagram you spent twenty minutes aligning in Keynote is gone the moment the talk ends. Dinghy's Slide Builder treats a slide like any other artifact: a folder of source files that compiles to a single deliverable. What it is A presentation builder layered on top of RevealJS , wired into the same Dinghy CLI you already have: YAML DSL: recognized keys map to semantic blocks, and any other key becomes the matching HTML element, so you describe slides in a structured, indented form. Markdown and HTML, auto-loaded: drop a .md or .html file in the slide folder and it becomes a section. Self-contained HTML output: assets inlined, a single file you can share or host anywhere. Live reload dev server: run dinghy slide start and edit-to-reload feedback. Prezi-style zoom and pan: exclusive to Dinghy, covered below. A YAML slide The core idea is that a slide is a tree of HTML elements, and YAML is a tidy way to describe nested structure: sections : - badge : Show Case title : Slide Builder subtitle : author RevealJS presentations in YAML ul : li : - YAML DSL maps keys to HTML elements - Self-contained HTML output - Live reload development server Some keys are recognized aliases: badge becomes <div class="badge"> , title becomes <h2 class="title"> , subtitle becomes <p class="subtitle"> . Any other key, such as p , h3 , ul , or li , becomes that HTML element directly. Multi-file slides are just multiple YAML files in the same folder, picked up in filename order. A Markdown slide If you want to add a section quickly, plain Markdown works: ## Slide Builder - YAML DSL maps keys to HTML elements - Self-contained HTML output - Live reload dev server Markdown files become slides automatically. You can mix YAML, Markdown, and raw HTML in the same slide and use whichever fits each section. The single-file output Run dinghy slid

Here's a bug that looks impossible until you understand Gson. You have this data class in your Pokedex app: data class PokemonStat ( @SerializedName ( "base_stat" ) val baseStat : Int , val stat : StatInfo ) A teammate cleans up the code and deletes what looks like a redundant line: data class PokemonStat ( val baseStat : Int , // @SerializedName removed val stat : StatInfo ) It compiles . No red errors. He runs the app — and every Pokemon's baseStat is 0 . Not the real 45 or 49. Just 0 . Everywhere. And Gson never threw an error, never logged a warning, never said a single word. If you can explain why this happens, you understand Gson better than most juniors. This is also a favorite interview question. Let's break it fully. What Gson actually does When the JSON comes back from the server, Gson goes key by key: Read a key from the JSON — say base_stat . Look for a Kotlin property with the exact same name . Found it? Pour the value in. Not found? Leave that property at its default value and move on. That's the entire matching game — exact name match, or nothing. Now look at the "cleaned up" class. The JSON key is base_stat . The Kotlin property is baseStat . Those are not the same string . Gson looks for a property called base_stat , doesn't find one, shrugs, and leaves baseStat at its default. The default for an Int is 0 . That's your bug. @SerializedName("base_stat") was never redundant. It was the sticky note telling Gson: "this property is called baseStat in Kotlin, but look for base_stat in the JSON." Delete the note, and Gson stops matching. Two ways to fix it: Rename the property to base_stat — works, but breaks Kotlin's camelCase convention. Put @SerializedName("base_stat") back — keeps the clean name and matches. This is the right one. But why 0 ? Why not a crash? This is the part that surprised me. A missing field feels like it should be an error. It isn't. Gson treats a missing key as allowed . It builds your object, fills the fields it found, and leaves

Note: If you're new to execution calculations, I'd recommend starting with my previous post which covers them in detail, then coming back here. What is attribute capturing and why would you use it Attribute capturing refers to directly exposing certain attributes to your execution calculation from attribute sets, to be used in execution calculations. Capturing attributes lets you handle things like damage in a more complex way. So for example, you could have advantages and weaknesses against certain damage types, or an attribute like armor that should reduce incoming damage. This is just two use cases, but once you learn how to do it, you should naturally be able to see in what other ways they can be used. In this example, I will show how to get attribute magnitude from captured attributes and using them in calculations, which is the most common way of using captured attributes. Capturing attributes Defining the struct, declaring and defining attribute capture definitions In this post, I will show how to use a static struct to access all your captured attributes, as a good performance-aware solution. You are also free to choose not to use the struct. For the struct approach, the first step is to go to the .cpp of the execution calculation, and define your struct there. In the struct, we first use the DECLARE_ATTRIBUTE_CAPTUREDEF() macro, passing in the name of your attribute. After that, we make the constructor of the struct, and in the constructor we use the DEFINE_ATTRIBUTE_CAPTUREDEF() macro, passing in the attribute set that has the attribute, the attribute from it, whether the attribute from the Target or Source should be used (the Target is the ASC the ExecCalc is outputting its result to, and the Source is the ASC that called it), and a bool for if the captured attribute should be snapshotted (if the attribute should be frozen at ExecCalc GE application (snapshotted) or if the value should be read at execution time). In my case, I will show 2 examples of non-

Installing Ubuntu Server 24.04 LTS in an offline mode (no LAN cable or WiFi connected) leaves you with an unmanaged network interface which requires manual configuration post-install. The interface enp4s0 is unmanaged by systemd-networkd and also the service itself is disabled. NOTE: This guide applies to Ubuntu server. On Ubuntu desktop you have the NetworkManager service installed and the steps would be different. To fix the problem follow the steps below: Create a Brand New Netplan Configuration File Look into /etc/netplan . You will probably see the dir is empty. This means the installer completely gave up on configuring the network and didn't create any profiles. Create a new file, e.g. vim /etc/netplan/01-netcfg.yaml . Paste the following configuration network : version : 2 renderer : networkd ethernets : enp4s0 : dhcp4 : true This makes the interface managed by networkd instead of the desktop NetworkManager. Fix permissions The new file needs to be readable only by root so fix it's permissions chmod 600 /etc/netplan/01-netcfg.yaml . Enable network service Enable the network service and make it start on boot: systemctl enable systemd-networkd systemctl start systemd-networkd Run netplan Finally, we need to tell Ubuntu to use our new configuration and activate the network netplan generate netplan apply Network should be up! ip a

Episode 1/4 of the mini-series The week Supabase lied to me 4 times . The three following episodes cover a mutation silently swallowed by the SDK [CANONICAL URL EPISODE 2: to fill in after push], an RLS recursion resolved by a JWT hook [CANONICAL URL EPISODE 3: to fill in after push], and a query that stops at exactly 1000 rows without saying so [CANONICAL URL EPISODE 4: to fill in after push]. The Tuesday the security probe spoke It's 9:12am on a Tuesday in May. The daily drift probe has been running automatically for three weeks — an aclexplode query across all public objects, filtered on anon . I don't open it every morning. That morning, it's waiting for me with a row that has no business being there. Niran sets a coffee on the corner of my desk without a word. He reads the output over my shoulder. A PII backup table — personal data in plaintext, created two days earlier for a bulk reclassification — shows up in the list with SELECT , INSERT , UPDATE , DELETE granted to anon . Accessible to any unauthenticated curl request. He lets three seconds pass and says: "It's not RLS." Then he goes back to his hoodie. He's right. It's not an RLS bug. The table itself is open, at the GRANT layer, before RLS even applies. Three objects, three doors, one mechanism That week, I realize I'm not dealing with an isolated incident. Three distinct objects, in three different migrations, each open a door nobody thought they'd opened. The backup table first. Then a policy set TO public because the public landing page needs it, which lets a POST {} from anon through with an HTTP 400 NOT NULL response instead of 401 Unauthorized . And finally four SECURITY DEFINER functions written to execute transactional operations with their owner's privileges, all invocable by anon because EXECUTE defaults to TO PUBLIC at CREATE time. Three objects, three superficially distinct mechanisms, yet one shared root. At every CREATE , Postgres completes the migration with an implicit GRANT the author nev

The question you don't dare ask out loud It's 10:40 PM on a Tuesday, I just closed an ADR — the seventy-fourth in this setup, written conscientiously, dated, cross-referenced with its migration, its contract test, and the commit that triggered it. And the question rises, the way it always rises at that hour when you've been coding alone for ten hours: who did I just write this for . No tech lead to convince, no PR review that'll catch it, no hypothetical acquirer to reassure, no architecture committee to brief tomorrow. Just the file, just me, just the doubt. It's the question of a solo dev at 70 days of serious practice. It has an honest answer, and that answer is neither "it'll pay when you sell" nor "it'll pay when you hire". Those two ROIs belong to other trajectories. The ROI of the solo dev who documents is an ROI he buys himself — deferred, intangible at moments, but materially countable if you force yourself to measure it in the first person. Here's mine, over 74 ADRs and 18 doctrine rules accumulated in 70 days, with no external observer to validate the grid. The false economy of "I'll remember" First trap, the one that cost me three weeks before I learned the lesson. The solo dev believes he doesn't need to write down what he decided because he decided it himself — his memory is worth an ADR. False at 14 days, systematically false at six weeks. Not because general memory fails, but because technical memory has a deceptive shape: you remember perfectly that you decided , you no longer remember why you decided that way. Three weeks after the May 5 session where I wrote ADR-0051 (FK ON DELETE SET NULL + CHECK NOT NULL incompatible, DELETE failing silently), I reopen the migration to add a column. I reread the diff, I don't understand why a certain CHECK constraint is phrased like this — the alternative I mentally dismiss today seems simpler, and I'm two clicks from refactoring. I go check the ADR. The answer is there, dated, sourced, in three lines. The simpl