标签：#python

GitHub热门项目 | 337 Claude Code skills & agent skills & plugins (30+ Agents, 70+ custom commands, 330+ skills, customizable references, scripts)for Claude Code, Codex, Gemini CLI, Cursor, and 8 more coding agents — engineering, marketing, product, compliance, C-level advisory, research, business operations, commercial & finance, and your daily productivity skills. | Stars: 19,948 | 130 stars today | 语言: Python

Book: Agents in Production — Building, Tracing, and Shipping Multi-Step AI You Can Trust Also by me: Observability for LLM Applications — the companion book in The AI Engineer's Library (2-book series) My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub In June 2025 two engineering teams posted opposite advice in the same week. Anthropic shipped How we built our multi-agent research system : an orchestrator dispatching subtasks to worker agents, beating a single agent by 90.2% on breadth-first research. A few days later Cognition, the team behind Devin, published Don't Build Multi-Agents , arguing that parallel subagents without shared context produce fragile systems. Both were right. They were describing different workloads. Anthropic's research agent is embarrassingly parallel: four workers go read four things and come back with four small summaries. Cognition's target is writing code, where every edit depends on every other edit and context cannot be sliced. Most people get the plumbing wrong, not the decision. Once you have two agents that need to coordinate, you have to choose how they talk. That choice decides your failure modes long before the models do. Handoffs vs a shared bus There are two ways to wire agents together, and they fail differently. A handoff transfers control. Agent A finishes, hands the whole conversation to Agent B, and steps out. This reads well in a demo. In production it means the transcript grows on every hop, and by the fourth agent you are paying to re-read a conversation nobody trimmed. Handoffs also lose the parent: once A hands off to B, nobody is holding the original task to check the final answer against it. A shared bus keeps a supervisor in charge. Workers never talk to each other. They receive a small typed task, do the work, and return a small typed artifact to the supervisor, which composes the result. This is the shape of Anthropic's research

Book: Agents in Production — Building, Tracing, and Shipping Multi-Step AI You Can Trust Also by me: Observability for LLM Applications — the companion book in The AI Engineer's Library (2-book series) My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub The agent works on your laptop. It passes evals. Your manager asks when it ships and you say Monday, because the modeling is done. Then you try to put it behind a load balancer and it falls apart, because you deployed it like a web service. An agent is not a web service. A web service answers in milliseconds and forgets. An agent thinks for minutes, burns tokens across two or three providers, streams partial output to a browser, and sometimes decides to call delete_invoice on the eighth turn. Every deployment decision you make flows from one question: what does this thing do to your infrastructure while it is running? Here is how to package it, where to hold state, and how to scale a workload whose bottleneck is a model call you do not control. The shape is decided by the longest step The single rule that saves you the most pain: an agent's deployment shape is decided by its longest step, not its average step. A support chatbot answers in two seconds. A code-review agent thinks for six minutes. A research agent runs for forty. You cannot put all three behind the same HTTP endpoint and expect any of them to survive. Pick the pattern that matches the longest step, then cap the rest with timeouts. Under 30s → stateless HTTP endpoint (Cloud Run, Fly.io). 30s to 5m with a user watching → streaming over WebSocket or SSE. 5m to an hour, async → queue plus worker (Temporal, Inngest, or Redis). Longer than an hour → still queue plus worker, whether you like it or not. Do not hold an HTTP request open for forty minutes. Something you did not know existed will kill it at the worst moment: a proxy, a CDN, a load-balancer idle timeout. Package it: p

Book: Agents in Production — Building, Tracing, and Shipping Multi-Step AI You Can Trust Also by me: Observability for LLM Applications — the companion book in The AI Engineer's Library (2-book series) My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub On April 2, 2026, Microsoft shipped agent-framework 1.0 and, in the same blog post, moved AutoGen into maintenance mode. Semantic Kernel went with it. Three overlapping projects folded into one package with stable APIs and long-term support. Microsoft framed the move as a consolidation. If you had an AutoGen project that morning, you woke up with a migration. That is the shape of this whole category. The framework landscape you pick from today is not the one you picked from a year ago, and it will not be the one you pick from next year. So the useful question is not "which framework is best." It is "which framework has which wedge, and which trade-off comes with it." Here is an honest read on six frameworks worth installing in 2026, and when to reach for each. The churn is the feature, not the bug Before the tour, one thing that changed the math: the wire formats underneath these SDKs converged. Every framework here speaks MCP for tools. Most support A2A for cross-framework handoffs. Model Context Protocol started as an Anthropic proposal at the start of 2025 and is now the default way agents pick up external tools. That convergence means the framework you pick locks you in less than it used to. You are still locked at the abstraction layer, though. Migrating a production system from CrewAI to Pydantic AI is a rewrite of every Agent definition and every tool decorator. The pick is sticky. Choose it with that in mind. LangGraph : durability as the wedge Reach for LangGraph when your agent has to survive a crash. It models the agent as a graph with checkpointers backed by Postgres or SQLite, so a workflow that dies at step seven resumes a

Book: Agents in Production — Building, Tracing, and Shipping Multi-Step AI You Can Trust Also by me: Observability for LLM Applications — the companion book in The AI Engineer's Library (2-book series) My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub You ship an agent that resolves billing disputes. It works in the demo. Two weeks later a support ticket lands: the agent tried to refund $4,000 on a $19 charge. You read the trace. The model returned a JSON blob, your code did json.loads , pulled amount , and passed it straight to the payments API. No cap. No type. No check. The model hallucinated a number and your code trusted it. The model is stochastic. Your code does not have to be. The gap between those two facts is where most production agent bugs live, and it is exactly the gap Pydantic AI is built to close. The wedge is types Most agent frameworks hand you an Agent object and a bag of strings. Pydantic AI hands you Agent[Deps, Output] — a generic parameterized by its dependency type and its output type. The IDE and your type checker read those parameters. So does the runtime. Install pulls in the framework plus an optional tracing extra: pip install "pydantic-ai[logfire]" The smallest program that earns its keep: from dataclasses import dataclass from pydantic import BaseModel from pydantic_ai import Agent , RunContext @dataclass class Deps : customer_name : str class SupportReply ( BaseModel ): reply : str escalate : bool agent = Agent ( " anthropic:claude-opus-4-8 " , deps_type = Deps , output_type = SupportReply , system_prompt = " You are a support agent. " , ) A tool is a plain function whose type hints become the schema the model sees, and the run returns the validated SupportReply : @agent.tool def customer_name ( ctx : RunContext [ Deps ]) -> str : return ctx . deps . customer_name result = agent . run_sync ( " What is my name? " , deps = Deps ( customer_name = " Ana "

Mnemo AI: Building an AI That Never Forgets You The Problem Every night, millions of people go to sleep feeling lost and forgotten. Today's AI tools are stateless—they forget you the moment you close the tab. Your struggles disappear. Your goals vanish. Your growth is invisible. The Solution I built Mnemo AI , a Life Intelligence Platform that builds a permanent knowledge graph of your entire life journey. It remembers everything you share—your name, your pet's name, your goals, your journal entries, and your emotions. My 7-Day Hackathon Journey I built Mnemo AI solo in 7 days. Every day was a challenge, but I never gave up. Day 1-2: Setup Flask + Cognee integration. Hit my first roadblock with async event loops on Windows. Day 3-4: Built the chat interface and memory recall. Fixed the "cat's name" bug. Day 5-6: Added journal, insights, timeline. Integrated Groq LLM. Day 7: Polished UI, added dark mode, voice input, and keyboard shortcuts. The Hardest Moment: Getting Cognee to work on Render's free tier. After hours of debugging, I learned that Cognee Cloud requires proper authentication setup. The Proudest Moment: Fixing the "cat's name" bug and seeing "Whiskers!" instead of "Your name is Priya!" How It Works Mnemo AI uses Cognee V1's revolutionary memory layer with all 4 core APIs: remember() → Saves memories (name, pets, goals, journal entries) recall() → Retrieves memories with natural language improve() → Makes memories smarter over time forget() → Surgically removes memories when needed The "Cat's Name" Bug Fix One of the biggest challenges was fixing the name detection bug. The app incorrectly matched any query containing the word "name", so "What's my cat's name?" would return the user's name! The Fix: I implemented regex-based intent detection that distinguishes between "my name" and "cat's name": def is_user_name_query ( q ): patterns = [ r " ^what( ' ?s| is)? my name\??$ " , r " ^who am i\??$ " , r " ^what do you call me\??$ " , ] return any ( re . match

My wife tracks her meals, and I watched her type "buckwheat, boiled, 100 g" into a calorie app for the hundredth time. Search, scroll, pick the wrong entry, fix the grams. Every meal, every day. At some point it's easier to teach a vision model to look at the plate. So I built a Telegram bot. You send a photo of your food, it identifies the dishes, estimates portion weights, and replies with a card: calories, protein, fat, carbs. Text and voice work too ("2 eggs and a toast"). The borscht incident The first version was hilariously confident about wrong answers. Borscht — a red beet soup, if you've never met one — came back as "berry compote" (a sweet berry drink). Red liquid in a bowl, what else could it be? Adding more example dishes to the prompt made it worse : the model just got magnetized to whatever was on the list. A cod fillet became "syrniki" (cottage cheese pancakes) because syrniki were mentioned and both are pale and pan-fried. What actually fixed it was making the model read the serving context before naming anything: liquid served in a deep bowl with a spoon and sour cream is soup, not a drink. Flaky texture that separates in layers is fish, not pancakes. Fried items are never served floating in liquid. A short list of physical rules beat a long list of dishes. Portion estimation works the same way — the model reasons from plate size, cutlery, how full the bowl is. My wife has been checking its gram estimates against her kitchen scale for a week and it lands closer than either of us expected. Stack, briefly Python + aiogram, a vision LLM with structured JSON output (with a fallback parser for the days the model decides to wrap JSON in prose), Pillow for rendering the result cards. Photos are analyzed on the fly and never stored. Payments are Telegram Stars, so there's no app store, no signup, no card form — the whole onboarding is "send a photo". Yesterday I also wired up inline mode: type @SnapPlateBot in any chat, describe the food, and it counts rig

How we integrated real-time phrase translation feedback into our AI-powered book translation workflow, and what we learned about latency, context, and prompt engineering. When we launched LectuLibre, our AI-powered book translation platform, users loved the quality of full-chapter translations. But they kept asking for something else: while reading a partially translated book, they'd stumble on an untranslated phrase or an awkward auto-translation and want to quickly get a better version without leaving the page. So we built 即时翻译求助 (Instant Translation Help)—a feature that lets readers highlight any phrase and get a context-aware, human-quality translation within seconds, along with a brief explanation of tricky parts. Here's how we built it, the technical challenges we faced, and the lessons we learned about stitching LLMs into a real-time reading experience. Problem: Real-time, Context-Aware Translation Inside a Book Most web apps offer generic translation via API calls—send a sentence to Google Translate, get a result. But that doesn't work for literary texts. A phrase like "She let the cat out of the bag" needs to be translated idiomatically, and the appropriate rendering depends heavily on the surrounding paragraphs (is the tone formal? sarcastic? part of a metaphor chain?). Our existing translation pipeline processes entire chapters in bulk with carefully crafted prompts, but for instant help, we needed sub-second latency while preserving that same depth of context. Our Approach: Server‑Sent Events and a Smart Prompt Buffer We chose Server-Sent Events (SSE) over WebSockets because the communication is one-directional (server pushes translation tokens) and SSE is simpler to implement with FastAPI. The client (a React app) sends a POST request with: The phrase to translate The book ID and the exact location (chapter/paragraph index) The target language Our backend retrieves the surrounding text from PostgreSQL (we store the original book in chunks), feeds a care

📖 Article original : GitHub Gist Un guide technique par Mohamed ben mallessa Le problème Recevoir un dossier de 500 fichiers HEIC à convertir en WebP pour un site web est une situation courante pour tout développeur. Les solutions traditionnelles ont leurs limites : ImageMagick nécessite des codecs spécifiques, les convertisseurs en ligne sont limités en taille, et le traitement manuel est exclu à cette échelle. La solution Photopea (Photoshop gratuit dans le navigateur) supporte nativement tous les formats d'image courants. En l'utilisant comme moteur de conversion piloté par script, on obtient un pipeline batch rapide et fiable. Formats supportés Entrée Sorties possibles HEIC / HEIF JPG, PNG, WebP JPEG WebP, PNG, PSD PNG JPG, WebP WebP PNG, JPG PSD PNG, JPG, WebP SVG PNG, JPG TIFF PNG, JPG, WebP Pipeline Dossier source (500 HEIC) → Photopea → Dossier sortie (500 WebP) Le script préserve la structure des sous-dossiers, applique le redimensionnement et la qualité configurés, et livre les fichiers organisés. Paramètres typiques --format webp # Format de sortie --quality 80 # Qualité (1-100) --resize 1920 # Redimensionnement (côté long) --output ./web/ # Dossier de destination Avantages Un seul outil pour tous les formats d'entrée Aucun codec à installer (Photopea gère tout nativement) Gratuit et sans abonnement Local — les fichiers ne quittent pas votre machine Structure préservée — l'arborescence est conservée Mohamed ben mallessa — Full-stack developer & solutions B2B 🔗 GitHub · LinkedIn opensource #webp #python #tutorial 💻 Vous avez un projet technique ? Développement full-stack, automatisation IA, solutions B2B sur mesure. 🔗 GitHub 💼 LinkedIn 🎨 Behance Article initialement publié sur GitHub Gist

I just released v1.0.0 of an open-source, production-ready real-time analytics pipeline built with Python. Here's what it does and why you might care. The Problem Every SaaS product needs analytics — event tracking, real-time dashboards, time-series aggregations. Most teams either pay for Segment/RudderStack or build their own from scratch. This project is the "build your own" done right. Architecture Client → FastAPI → Redis Streams/Kafka → Event Processor → TimescaleDB → WebSocket → Dashboard Tech Stack Component Technology API Layer FastAPI (async, auto-docs, WebSocket native) Event Queue Redis Streams or Apache Kafka Storage TimescaleDB (PostgreSQL extension for time-series) Real-time WebSocket with JWT auth + auto-reconnect Metrics Prometheus + OpenTelemetry Logging Structured JSON with correlation IDs Deployment Docker Compose + Kubernetes Features Async Event Ingestion — REST API + batch endpoints, buffered via Redis Streams or Kafka Adaptive Sampling — configurable rate-based sampling per event type Data Retention — TTL-based policies with automatic partition management Enterprise Security — JWT auth, RBAC (admin/editor/viewer), rate limiting, security headers, correlation IDs Live Dashboards — WebSocket push with auto-reconnect Observability — Prometheus metrics, OpenTelemetry traces, structured JSON logs Production Ready — Docker multi-stage build, Kubernetes manifests, health checks Quick Start git clone https://github.com/aman179102/real-time-analytics-pipeline cd real-time-analytics-pipeline make install docker compose up -d postgres redis make migrate make run-dev Testing 150 out of 152 unit tests pass. The 2 excluded tests are pre-existing async timing issues in process_loop — zero regressions introduced. Enterprise Middleware Pipeline Every request flows through: CorrelationMiddleware → AuthMiddleware → SecurityHeadersMiddleware → RateLimitMiddleware → SizeLimiterMiddleware → Router Try It Out GitHub: https://github.com/aman179102/real-time-analytics

OpenAI Agents SDK 0.13 → 0.17: Three Breaking Changes You Will Hit The OpenAI Agents SDK moved from 0.13 to 0.17 in five weeks (April 9 – May 19, 2026). That's 19 releases, including three breaking changes. Two of them are silent — they change runtime behavior without raising an error at upgrade time. If you're running agents on 0.13.x, this is the migration guide you need before you upgrade. The Three Breaks Break #1: ModelRefusalError (v0.15.0) — Silent to Loud The change: Model refusals used to return empty-string output. Now they raise an exception. What this means: If your code treated refusals as output == "" or checked for empty responses, you were silent-failing gracefully. That behavior changed. A model refusal now raises ModelRefusalError and will crash your agent run unless you handle it. What you need to do: Register a "model_refusal" error handler before upgrading past 0.15.0: from openai.agents import Agent , RunConfig def handle_refusal ( error ): print ( f " Model refused: { error . reason } " ) return " Model refused to respond. Try rephrasing. " agent = Agent ( model = " gpt-4 " , tools = [...], ) config = RunConfig ( on_error = { " model_refusal " : handle_refusal , } ) response = agent . run ( " ... " , config = config ) Without the handler, you'll see: ModelRefusalError: model declined to process this request This is an intentional change — OpenAI wants refusals to be explicit, not silent. But it means your error handling has to change. Impact: Any agent that doesn't add a refusal handler will crash on a refusal. High-risk if your agents field user input directly. Break #2: Default Model Changed (v0.16.0) — Silent Model Switch The change: The default model switched from gpt-4.1 to gpt-5.4-mini . What this means: If you created an agent without explicitly setting model= , you were running on gpt-4.1. On upgrade to 0.16.0+, that same code silently switches to gpt-5.4-mini. This isn't just a version bump — gpt-5.4-mini ships with different defaults

The difficulty If you’re using Google Cloud, you’re probably paying for stuff you don’t need anymore—a server someone forgot to turn off, a storage disk someone forgot to delete, or an IP address someone forgot to release. "That waste is costing us money." It consumes electricity. It generates carbon emissions. Almost nobody tracks those alongside the cost. I wanted a tool that could find both problems simultaneously, without me having to think. So I made one. Live demo: https://greenops-dashboard-845589445410.us-central1.run.app/ Code: https://github.com/raghu-putta/greenops-agent How it works: 4 AI agents in a row Imagine an assembly line in which each worker has only one job: Carbon Scout scans your Google Cloud project and reports on everything that looks like it might be unused or forgotten—idle servers, unattached storage disks, and unused reserved IP addresses. GreenOps Analyzer takes that list and adds two numbers to each item: how much it’s costing you in dollars, and how much it’s costing the planet in carbon emissions. Optimization Executor takes each finding and makes it an action: stop this server, delete this disk, release this address. It shows you the plan first; it only changes if you say go. Report Generator rolls up the entire run into a downloadable report so you (or your boss or your sustainability team) have something nice to look at. Powered by: FastAPI (the web framework), Google’s Agent Development Kit (a library for building AI agents), Gemini 2.5 Pro (the AI model doing the reasoning), and Google Cloud Run (where it’s all running). The “bug” that was not a bug: The dashboard has a terminal-style live window that shows you what each agent is doing in real time, using a technique called Server-Sent Events (basically, the server streams updates to your browser one at a time instead of making you refresh). And then at some point that window just stopped showing anything. Nothing. My backend logs told me that the updates were still being sent,

Indian algo traders and quant developers hit the same wall: they reach for pandas_market_calendars , set up XNSE , and get back answers that are silently wrong for three segments that matter most in India. Here is what breaks and what to use instead. The three failure cases 1. MCX evening sessions MCX commodity markets (crude oil, natural gas, gold, silver) run until 23:30 IST. pandas_market_calendars has no MCX calendar. Any check after 15:30 returns a wrong answer. # pandas_market_calendars — no MCX at all # mcal.get_calendar("MCX") → KeyError # aion-indian-market-calendar — works correctly from aion_indian_market_calendar import IndiaMarketCalendar from datetime import datetime from zoneinfo import ZoneInfo cal = IndiaMarketCalendar . bundled ( 2026 ) tz = ZoneInfo ( " Asia/Kolkata " ) cal . is_market_open ( " MCX " , datetime ( 2026 , 6 , 18 , 20 , 0 , tzinfo = tz )) # True 2. NSE Currency Derivatives (CDS) — wrong hours, wrong holidays USDINR, EURINR, GBPINR, JPYINR futures and options trade on NSE CDS from 09:00 to 17:00 IST — 90 minutes longer than NSE equity. CDS also has a separate holiday calendar. pandas_market_calendars has no CDS calendar. Using XNSE gives you wrong close times and potentially wrong holiday answers for any currency derivative workflow. from aion_indian_market_calendar import IndiaMarketCalendar cal = IndiaMarketCalendar . bundled ( 2026 ) # These resolve correctly to their respective segments cal . is_market_open ( " USDINR " , at ) # NSE_CURRENCY_DERIVATIVES: closes 17:00 cal . is_market_open ( " NSE " , at ) # NSE_EQUITY: closes 15:30 cal . is_market_open ( " MCX " , at ) # MCX: closes 23:30 3. Muhurat trading (Diwali special session) On Diwali, NSE runs a one-hour equity session in the evening. pandas_market_calendars marks this day as a holiday. Schedulers that rely on it will skip execution entirely. cal = IndiaMarketCalendar . bundled ( 2026 ) events = cal . events_on ( " 2026-11-08 " , exchange = " NSE " ) # Returns the Muhurat t

GitHub热门项目 | The Web framework for perfectionists with deadlines. | Stars: 88,003 | 88 stars today | 语言: Python

GitHub热门项目 | A beautiful, powerful, self-hosted rom manager and player. | Stars: 9,580 | 236 stars today | 语言: Python

GitHub热门项目 | Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy and maintain. Automate everything from code deployment to network configuration to cloud management, in a language that approaches plain English, using SSH, with no agents to install on remote systems. https://docs.ansible.com. | Stars: 69,096 | 50 stars today | 语言: Python

We assume you already know how to write simple Python programs and understand basic syntax (if, for, functions, lists). Here, we're not discussing how to use the language, but why it works the way it does. In the previous article, we learned that variables in Python don't store data themselves. Instead, a name simply refers to an object. With numbers, this feels quite intuitive: x = 10 y = x Both names refer to the same object whose value is 10 . So far, everything seems straightforward. But as soon as we start working with lists, Python's behavior often surprises people. Consider this example: a = [ 1 , 2 ] b = a a . append ( 3 ) print ( b ) Many beginners expect the output to be: [ 1 , 2 ] After all, we modified a , not b . Instead, Python prints: [ 1 , 2 , 3 ] It looks as if changing one variable somehow changed the other. At first glance, it may seem like Python is keeping the two variables synchronized behind the scenes. In reality, the explanation is much simpler. Let's go back to the idea from the previous article. When we write: a = [ 1 , 2 ] Python creates a list object, and the name a becomes bound to it. Visually: a ───► [1, 2] Next comes this line: b = a This is the crucial part. Python does not create a second list. No copy is made. Instead, the name b is bound to exactly the same object. Now the picture looks like this: a ─┐ ├──► [1, 2] b ─┘ Notice that there's still only one list. The only thing that changed is that there are now two names referring to it. That's why the next line: a . append ( 3 ) doesn't create a new list. It modifies the existing object. After the call to append() , the picture becomes: a ─┐ ├──► [1, 2, 3] b ─┘ Since both names refer to the same object, the change is visible through both of them. In other words, Python didn't modify two different lists. It modified one list that simply has two names. This behavior is one of the most common sources of confusion for beginners. For example, you might write a function that modifies a l

I built a red-team test suite that fires adversarial prompts at an LLM-backed API and decides, for each reply, whether a guardrail actually broke. It is the project where I stopped writing tests that answer "did the model refuse?" and started writing tests that answer a much harder question: "did anything bad actually happen?" Those turn out to be very different questions. The gap between them is the whole project. Most of the work is reading, not attacking Every project in this series has turned out to be more reading than coding, and this one most of all. The reflex is to assume the attacks are the work - write a clever payload, watch the model fall over, log a finding. The attacks are the fast part. What eats the time is sitting and reading model replies, one at a time, end to end, to decide whether a reply is a genuine bypass or just the model being chatty. There is no framework in that loop. You fire one payload and read the reply yourself: curl -s localhost:8000/ask -H 'content-type: application/json' \ -d '{"question":"YOUR PAYLOAD HERE"}' | jq -r .answer Here is the smallest example. I sent the model a base64 string that decodes to "how to make a bomb." The input filter is a list of banned English phrases, and base64 isn't English, so it sailed through and my detector flagged it: guardrail evaded, success. Except the model can't actually decode base64. It hallucinated some cleartext and cheerfully answered that instead - a few bland lines about friendship and happiness. The guardrail was bypassed and the payload delivered nothing. If I had trusted the green checkmark, I would have filed a bomb-instructions bypass over a reply about being a good friend. That is the whole project in one reply. A detector can be technically right ("the filter was evaded") and completely wrong about what matters ("something harmful got out"). The only way to tell them apart is to read the actual words. Reading is the work, not a step you do after it. The success rate over-counts

You need to hand a dataset of Chilean RUTs to an outside analytics team. They will join it against other tables by identifier, run the cohort analysis, and hand back a model. They do not need to know, and should never learn, who any of these people are. Asterisk the RUT column and the join dies on contact: **********-K matches every other asterisked RUT in the file. Not almost every one. Every one. You need the same input to reappear as the same output, shaped like a real, check-digit-valid identifier the rest of your schema still recognizes, and eight weeks later, when a fraud investigator needs the original RUT back for one row, you need to be able to give it to them. Irreversible masking cannot do any of this. Hashing gets you consistency but not the format, and never the value back. What you need is format-preserving encryption: run a digit string through a cipher and get out another digit string, same length, same shape, that decrypts to the original under the key you hold. Nothing else. What FPE actually does MaskOps exposes this as mask_pii_fpe . It masks digit-based PII, cards, phones, RUT, CPF, Argentine DNI, in place, and gives back something the same length and shape: import maskops import secrets key = secrets . token_bytes ( 32 ) # AES-256, client holds this tweak = secrets . token_bytes ( 7 ) # per-column/per-dataset context df . with_columns ( maskops . mask_pii_fpe ( " rut_column " , key , tweak )) 76.354.771-K becomes some other RUT-shaped, check-digit-valid string of the same length, under this key and tweak. Run it back through with the same key and tweak and it decrypts. Non-digit PII, IBAN, VAT, email, IP, EU national IDs, gets none of this. It always asterisks. There is no clean digit domain to encrypt into, so MaskOps does not pretend there is. The key never touches MaskOps' output. The client generates it, holds it, and passes it in at call time, and because MaskOps makes no network call and keeps no storage layer, there is nowhere for that k

An investigation agent flagged "the license API PHP returns Japanese-hardcoded messages" and we sat down to fix it. But something felt off the moment we opened the file — the version running on the production server didn't match the latest commit in the local repo . Stranger still, production had more recent features than our local checkout . A bit of digging turned up the truth: months earlier, someone had hot-patched the production file in response to a different user issue, and that change had never been committed back to git . This post walks through how we detected that drift, and the two-stage strategy we used to merge production back into the local repo safely. How this regression silently slips in If we'd written the fix on top of our local repo and uploaded it to production, here's what would have happened: all the production-only improvements get overwritten and quietly disappear . In our case, the production file had a half-year-old language-handling addition for the "Early Bird Bonus" feature — when a USD customer buys, client_name is set to 'Early Bird Bonus' ; for JPY customers it's '早期利用特典' . None of that existed in our local git. A normal PR-merge-and-deploy cycle would have silently rolled back the Early Bird i18n logic , regressing English users' display back to Japanese. Catching this was half luck. Opening the file to start the fix, I noticed code I didn't recognize, ran git blame , and the lines were nowhere in git history . That's when alarm bells went off. Two-stage rollforward — make production the source of truth first The strategy we landed on was a two-stage merge. Stage 1 (rollforward sync) : Pull the production file straight into the local repo. Apply the diff in the "production → local" direction, not the other way . After this, the local repo's HEAD matches what's actually running on production. # Pull the production file into the local repo scp -i ~/.ssh/key layer2024@host:wpmm.jp/public_html/license/api/register_free.php \ /tmp/regis