标签：#Python

I kept re-running the exact same input through my translation app. Same code. Same model. Same everything. And the word "machines" kept flipping between two different translations. Sometimes it came out as "機械" (machine). Sometimes as "あなたのPC" (your PC). No code changed between runs. No input changed either. My first assumption was a race condition somewhere in my pipeline. It wasn't. Where I actually looked I checked the obvious suspects first: caching, threading, anything stateful that could make the same input behave differently on different runs. All clean. So I went one level deeper, into how the model picks the winning word. Translation models score every candidate word and pick whichever scores highest. When I logged the actual scores for "machine" vs "your PC" on this input, they were almost exactly tied. That's the part that mattered. When two candidates are separated by a tiny margin, the order floating-point operations get summed in can nudge the score just enough to flip which one wins. Same math, same inputs, different accumulation order between runs — and a near-tie flips sides. Nothing was actually random. It was deterministic all the way down. It just wasn't deterministic in a way I could predict, because the thing that decided the winner was rounding noise several layers below anything I was testing. The fix wasn't "make it deterministic" Forcing strict floating-point determinism across an ML pipeline is its own rabbit hole, and not one I wanted to go down for one word. Instead, I looked at why the tie was so close in the first place. "Machine" and "your PC" were close enough in meaning, in this context, that the model wasn't confident either way. So I widened the margin instead of trying to eliminate the noise: I swapped the input word choice from "machines" to "equipment," which the model was much more decisively confident about. Scores stopped being close enough for rounding noise to matter. The flip-flopping stopped. I want to be honest about a

AudioTrust: reconciliar C2PA y watermark AudioSeal en audio sintético Un verificador local que lee las dos marcas de confianza de un audio generado por IA (procedencia C2PA + watermark AudioSeal) y emite un veredicto auditable sobre si coinciden, se contradicen o faltan. El problema Un audio sintético puede llevar dos marcas de confianza distintas: Procedencia C2PA : un certificado digital embebido en el archivo (su "DNI" de origen — quién, cuándo, con qué herramienta). Watermark AudioSeal : un código inaudible incrustado en el sonido, detectable aunque el audio se comparta o transcodifique. Cada una por separado es útil, pero ninguna es suficiente. La procedencia puede faltar (mucho audio generado no la incluye) y el watermark puede estar presente en audio totalmente legítimo. El caso interesante es cuando se contradicen : el manifest C2PA dice "grabado por un humano con una grabadora" pero el watermark de una herramienta de IA está presente. Eso es una señal de manipulación — el llamado Integrity Clash . AudioTrust no genera ni firma nada. Es un verificador : lee ambas capas y las reconcilia. Qué hace audio.wav ──► AudioTrust verify ──► veredicto + explicación C2PA watermark Veredicto ausente ausente unverifiable ausente presente partial origen sintético presente trusted origen humano presente contradiction (Integrity Clash) Salida JSON: { "file" : "audio.wav" , "verdict" : "trusted" , "c2pa" : { "present" : true , "source_type" : null , "claims" : [ "action=c2pa.created by TestTTS" , "generatedBy=TestTTS" ]}, "watermark" : { "present" : true , "detect_prob" : 0.92 }, "explanation" : "C2PA declara origen sintético y hay watermark fuerte: coherentes." } Cómo funciona Lectura C2PA con c2pa-python (el Reader de la librería oficial). Si no hay manifest, devuelve present=False sin crashear. Detección de watermark con audioseal . Devuelve solo detect_prob (P(audio watermarked) en [0,1]). Reconciliación determinista en reconcile.py . Dos decisiones de diseño que vale la

One day the browser automation flow started failing right after plugin updates with NameError: name 'plugin_form_selectors' is not defined in the post-update "residual check" step. The refactor that introduced this had landed back in v1.6.1. The error didn't surface until many rounds later. Reading the code, the cause is obvious in seconds — but nobody hit it for ages, because Python's lazy evaluation kept the leftover reference hidden until exactly the right execution path ran. This post walks through what the bug was and how we structurally prevented its kind via an AST static-analysis test. What happened — a reference that crossed a scope boundary browser_utils.py has two functions involved: run_browser_update_flow() , which orchestrates the whole update flow, and browser_update_remaining_plugins() , which handles only the plugin-update logic. The list of plugin-form selector candidates, plugin_form_selectors , used to be a local variable inside run_browser_update_flow() . In the v1.6.1 refactor — "let's split plugin update into its own function" — we created browser_update_remaining_plugins() and moved the plugin_form_selectors definition into it . # After v1.6.1 refactor def browser_update_remaining_plugins ( page , site , update_url ): plugin_form_selectors = [ # ← defined here ' #update-plugins-table-wrap form ' , ' form[name= " upgrade-plugins " ] ' , ' form[action*= " do-plugin-upgrade " ] ' , ' .plugins-php form ' , ] # ... update logic ... def run_browser_update_flow ( site , page ): # ... call to plugin updater ... browser_update_remaining_plugins ( page , site , update_url ) # ★ post-update "residual check" still uses the old local name for selector in plugin_form_selectors : # NameError if page . locator ( selector ). count () > 0 : pending_browser . append (...) The " after updating, make sure no plugin update forms are still visible " residual check stayed in run_browser_update_flow() . During the refactor, the call to extract this loop alongside the

While we can build simple applications without using classes using PySide6, But in big applications and Massive coding systems We should use Classes But why? To understand why do we need classes in PySide6 We should first see the Python code First from PySide6.QtWidgets import QApplication , QWidget , QPushButton , QLineEdit import sys class MainWindow ( QWidget ): def __init__ ( self ): super (). __init__ () button1 = QPushButton ( " Button 1 " ) input = QLineEdit () if __name__ == " __main__ " : app = QApplication ( sys . argv ) window = MainWindow () window . show () app . exec () Before talking about why do we need Classes for PySide6 Let's Explain the code first line by line The imports first thing we make the imports we do need: from PySide6.QtWidgets import QApplication, QWidget, QPushButton, QLineEdit The QApplication Is the simply the application we will make, Like empty app on the RAM it do nothing but it's on the RAM if it's alone And the QWidget Is the Blank screen That will be placed on the Empty Application in the RAM The QPushButton Is like any button we are saying in any app Like the Subscribe button on YouTube or like Post button on Twitter QLineEdit is the input bar, Like the input bar of ChatGPT where you put on it your prompt or like The input bar in WhatsApp Where you type any thing on it to send it to your friends The class And finally The thing You clicked on the post for First thing we define the class How can we define it? Why do we need to define it? Why do even we want it? Who created it? (NOOO IAM JUST KIDDING) We can simply define the class in python by just typing class That's it just class then the name of it For Example MainWindow and then a little semi-colon : OR EVEN WE GIVE IT A Parents And Why do we need to define it, For simply use it BRILLIANT RIGHT? And we want the classes in PySide6 for give it a parents QWidget or even QMainWindow , And we will explain both of them right now but before it Let's explain first what does parents

Your customers have stopped Googling "best note-taking app." They're asking ChatGPT, Perplexity, and Gemini instead — and getting back a short list of three or four products. If your brand isn't on that list, you're invisible, and unlike a Google ranking you can't even see where you stand. That's the problem I set out to measure. This post is the build breakdown: five AI answer engines, one uniform result shape, a mention-detection core that doesn't lie to you, and the honest gotchas I hit around cost and billing. The whole thing runs as a paid Apify Actor written in async Python. The niche has a name now — GEO (Generative Engine Optimization) or AEO (Answer Engine Optimization). Think SEO, but the search engine is a language model and the "ranking" is whether you get named in the answer. The core question Give the tool a brand, its competitors, and the buyer-intent questions your customers actually type: { "brand" : "Notion" , "competitors" : [ "Obsidian" , "Coda" , "Evernote" ], "prompts" : [ "best note taking app for students" , "Notion vs Obsidian which should I use" ], "engines" : [ "perplexity" , "chatgpt" , "gemini" , "claude" , "aiOverview" ], "samplesPerPrompt" : 3 } It asks each engine each prompt (several times, because LLM answers vary run-to-run), then analyzes every answer for: were you mentioned, how early, were you recommended or just listed, what's the sentiment, who else got named, and — the part incumbents skip — which domains each engine cited. That last one is the actionable output: it tells you which websites the AI trusts for your category, i.e. where you need coverage. Architecture: one shape to rule them all The trick that keeps the whole thing sane is that every engine adapter — whether it's a clean REST API or a messy HTML scrape — returns the exact same record shape : { " engine " : " perplexity " , " prompt " : " best note taking app for students " , " sampleIndex " : 1 , " responseText " : " ... " , " citations " : [{ " url " : " ... "

How I shipped structured JSON logging + Prometheus metrics with zero new dependencies I almost added structlog and prometheus_client to my pyproject.toml . Then I read what they actually do. Both libraries are excellent. structlog is the right call when you have a 30-engineer team shipping 50 services. prometheus_client is the right call when you have five teams of consumers scraping different metrics. For a single-author Python project with one process and one user, both are over-engineered. The 80 lines of code I would have pulled in, I can write in 200. The result: zero new runtime dependencies, full control over the output, and a smaller pip install footprint for every user. Here is what I did instead. The minimum useful observability surface A small Python service needs four things, in order of importance: Every log line is one JSON object. (No parsing for downstream tools.) Every request has a trace id. Every log line in that request carries the same trace id. (So you can grep by id and see the whole story.) Every log line goes to stderr. (So journald , Docker, and kubectl logs all see it without any extra configuration.) Every metric is exposed in Prometheus text format at a stable URL. structlog gives you #1, #2, #3 with a lot of flexibility. prometheus_client gives you #4 with a lot of flexibility. Both are about 16 MB of transitive dependencies combined. For a service that runs in a single process and exports maybe 20 metric names, the libraries are doing more work than the project. The 80-line JsonFormatter The custom logging formatter is the simplest part. The whole thing is here: import json import logging from contextvars import ContextVar from datetime import datetime , timezone _trace_id_var : ContextVar [ str | None ] = ContextVar ( " trace_id " , default = None ) class JsonFormatter ( logging . Formatter ): def format ( self , record : logging . LogRecord ) -> str : payload = { " ts " : datetime . now ( tz = timezone . utc ). isoformat (), " level

If you've ever tried to pull job listings by scraping LinkedIn or Indeed, you know the pain: anti-bot systems, CAPTCHAs, rotating proxies, and scripts that silently break every few weeks. Here's the thing — you usually don't need any of that. Companies don't post jobs on LinkedIn first. They post them in their ATS (Applicant Tracking System) — Greenhouse, Lever, Ashby, Workday, etc. — and most ATS platforms expose the company's board as a public JSON endpoint . No key, no login, no browser. It's the company's own source of truth, so it's cleaner and fresher than any aggregator. The endpoints A few that work with a plain GET ( {company} = the company's slug): Greenhouse — https://boards-api.greenhouse.io/v1/boards/{company}/jobs?content=true Lever — https://api.lever.co/v0/postings/{company}?mode=json Recruitee — https://{company}.recruitee.com/api/offers/ Breezy HR — https://{company}.breezy.hr/json SmartRecruiters, Ashby, BambooHR and Personio have their own equivalents. Workday is the one annoying exception — it's a POST and needs the full board URL (tenant + datacenter + site), so you can't guess it from a bare company name. Example: pulling Stripe's open roles (Python) Stripe uses Greenhouse: import requests company = " stripe " url = f " https://boards-api.greenhouse.io/v1/boards/ { company } /jobs?content=true " jobs = requests . get ( url ). json ()[ " jobs " ] for j in jobs [: 5 ]: print ( j [ " title " ], " — " , j [ " location " ][ " name " ]) That's it. No Selenium, no proxy, no CAPTCHA solver. Runs in ~200ms and won't break next Tuesday because Cloudflare changed something. Auto-detecting the ATS If you don't know which ATS a company uses, just try them in order and take the first one that returns jobs. A bare 404 means "not this ATS, try the next." Greenhouse → Lever → Ashby → SmartRecruiters → Recruitee → Breezy covers a huge chunk of tech companies. Gotchas Rate limits are lenient but real — be polite, set a User-Agent . Descriptions : Greenhouse/Leve

Python Redis: Caching and Fast Data Structures Redis is an in-memory data store used for caching, session storage, pub/sub messaging, leaderboards, rate limiting, and more. With redis-py 's async client, it integrates cleanly into any asyncio application. Installation pip install redis[hiredis] # hiredis is a C parser — 2-5× faster protocol parsing Connect and Verify import asyncio import redis.asyncio as aioredis from datetime import timedelta import json REDIS_URL = " redis://localhost:6379/0 " async def get_redis () -> aioredis . Redis : client = aioredis . from_url ( REDIS_URL , encoding = " utf-8 " , decode_responses = True , socket_connect_timeout = 5 , socket_timeout = 5 , retry_on_timeout = True , ) pong = await client . ping () print ( f " Redis connected: { pong } " ) return client Strings — Basic Cache with TTL async def cache_set ( r : aioredis . Redis , key : str , value : str , ttl : int = 300 ) -> None : await r . set ( key , value , ex = ttl ) async def cache_get ( r : aioredis . Redis , key : str ) -> str | None : return await r . get ( key ) # Cache-aside pattern async def get_user_profile ( r : aioredis . Redis , user_id : int , db ) -> dict : cache_key = f " user:profile: { user_id } " cached = await r . get ( cache_key ) if cached : print ( f " Cache HIT for user { user_id } " ) return json . loads ( cached ) print ( f " Cache MISS for user { user_id } — querying DB " ) user = await db . fetch_user ( user_id ) # your DB call if user : await r . set ( cache_key , json . dumps ( user ), ex = 600 ) return user or {} # Atomic counter async def increment_page_views ( r : aioredis . Redis , page : str ) -> int : key = f " views: { page } " count = await r . incr ( key ) await r . expire ( key , 86400 ) # reset counter after 24 h return count Hashes — Structured Objects async def save_session ( r : aioredis . Redis , session_id : str , data : dict , ttl : int = 3600 ) -> None : key = f " session: { session_id } " await r . hset ( key , mapping = data )

Builder Journal · ARC Prize 2026 There is a specific horror in a detective story when you realize the witness everyone trusted has been lying, or just wrong, the whole time, and every conclusion built on their testimony has to come down with them. I had that moment with my own notes this month. The unreliable witness was me. Context, if you are new to this thread : I'm competing in the ARC Prize 2026, building an agent that has to win games it has never seen. It had been stuck, underperforming on the hidden test in a way I could see on the scoreboard but could not explain, and I had been hunting the cause across several sessions. The two comforting facts In two earlier work sessions I had written down, as settled conclusions, two things about why the agent was failing. One: the failure was a kind that only happens on the hidden online games, so it could not be taken apart and studied on my own machine. Two: the practice games I did have were useless for investigating it anyway, because they scored a flat zero on the relevant measure. Notice what those two beliefs do when you put them together. They say, in a calm and reasonable voice, that there is nothing to be done here. The problem is unreachable, the practice data is a dead end, the smart move is to spend your energy elsewhere. They were not just facts. They were permission to stop looking. So I stopped looking. Twice. The hour that knocked it all down Eventually I made myself do the one thing I had been quietly avoiding. Instead of rereading my own notes for the third time, I went and checked. I wrote small probes and ran them against the real artifacts, the actual code and the actual game data, rather than against my memory of what they did. Both beliefs collapsed inside an hour. The failure was not unreachable. It came apart cleanly, deterministically, on the games I already had sitting on my disk. And the "dead end" practice data was not a dead end at all. It showed the problem plainly the moment I asked it

Module caching, execution order, and circular imports explained by tracing what actually happens. How Python's Import System Works and Why It Matters for Debugging The import system is one of the least understood parts of Python and one of the most practically important for debugging production issues. Circular import errors, unexpected code execution, and module state bugs all stem from not understanding what happens when Python encounters an import statement. What Happens on the First Import When Python executes import mymodule for the first time: Python checks sys.modules for mymodule . If found, returns the cached module object immediately. If not found, Python locates the module file. Python creates a new module object and adds it to sys.modules under the module name. Python executes the module file's code in the new module's namespace. The name mymodule in the importing module is bound to the module object. Step 3 happens before step 4. This is critical for understanding circular imports. The Module Cache import sys import os print ( " os " in sys . modules ) print ( sys . modules [ " os " ] is os ) Output: True True Every imported module is cached in sys.modules . Subsequent imports return the cached object without re-executing the module code. Module-Level Code Executes on Import # config.py print ( " config module loading " ) DEBUG = True print ( f " DEBUG is { DEBUG } " ) # main.py import config import config # second import print ( config . DEBUG ) Output: config module loading DEBUG is True True The print statements in config.py run exactly once — when the module is first imported. The second import returns the cached module object without re-executing the code. Circular Import Behavior # module_a.py print ( " loading module_a " ) from module_b import b_function def a_function (): return " from a " # module_b.py print ( " loading module_b " ) from module_a import a_function def b_function (): return " from b " When you import module_a , Python starts exe

I spent a chunk of last year around legacy modernization work — the kind of project where a bank or an insurer is taking twenty years of accumulated code and rebuilding it as modern services, one system at a time. Every one of those systems starts the same way: a PRD or a requirements document says what the business needs, that gets translated into a spec precise enough for an AI to implement, and eventually someone tests what came out. What struck me, watching this happen at scale, wasn't that the code was bad. It was that nobody was testing the thing that actually determined whether the code would be bad: the spec itself — the technical description handed to the model, not the PRD that motivated it. Every security tool I looked at — SAST scanners, DAST tools, even the AI coding assistants themselves — waited until an implementation existed before doing anything adversarial. Attack the code, once it's there. That's the whole industry's model, and it's worked fine for forty years because the volume was always survivable. A team ships a handful of PRs a week, a human reviews them, and eventually a pentest catches whatever slipped through. That math falls apart at modernization scale. When you're regenerating a few million lines of code, you're also generating a few thousand specs, faster than any review process was ever built to absorb. Testing after the fact doesn't just get slower under that load — it quietly stops happening, spec by spec, until the aggregate exposure is enormous and nobody can point to when it happened. So I built GAUNTLEX to test the thing that happens before the code does: the spec. This is also where I want to be precise about a word that gets overloaded. "Spec-driven development" — the broader industry shift toward writing structured, agent-facing specs instead of prompting an AI free-form — is exactly the world GAUNTLEX lives in. But a spec (what to build, precise enough for a model to implement) and a PRD or requirements doc (why it's needed

FROST周报 | 为什么智能体需要「谱系」？从生物学隐喻看AI治理新范式 作者按 ：本文是 FROST 开源项目的每日推广系列文章，周一深度篇。 一、一个被忽视的根本问题 当我们谈论 AI Agent 时，大多数讨论都聚焦于「能力」：能不能写代码？能不能调用工具？能不能规划任务？ 但有一个根本问题很少被触及： 当一个 Agent 执行了错误的决策时，谁来负责？当它消亡后，它的经验能否被传承？ 就像一个没有记忆的人，每次醒来都是白纸一张——这不叫智能体，这叫复读机。 FROST 正是为了解决这个「治理真空」而诞生的。 二、从细胞分裂到 Agent 家族 FROST 的核心哲学只有一句话： 细胞会死，但谱系会存续。Agent 会消亡，但宪法会传承。资产会永存。 这不是文学修辞，而是一套完整的技术架构。 四个原子：最小可行集合 FROST 只定义了四个原子，却能构建任意复杂度的智能体系统： 原子 职责 生物类比 Store 记忆容器，只做 save/load/delete 细胞核 Skill 纯能力单元，无状态无副作用 蛋白质 Agent 膜包裹的细胞，拥有 Store + Skills 神经细胞 SOP 有序步骤列表，可教学、校验、优化 宪法文本 from core import Store , Agent , skill_set , skill_get # 创建一个最小 Agent store = Store () agent = Agent ( " cell " , store , skills = { " set_context " : skill_set , " get_context " : skill_get }) # 执行任务 result = agent . run ( sop_steps = [ " set_context " , " get_context " ], initial_context = { " key " : " message " , " value " : " FROST is alive " } ) # result["_result"] == "FROST is alive" 关键洞察 ：Store、Skill、Agent、SOP 这四个概念彼此正交，可以自由组合。就像乐高积木，从简单到复杂，始终保持可解释性。 三、家族治理：超越扁平架构 传统的多 Agent 系统通常是扁平的：所有 Agent 平等对话，没有层级，没有记忆，没有责任边界。 FROST 引入了「家族治理模型」——一个三层递归结构： 祖辈 (Ancestor) ：定义不可违背的宪法与长期目标 父辈 (Parent) ：领域协调者，可递归委托 孙辈 (Leaf) ：执行具体原子任务，瞬态存在 四个协议保障治理闭环 ： 层级 Store 继承 ：祖先记忆只读，后代自动继承 SOP 宪法校验 ：祖辈审核后代 SOP，拒绝违规执行 编排层级限制 ： max_spawn_generation 硬编码，禁止越级 spawn 选择性持久化 ：父辈收割有价值产出，淘汰冗余 Agent 四、V5.0 五维元模型：多维治理架构 2026年7月发布的 V5.0 引入了一个重大升级—— 五维元模型 ： 维度 模块 核心职责 武器注册表 Armory 能力的元数据管理与发现 任务注册表 TaskRegistry DAG 任务编排与图谱 SOP 事件编目 EventCatalog + Strategist 态势感知与双模式事件分析 平台注册表 PlatformRegistry 外部能力的发现、调用与健康检查 规则注册表 RuleRegistry 可版本化的治理约束与合规检查 197 个测试用例 保障了每个维度的质量。 五、与现有框架的差异 维度 LangChain CrewAI FROST 状态管理 链式传递 角色记忆 层级 Store 权限边界 无 提示词软约束 代码强制只读 治理可审计 无 对话日志 结构化执行历史 架构无关 ✅ ✅ ✅ FROST 不重复造轮子。它填补的是「治理」这个空白地带： 让多智能体系统真正可控制、可追溯、可进化 。 六、快速体验 # 克隆仓库 git clone https://gitee.com/liao_liang_7514/frost.git cd frost # 运行测试 python -m pytest # 查看示例 python frost_run.py 完整文档： https://gitee.com/liao_liang_7514/frost 七、写在最后 AI Agent 的下一阶段，不是更强的模型，而是 更好的治理 。 当我们把 100 个 Agent 放在一起时，如果没有宪法、没有层级、没有记忆传承

I Tested Direct Provider APIs vs Aggregators — Here's the Truth Six months ago I was staring at a $48,000 invoice from an AI provider that shall not be named. We had committed to a six-month contract because the sales rep promised "priority routing" and "negotiated rates." What we got instead was a rate hike, an outage during our biggest product launch, and a support team that took 72 hours to respond. That was the moment I decided to stop signing contracts with AI providers entirely. This is the playbook I wish someone had handed me on day one — the architecture decisions, the math, and the code that lets a small team punch way above its weight class without betting the company on a single vendor. The Trap Most Startups Fall Into When I started my last company, I did what every founder does. I read the docs, got an API key, shipped a feature. The model worked, the demo went well, the investors nodded. Then we hit production traffic and the bills started arriving like clockwork. Here's what nobody tells you about going direct to a model provider as a startup: The pricing page you see on the website is the retail price. The actual cost of running production workloads includes rate limits you didn't anticipate, caching you forgot to implement, context windows that blow up your token count, and prompt engineering iterations that look cheap per call but compound fast. I watched one team burn $20K in a single weekend because they were streaming completions without setting a max_tokens guardrail. Direct providers also lock you into their ecosystem. Their SDK, their tools, their prompt format, their authentication scheme. The moment you want to A/B test a different model — which you will, probably next quarter — you're rewriting integration code instead of shipping features. And then there's the geopolitical mess. Some of the best models in 2026 come from providers that don't accept US credit cards. I've personally lost an afternoon trying to sign up for an account that re

Python is an excellent language for automating cloud infrastructure, but the official Kubernetes Python client leaves developers with an important architectural decision: Where should Kubernetes manifests live? Should they be constructed directly with Python objects? Embedded as multiline strings? Or stored as external files and rendered at runtime? Each approach works, but they have very different implications for readability, maintainability, and long-term operational cost. The key is recognizing that deployment logic and platform configuration evolve on different lifecycles. Your deployment code, the part that authenticates to Kubernetes, renders templates, and applies resources, may remain unchanged for months. Your manifests, however, often change weekly as applications evolve, resource limits are tuned, cloud-provider annotations are added, or networking requirements change. When those two concerns are tightly coupled, even a configuration, only change forces you to modify, test, and redeploy the delivery or application code itself. Over time, this increases maintenance costs, slows platform changes, and makes configuration drift and production mistakes more likely. This is a familiar software engineering principle: separate concerns that evolve independently. The same thinking that keeps application configuration separate from executable code also applies to Kubernetes manifests. Treating manifests as first-class configuration artifacts allows them to evolve independently from the Python code that delivers them. In this article we'll compare three ways of deploying Kubernetes resources with the official kubernetes-python-client , ranging from tightly coupled implementations to a design that cleanly separates deployment logic from platform configuration. The Landscape at a Glance The comparison below assumes a common application deployment scenario, where the desired state is largely known ahead of time. Controllers and Operators have fundamentally different r

"AI agent" gets thrown around so much I figured I should just build one instead of reading more threads about it. The core idea turned out to be small: you put Claude in a loop and hand it some tools. It picks a tool, you run it, you hand back the result, and it keeps going until it has an answer. Code is here if you want to skip ahead: claude-research-agent . What it can do Give it a task and it works out the steps on its own. Mine can: search the web (no API key for this part, it just hits DuckDuckGo's HTML page) open a URL and pull the readable text out do math without me trusting eval run read-only SQL against a SQLite file read local files, but only inside the project folder save findings to a notes file The loop is basically the whole thing Honestly this is most of it: messages = [{ " role " : " user " , " content " : user_message }] for _ in range ( MAX_STEPS ): response = client . messages . create ( model = " claude-sonnet-5 " , max_tokens = 2048 , tools = TOOL_SCHEMAS , messages = messages , ) messages . append ({ " role " : " assistant " , " content " : response . content }) if response . stop_reason != " tool_use " : return final_text ( response ) tool_results = [] for block in response . content : if block . type == " tool_use " : result = run_tool ( block . name , block . input ) tool_results . append ({ " type " : " tool_result " , " tool_use_id " : block . id , " content " : result , }) messages . append ({ " role " : " user " , " content " : tool_results }) The thing to watch is stop_reason . If Claude says tool_use , it wants you to run something. You run it, drop the result back into the conversation as a tool_result , and loop. When it stops asking for tools, you're done. The MAX_STEPS cap is just there so a confused agent can't spin forever. Tools are just functions Each tool is a Python function plus a little JSON schema telling Claude when to reach for it. Want a new capability? Write a function, add its schema. The loop never changes, which w

Hi guys, I've been working on Regdrift, my first open-source project. It's a CLI and GitHub Action that compares two CMSIS-SVD files to check whether there are any register-map changes that could affect firmware functionality. It catches changes such as moved registers, interrupt renumbering, access changes, and altered read/write behavior. It then classifies those changes as BREAKING , WARNING , or SAFE so the tool can act as a CI gate. I'm looking for feedback from people who maintain SVDs, HALs, PACs, SDKs, or firmware repositories. If possible, I'd like to test it against real old/new SVD pairs and learn where the classifications produce false positives, miss important changes, or are unclear. For people who work frequently with CMSIS-SVD files: which types of register-map changes are most detrimental to firmware or cause the most difficult code-related problems? Resources GitHub: https://github.com/Pranav-s79/regdrift Install pip install regdrift Usage regdrift check old.svd new.svd

skip에서 partition overwrite로: business_date 재처리를 Iceberg로 다시 표현하기 이전 글에서는 같은 source_hash 가 다시 들어왔을 때 기존 successful run을 재사용하는 idempotency를 다뤘다. 하지만 재처리에는 두 종류가 있다. 1. 같은 입력이 다시 들어온 경우 -> skip이 맞다. 2. 같은 business_date의 정정 입력이 들어온 경우 -> skip하면 안 된다. -> 같은 날짜의 gold 결과를 중복 없이 교체해야 한다. manufacturing-data-platform-mini 의 B5 slice는 두 번째 문제를 아주 작게 다룬다. 전체 Spark pipeline을 만든 것이 아니다. gold_daily_metrics Iceberg table 하나를 local Spark에서 만들고, business_date partition overwrite와 snapshot evidence만 검증했다. Scenario 이미 아래 gold row가 있다. business_date=2026-06-29 plant-a / line-1 / gearbox-a units_produced=120 defect_count=3 나중에 같은 business_date=2026-06-29 에 대한 정정 source가 들어온다. 운영자가 원하는 것은 append가 아니다. 원하지 않는 상태: 2026-06-29 old row 2026-06-29 corrected row -> 같은 날짜 결과가 중복됨 원하는 상태: 2026-06-29 corrected row만 남음 2026-06-30 같은 다른 날짜 partition은 그대로 유지됨 재처리 전후 snapshot evidence가 남음 그래서 이 slice의 질문은 이렇다. 같은 business_date의 정정 source를 처리할 때, gold table에서 해당 날짜 partition만 중복 없이 교체하고, 어떤 run이 어떤 Iceberg snapshot을 만들었는지 남길 수 있는가? Decision Pressure Slice1의 CSV pipeline은 already-successful source를 안전하게 skip할 수 있다. dataset_id + business_date + source_hash 이 key가 같으면 같은 입력이다. 다시 계산해도 같은 결과이므로 기존 run을 재사용한다. 하지만 source_hash 가 달라졌다면 의미가 다르다. same business_date different source_hash 이건 retry가 아니라 correction이다. CSV run-folder 방식에서는 새 run output을 만들 수는 있지만, "현재 gold table에서 해당 날짜를 원자적으로 교체한다"는 table-level 의미가 약하다. Iceberg를 붙이는 이유는 여기 있다. source_hash -> 같은 입력인지 판단하는 idempotency key business_date partition -> 정정 시 교체할 gold table 범위 snapshot_id -> table commit의 evidence 즉 Spark/Iceberg는 도구 이름을 추가하려고 붙인 것이 아니라, 재처리 상태 전이를 더 명확히 표현하기 위해 붙였다. Options Option 장점 문제 판단 same source면 항상 재계산 단순함 retry 때 불필요한 commit이 계속 생김 제외 corrected source를 append 구현 쉬움 같은 날짜 gold row가 중복될 수 있음 제외 whole-table overwrite 단순함 다른 날짜 partition까지 지울 위험 제외 business_date partition overwrite correction 범위가 명확함 Spark/Iceberg 설정과 test가 필요 선택 MERGE/upsert 강력함 이번 skeleton에 과함 backlog 이번 구현은 DataFrameWriterV2.overwritePartitions() 를 사용했다. corrected_d

wide CSV 여러 개를 EAV로 모아 gold mart 만들기 현실의 데이터 소스는 한 가지 모양으로 오지 않는다. 같은 의미의 값도 어떤 파일에서는 생산수량 , 다른 파일에서는 units , 또 다른 파일에서는 made 로 올 수 있다. 온도도 어떤 곳은 섭씨, 어떤 곳은 화씨일 수 있다. 이걸 매번 pipeline code에 if source == ... 로 박기 시작하면 source가 늘 때마다 코드가 지저분해진다. manufacturing-data-platform-mini 의 EAV mini slice는 이 문제를 작게 다룬다. 여러 wide CSV를 mapping config로 표준 attribute에 맞춘 뒤, EAV long format으로 모으고, 다시 gold metric mart로 pivot/aggregate한다. 데이터는 모두 synthetic이고, 회사 코드·고객 데이터·실제 schema는 쓰지 않았다. 1. Scenario 서로 다른 공장/라인/벤더에서 비슷한 제조 지표 파일이 들어온다. 예: plant_a.csv: 설비ID, 생산수량, 불량수, 온도C, 압력kPa plant_b.csv: machine_id, output_units, defects, temp_f, pressure_bar vendor_d.csv: unit_name, made, scrap, deg_c, kpa 비즈니스적으로는 같은 지표를 보고 싶다. units_produced defect_count temperature_c pressure_kpa 문제는 source마다 column name과 unit이 다르다는 점이다. 2. Decision Pressure 단순 구현은 source마다 코드를 늘린다. if source == "plant_a": 생산수량을 units_produced로 읽는다 if source == "plant_b": output_units를 units_produced로 읽는다 temp_f를 섭씨로 변환한다 if source == "vendor_d": made를 units_produced로 읽는다 이 방식은 작게는 빨라 보이지만 source가 늘수록 문제가 된다. 새 파일 형식마다 pipeline code를 고쳐야 한다. column mapping과 transform logic이 섞인다. unit conversion이 흩어진다. quality check가 source별로 갈라진다. gold mart grain을 설명하기 어려워진다. 그래서 mapping은 config로 빼고, pipeline은 표준 attribute를 처리하게 만들었다. 3. Options option result risk source별 hard-coded parser 처음엔 빠름 source가 늘 때 code change 반복 모든 source를 wide table 하나로 합치기 보기 쉬움 sparse/heterogeneous column 폭발 EAV long format 이종 attribute를 표준 형태로 모음 pivot/quality 설계가 필요 full mapping DSL/rules engine 유연함 mini project에는 과함 이 프로젝트의 선택은 단순한 JSON mapping + EAV long + gold pivot이다. 4. Decision 각 source는 JSON config로 자신의 column을 표준 attribute에 매핑한다. source column -> standard attribute output_units -> units_produced temp_f -> temperature_c with f_to_c pressure_bar -> pressure_kpa with bar_to_kpa pipeline 흐름: wide CSVs -> mapping configs -> EAV long rows -> gold entity_daily_metrics -> quality checks -> catalog/lineage EAV row의 핵심 shape: entity_id business_date attribute value v

schema drift를 fail이 아니라 warn으로 둔 이유 데이터 파이프라인에서 source schema가 바뀌는 순간은 애매하다. 무조건 무시하면 운영자는 입력 구조가 바뀐 사실을 모른다. 반대로 모든 schema 변화를 실패로 처리하면, 정상적인 컬럼 추가까지 daily run을 막아버린다. manufacturing-data-platform-mini 에서는 이 문제를 작게 다뤘다. synthetic manufacturing CSV의 실제 header를 기준으로 schema_hash 를 만들고, previous successful run과 비교해 달라졌으면 schema_drift quality check를 warn 으로 남긴다. 단, required column이 빠져 silver/gold contract를 만들 수 없는 경우는 현재 ValueError 로 빠르게 실패한다. 1. Scenario 어느 날 source CSV에 새 컬럼이 추가된다. 기존 header: event_time,plant_id,line_id,work_order_id,machine_id,product_code, operation,units_produced,defect_count,cycle_time_ms,business_date 새 header: event_time,plant_id,line_id,work_order_id,machine_id,product_code, operation,units_produced,defect_count,cycle_time_ms,business_date,operator_id operator_id 는 아직 silver/gold mart에서 쓰지 않는다. 하지만 source 구조가 바뀐 사실은 기록되어야 한다. 2. Decision Pressure schema drift에서 중요한 질문은 단순히 "바뀌었나?"가 아니다. 바뀐 것을 운영자가 알 수 있는가? 정상적인 컬럼 추가 때문에 pipeline을 멈춰야 하는가? downstream gold mart contract가 조용히 바뀌지는 않는가? 이전 successful run과 지금 run의 schema identity를 비교할 수 있는가? 초기 구현에서는 한 가지 실제 버그가 있었다. schema_hash 가 고정된 required column 목록에 너무 묶여 있어서, 추가 컬럼이 들어와도 hash가 바뀌지 않았다. 즉 operator_id 가 추가되어도 drift가 보이지 않았다. 이 문제를 고치기 위해 read_rows 가 실제 CSV header를 반환하고, 그 실제 header 기준으로 schema_hash 를 계산하도록 바꿨다. 3. Options option result risk ignore drift pipeline은 계속 돈다 source 변화가 보이지 않음 fail every drift 변화에 강하게 반응 정상적인 컬럼 추가도 막음 warn and continue 변화가 보이고 run도 계속됨 warning을 inspect해야 함 auto-evolve silver/gold 새 컬럼을 바로 사용 가능 downstream contract가 조용히 바뀔 수 있음 full schema registry production에 가까움 mini slice에는 무거움 이 프로젝트의 선택은 warn and continue 다. 4. Decision 현재 contract는 이렇다. previous successful run이 없으면: schema_drift = pass baseline schema established current schema_hash == previous successful schema_hash: schema_drift = pass current schema_hash != previous successful schema_hash: schema_drift = warn quality_passed는 true 유지 run/lineage record에 previous/current schema_hash 저장 required column missing: V

source_hash로 같은 입력 재처리를 안전하게 skip하기 작은 데이터 파이프라인도 한 번만 실행된다고 가정하면 금방 거짓말이 된다. 실제로는 같은 파일을 다시 실행할 수 있다. 실패한 run을 재시도할 수도 있고, 과거 날짜를 backfill할 수도 있고, 운영자가 실수로 같은 입력을 다시 넣을 수도 있다. 이때 결과가 중복되면 gold metric은 더 이상 믿을 수 없다. 이 글은 개인 포트폴리오 프로젝트 manufacturing-data-platform-mini 에서 source_hash 를 이용해 같은 입력 재처리를 안전하게 skip하도록 만든 작은 설계 판단을 정리한 글이다. 데이터는 모두 synthetic이며, production platform이 아니라 검증 가능한 mini slice다. 1. Scenario 같은 business_date 의 제조/로봇 이벤트 파일을 다시 처리해야 하는 상황이 있다. 예: retry: 앞 run이 중간에 실패해서 다시 실행한다. backfill: 과거 날짜를 다시 채운다. operator mistake: 같은 파일을 실수로 다시 실행한다. 단순히 매번 append하면 같은 날짜의 gold metric이 중복될 수 있다. 2. Decision Pressure 단순 CSV pipeline은 보통 이렇게 끝난다. CSV 읽기 -> silver 만들기 -> gold 집계 -> 결과 저장 하지만 운영 관점에서는 질문이 생긴다. 이 입력은 전에 처리한 파일과 같은가? 같은 파일을 다시 돌리면 중복 output이 생기나? 다른 파일로 같은 날짜를 다시 돌리면 어떻게 구분하나? 어떤 run이 어떤 source에서 만들어졌나? 그래서 재실행을 판단할 identity가 필요했다. 3. Options option result problem always append 모든 run 결과를 계속 추가 같은 입력 재실행 시 중복 always overwrite 결과를 항상 덮어씀 이전 결과/원인 추적이 약함 skip by business_date only 같은 날짜면 무조건 skip 정정 파일을 반영할 수 없음 skip by dataset_id + business_date + source_hash 같은 입력만 no-op 정정 파일은 새 run으로 처리 가능 이 프로젝트의 Slice1은 마지막 선택지를 쓴다. 4. Decision 현재 mini pipeline은 입력 파일의 content hash를 source_hash 로 계산한다. idempotency key: dataset_id + business_date + source_hash 이미 성공한 run이 있으면 새로 처리하지 않고 기존 run을 재사용한다. same dataset_id same business_date same source_hash prior successful run exists => status = skipped 이 선택은 작지만 중요하다. 같은 파일 재실행: skip -> 중복 없음 같은 날짜의 정정 파일: source_hash가 다름 -> skip하지 않고 새 run으로 처리 가능 단, 여기서 조심해야 할 경계가 있다. Slice1은 다른 source_hash 를 새 run으로 처리할 수 있지만, 이전 gold partition을 원자적으로 교체하는 Iceberg-style overwrite까지 구현한 것은 아니다. 그 문제는 다음 Slice2의 business_date partition overwrite 주제다. 5. Evidence 관련 코드와 검증 evidence: src/manufacturing_data_platform/pipeline/lakehouse.py tests/test_lakehouse_pipeline.py VERIFICATION_LOG.md README.md 검증 로그: 2026-07-08 publication readiness check: pytest: 33 passed lakehouse JSON CLI: passed, status=processed, quality_passed=true EAV JS