标签：#Python

I game in a room that warms up fast. I could see CPU usage in Task Manager and watts in HWiNFO if I went looking. What I actually wanted was simpler: How much heat is this machine putting into the air right now? Not in a spreadsheet. In plain language I could glance at while the PC was running. The gap Lots of tools show watts and temperatures . Almost none answer room heat : BTU per hour Heat accumulated over a session Plain context like "about a quarter of a space heater" With ambient temp: still-air rise or rough exhaust CFM The conversion is straightforward ( BTU/hr ≈ watts × 3.412 ), but I didn't want to do it in my head every time. So I built HeatLens — a small desktop widget built around room heat, not raw sensor dumps. What HeatLens shows Total wattage — what the PC is drawing now Heat dissipation — BTU/hr or kW Session heat — BTU or kWh since launch Max temperature — hottest live sensor Trend graphs — watts, heat, and temp over time CFM estimate — with ambient temp: rough exhaust airflow for a +10 °F rise Still-air rise — how fast a reference room would warm with no ventilation Estimated power is labeled separately from measured sensors. Where the data comes from LibreHardwareMonitor / Open Hardware Monitor (HTTP + WMI on Windows) nvidia-smi for NVIDIA GPUs Linux RAPL / hwmon when exposed by the kernel Labeled fallbacks when direct power sensors aren't available On Windows, best results: LibreHardwareMonitor with Remote Web Server on port 8085 . What it is not HeatLens is not a replacement for a Kill-A-Watt at the wall. Software usually can't see monitor power, full PSU loss, or every platform rail. A plug-in meter is still the most accurate whole-system reading. HeatLens is for context : "~400 W gaming → ~1,400 BTU/hr into the room" Session heat over an hour or two Rough CFM / still-air numbers as sanity checks — not duct design Things I learned building it Sensor coverage is messy. Different backends, missing rails, and estimates that need clear labeling.

Let's be real: our personal medical history is a mess. It’s a chaotic mix of PDF lab results, grainy scans of prescriptions, and cryptic Electronic Medical Records (EMR) scattered across different hospital portals. If you’ve ever tried to remember exactly when a specific symptom started or how your cholesterol has trended over the last decade, you know the "search" struggle is real. In this guide, we are moving beyond simple folders. We are architecting a Personal Health Knowledge Base using a modern Vector Database and RAG (Retrieval-Augmented Generation) pipeline. We’ll leverage Qdrant for high-performance similarity search, Unstructured.io for complex document parsing, and Sentence-Transformers to turn 10 years of medical jargon into searchable embeddings. By the end of this post, you'll have a system capable of cross-year symptom correlation and instant medical history retrieval. The Architecture: From Pixels to Insights 🏗️ The biggest challenge with medical records isn't storage; it's ingestion . Medical PDFs are notoriously difficult to parse because they often contain nested tables and checkboxes. Our pipeline handles this by isolating the layout before embedding. graph TD A[Raw Medical Data: PDFs, Scans, EMRs] --> B[Unstructured.io: Partitioning & OCR] B --> C[Text Chunking & Cleaning] C --> D[Sentence-Transformers: Vector Embedding] D --> E[(Qdrant Vector DB)] F[User Query: 'Show me my blood sugar trends since 2015'] --> G[FastAPI Interface] G --> H[Query Embedding] H --> I[Vector Search in Qdrant] I --> J[Contextual Results + LLM Synthesis] J --> K[Actionable Health Insight] Prerequisites 🛠️ To follow along, you'll need: Python 3.9+ Unstructured.io : For the heavy lifting of PDF/Image parsing. Qdrant : Our vector engine (run it via Docker: docker run -p 6333:6333 qdrant/qdrant ). Sentence-Transformers : To generate local embeddings without sending sensitive data to the cloud. FastAPI : To wrap it all in a slick API. Step 1: Parsing the Chaos with Unstructu

GitHub热门项目 | The official Python library for the OpenAI API | Stars: 31,211 | 92 stars today | 语言: Python

GitHub热门项目 | CLI tool for configuring and monitoring Claude Code | Stars: 28,654 | 104 stars today | 语言: Python

Builder Journal · ARC Prize 2026 There is a moment in every role-playing game where you load your character with so much heavy gear that they can barely walk. Strongest sword in the game, can't reach the fight. I did the machine-learning version of that this month. I kept making my agent more capable, and the scoreboard kept punishing me for it, and it took me two tries to understand that the upgrades were the problem. A quick frame, in case this is your first entry in this thread : I'm in the ARC Prize 2026, building an agent that has to learn small games it has never seen, with no instructions. As the benchmark's creator measured it, the hardest part by far is the piece that figures out the rules of a game by experimenting on it. So that piece is where I have been pouring my effort. The obvious upgrade The obvious way to make that piece better is to teach it more kinds of games. If it can model three families of puzzle today, teach it a fourth, and it should win more. So I did exactly that. I built support for a new class of game it could recognize and solve, wrote it carefully, tested it, and confirmed the thing I wanted to confirm: the agent now beat a game it provably could not beat the day before. Real, verified, new capability. Not a story I was telling myself, a genuine new skill on the board. Then I submitted, and the score went down. Twice This is the part I want to be honest about, because one bad result is noise and two is a pattern. My agent's attempts to use this theory-building component had already been underwhelming on the real board, landing around 0.05, 0.07, and 0.09 across earlier tries, all of them under the 0.25 my plain, careful agent scores when it does not reach for the fancy component at all. The fourth skill was supposed to turn that corner. Instead the next submission came in at 0.04, the worst of the lot. I had added ability and the number had dropped, again. So I stopped adding and started counting. I ran a survey across twenty-five of

BDE Score™ — Open-Source Multi-Factor Stock Analysis One number. 0-100. Every stock. A composite score combining 5 dimensions: Momentum (30%), Volatility (25%), Volume (20%), Trend (15%), Risk (10%). Coverage : 74 stocks across US (25), Hong Kong (26), and A-Share China (23) markets — all in real-time. Why It's Different Zero signup — REST API works without authentication Multi-market — US, HK and A-Share coverage Transparent scoring — Every factor weight is documented Open source — Full methodology on GitHub Real-time badges — Embed live scores in any README Quick Start curl "https://atlantic-remains-atomic-floor.trycloudflare.com/api/analyze?market=ALL" Links GitHub: https://github.com/hbhqq9/bde-score Live Demo: https://atlantic-remains-atomic-floor.trycloudflare.com/api/snapshot?market=ALL Not financial advice. Technical service for educational purposes. ⭐ Star us on GitHub!

When you first step into the world of embedded systems, one of the earliest and most consequential decisions you will face is choosing a programming language. Two names come up more than any others: Python and C++. Both are powerful, both have passionate communities, and both are genuinely useful — but for very different reasons and in very different contexts. This article is not about declaring a winner. It is about understanding why each language exists in this space, what trade-offs you are actually making, and how to make a confident, informed decision for your next project. Understanding the Fundamental Difference Before comparing features, it helps to understand why these two languages feel so different at a deeper level. C++ is a compiled, statically-typed, systems-level language . When you write C++, you are writing code that gets translated directly into machine instructions. You manage memory manually. You control exactly when objects are created and destroyed. The hardware does precisely what you tell it to, nothing more and nothing less. This directness is both its superpower and its source of complexity. Python, by contrast, is an interpreted, dynamically-typed, high-level language . A Python runtime sits between your code and the hardware, managing memory automatically through garbage collection, resolving types at runtime, and handling a lot of bookkeeping so you don't have to. This makes Python wonderfully expressive and fast to write, but it introduces overhead that matters enormously on constrained hardware. The mental model to hold onto is this: C++ gives you control, Python gives you speed of development . Both are valuable. The question is which one your project needs more. Where C++ Shines in Embedded Systems 1. Bare-Metal and Resource-Constrained Environments If you are programming a microcontroller like an STM32, an AVR ATmega, or an ESP32 running its native SDK, C++ is almost always your primary language. These devices often have kilobytes —

GitHub热门项目 | YouTube到AcFun和bilibili自动化搬运工具，支持AI翻译、字幕生成、内容审核、智能监控 | Stars: 1,535 | 211 stars today | 语言: Python

If you’ve built AI applications in production recently, you’ve probably hit the "Agent Wall." You build a ReAct agent, give it 10 granular tools (search, extract, route, format), a massive system prompt, and tell it to go to work. It feels like magic...until you look at your latency metrics and token bills. Today’s agents act as interpreters. They re-derive the exact same routines from scratch on every single request . They embed massive tool schemas and reasoning histories into every loop. It's slow, it's incredibly token-hungry, and occasionally, they hallucinate tool calls, drop constraints, or get stuck in endless reasoning loops. In a production environment, even occasional errors can be critical failures that waste time and tokens. The problem isn't the ReAct pattern itself. The problem is that we are forcing the LLM to orchestrate low-level, predictable logic that should be deterministic code. We got tired of paying the "reasoning tax" for sub-routines that don't need it. So, we built Sparsi —a framework for shifting complex logic out of your ReAct agent's prompt and into deterministic "Macro-Tools" built as DAGs (Directed Acyclic Graphs). The Macro-Tool Pattern There are two ways to use Sparsi: as an end-to-end solution for a specific task, or to create higher-level tools that plug into your existing agents. The latter is where the magic happens. Instead of giving your ReAct agent 10 tiny, flaky tools and hoping it chains them correctly, you build one highly reliable, deterministic Sparsi DAG to handle that specific sub-routine. You then expose that DAG to your agent as a single Model Context Protocol (MCP) tool. The overall agent still drives the conversation, but it delegates the heavy lifting to a reliable macro-tool. We chose the DAG architecture for three main reasons: Deterministic & Testable: The graph is made of plain code. You only run AI where natural language understanding is strictly required. Parallel by Architecture: Independent branches run co

When a CLI becomes useful, someone eventually tries to automate it. That is where a progress bar can quietly become a problem. For a person, this kind of output is helpful: Translating markdown files 12/40 30% docs/intro.md It tells me that the command is alive, how far it has moved, and which file it is working on. But when another system starts reading that same output, the progress bar stops being only a user interface. It becomes an accidental contract. That was the real problem behind one of the changes in Co-op Translator v0.20.0. The release added a Rich-powered CLI progress UI, but it also added structured translation events. At first, those may look like two separate improvements. They are really two surfaces for the same state: Rich output gives a person something readable, while structured events give integrations something stable. This article is about three things: First, why console output is tempting to parse. Second, how Co-op Translator separated the Rich UI from the event stream. Third, why that separation matters for CLI, Python API, MCP, and product integrations such as Localizeflow. The problem appears when logs become state Console output is written for people. When I run a translation command, I want a quick answer to a few practical questions: Is the command still running? Which stage is active? Which file is being processed? How much work is left? Did anything fail? A progress bar is good for that. It compresses the state of the run into something I can scan quickly. But a product integration needs a different kind of information. Imagine Localizeflow running Co-op Translator as part of a larger workflow. It does not only need to know that text was printed. It needs durable state: Which translation job started Which target language is active Which stage is running Which file completed Which file failed How many items are done Whether the run succeeded If all of that only exists inside console text, the integration has to parse human language

When you're running maintenance across several WordPress sites in sequence, a list view with text-only status doesn't make "which site is being processed now" or "which ones are already done" easy to spot at a glance. A client put it plainly: " Make it visually obvious in the list which sites are in maintenance and which are finished. " A colored border is the obvious move, but there are real choices to make. What colors? Where do we get the state from? When does the "done" mark go away? And — can we ship this without touching the backend? This post walks through those four calls and the minimal frontend-only implementation we landed on. Color picking — "red flashing" was the first thing we ruled out How do you make the running site stand out? The intuitive answer is "blinking red," but that got cut early. Multi-site maintenance runs are long . Having something blink red somewhere on screen the whole time is a fatigue source. We went with "a gentle blue pulse + a solid green border" instead: Running : blue #2563eb border + a soft pulsing box-shadow (2.2s ease-in-out) Done (within 24h) : green #10b981 solid border + a faint inset shadow @keyframes site-running-pulse { 0 %, 100 % { box-shadow : 0 0 0 0 rgba ( 37 , 99 , 235 , 0.4 ); } 50 % { box-shadow : 0 0 0 6px rgba ( 37 , 99 , 235 , 0 ); } } .site-running { border-color : #2563eb !important ; animation : site-running-pulse 2.2s ease-in-out infinite ; } @media ( prefers-reduced-motion : reduce ) { .site-running { animation : none ; } /* respect OS-level reduced motion */ } .site-completed { border-color : #10b981 !important ; box-shadow : inset 0 0 0 1px rgba ( 16 , 185 , 129 , 0.25 ); } The prefers-reduced-motion: reduce rule stops the pulse for users who have reduced-motion enabled at the OS level (often people with vestibular sensitivity). If you're adding motion to grab attention, this is essentially required. Zero backend changes — reuse the existing log stream To tell the list UI "this site is being processed

Writing changelogs is one of those developer tasks that sounds simple until you are staring at a messy commit history. Some commits matter to users. Some are internal cleanup. Some are merge commits. Some are meaningful only if you already know the codebase. I built a small Python example that turns commit messages or git diffs into structured changelog JSON using Telnyx AI Inference. Code: https://github.com/team-telnyx/telnyx-code-examples/tree/main/changelog-generator-python What it does The Flask app exposes: POST /generate POST /generate/from-diff GET /changelogs GET /changelogs/<id> GET /health POST /generate accepts a list of commit messages: { "version" : "v1.4.0" , "repo_name" : "billing-service" , "commits" : [ "feat: add Stripe webhook retry with exponential backoff" , "fix: correct tax calculation for EU VAT exemption" , "docs: update API reference for invoice endpoint" ] } The app asks Telnyx AI Inference to return grouped changelog JSON with sections like: Features Bug Fixes Improvements Breaking Changes Documentation Other There is also a POST /generate/from-diff endpoint if you want to summarize a git diff instead of commit messages. Why structured output matters For a changelog tool, plain text is useful, but structured output is more flexible. If the response comes back as JSON, you can: render it in a docs site save it in a release database post it into a PR comment send it to Slack open a release-note review workflow let a human approve it before publishing The example stores generated changelogs in memory and gives each one an ID, so you can list recent changelogs or retrieve a specific one. Run it Clone the examples repo: git clone https://github.com/team-telnyx/telnyx-code-examples.git cd telnyx-code-examples/changelog-generator-python Create your .env file: cp .env.example .env Add your Telnyx API key: TELNYX_API_KEY=your_telnyx_api_key AI_MODEL=moonshotai/Kimi-K2.6 HOST=127.0.0.1 Install and run: pip install -r requirements.txt python app.py

Ingeniería de Datos aplicada a la Biodescodificación: Presentando Bio-Mapping Engine 🧬 ¿Es posible aplicar ingeniería de datos de alta fidelidad a campos de conocimiento no estructurados? La respuesta es un rotundo sí. Hoy quiero presentarles Bio-Mapping Engine , un framework diseñado para resolver un problema clásico de la extracción de información: convertir literatura densa y desorganizada en una base de conocimientos semántica, estructurada y totalmente navegable. El Problema: El caos de la información no estructurada En campos como la Biodescodificación , la información suele residir en libros o archivos PDF donde los conceptos (síntomas, emociones, zonas anatómicas) están entrelazados de forma narrativa. Para un investigador o un desarrollador de herramientas de salud alternativa, extraer relaciones precisas entre un síntoma físico y su conflicto emocional mediante métodos tradicionales es una tarea manual, lenta y extremadamente propensa a errores. La Solución: Bio-Mapping Engine Bio-Mapping Engine no es un simple scraper . Es un motor de segmentación semántica y mapeo topológico. Su propósito es transformar un PDF bruto en un grafo de conocimiento estructurado en formato JSON, permitiendo realizar consultas multidimensionales con precisión quirúrgica. 🚀 Características Principales Segmentación Semántica Avanzada: Implementa un parsing topológico que distingue inteligentemente entre encabezados de síntomas, contenido emocional y el "ruido" estructural (como índices o números de página). Mapeo Relacional Multidimensional: Realiza una extracción de alta fidelidad a través de tres vectores fundamentales: Síntomas Canónicos: Estandarización de la nomenclatura de síntomas y condiciones. Jerarquía Anatómica: Mapeo inteligente que escala desde Sistemas $\rightarrow$ Regiones $\rightarrow$ Órganos. Arquetipos Emocionales: Extracción estructurada de modelos mentales y conflictos (ej. "Causa probable" , "Bloqueo emocional" ). Consultas Multi-Eje (CLI): Una potente inte

RQ (Redis Queue) is a delightfully simple way to run background jobs in Python. That simplicity is also why teams under-monitor it: it just works, until a downstream API gets slow or a bad deploy ships, and jobs start failing in bulk — quietly. Here's what to watch and how to get alerted before a customer tells you. RQ failures don't announce themselves When a job raises, RQ moves it to the FailedJobRegistry and moves on. The worker keeps running; nothing crashes. If you're not looking at that registry, the failure is invisible — the same trap BullMQ, Celery, and every robust queue share. So the job is to reach into the queue's state and turn it into a signal. The four signals that matter for RQ Failure count / rate — jobs landing in the FailedJobRegistry over a window. Backlog — how many jobs are queued vs. being worked; is the worker keeping up? Latency — how long jobs take, and how long they wait before a worker picks them up. Worker liveness — are your workers actually alive and heartbeating? Where to read them RQ exposes queue and registry state directly: from redis import Redis from rq import Queue from rq.registry import FailedJobRegistry , StartedJobRegistry redis = Redis () q = Queue ( " default " , connection = redis ) queued = len ( q ) # backlog failed = FailedJobRegistry ( queue = q ) # failures started = StartedJobRegistry ( queue = q ) # in-flight print ( " queued: " , queued ) print ( " failed: " , len ( failed )) print ( " started: " , len ( started )) Poll this on an interval and store the series — a single snapshot hides the trend , which is the part that matters. For failures specifically, walk the registry to get the actual exceptions: for job_id in failed . get_job_ids (): job = q . fetch_job ( job_id ) print ( job . id , job . exc_info . splitlines ()[ - 1 ] if job . exc_info else "" ) Two gotchas: Group by exception, not by job. A thousand jobs failing with the same traceback is one incident. Normalize the message (strip IDs, timestamps, host

A lot of Robot Framework projects still look like plain Selenium scripts with .robot file extensions. Someone imports webdriver , creates driver = webdriver.Chrome() , then calls find_element and send_keys in Python helpers. Robot Framework runs the suite, but readable keywords, shared libraries, and consistent waits never show up in the tests. If you already use Robot Framework with SeleniumLibrary , you do not need the raw WebDriver API. SeleniumLibrary gives you high-level keywords. The Page Object Model gives you structure. Together they keep tests short and UI changes localized. We published a small MIT template that shows the layout: rf-seleniumlibrary-pageobject-template . It targets Sauce Demo — clone it, run four tests, fork the folder structure. What breaks when you mix in raw WebDriver driver = webdriver . Chrome () driver . find_element ( By . ID , " user-name " ). send_keys ( " standard_user " ) driver . find_element ( By . ID , " password " ). send_keys ( " secret_sauce " ) driver . find_element ( By . ID , " login-button " ). click () Fine for a script. Painful in a growing suite. Locators spread across helpers and test files. Waits become time.sleep(2) in one place and missing in another. You end up maintaining SeleniumLibrary and a parallel WebDriver stack. CI fails on a Tuesday night and you are not sure which path opened the browser. Before and after Before After driver.find_element(...).send_keys(...) Login With Valid Credentials ${VALID_USER} ${VALID_PASSWORD} Locators in every file LoginLocators.USERNAME in one module Ad-hoc sleeps wait_until_element_is_visible in BasePage.click() Two browser stacks One SeleniumLibrary instance per suite Four layers Layer Job Example Locators Selectors per screen login_locators.py BasePage Shared waits and actions click() , enter_text() Page library Screen keywords LoginPage.login() Robot test Scenario only Inventory Should Be Visible Folder layout in the repo: resources/locators/ → selectors pages/ → Python pa

A fintech RAG pipeline poisoned its own vector store and the LLM-as-a-judge validator approved every hallucination. The fix: gate writes with code.

GitHub热门项目 | Open-source intelligence for the global theater. Track everything from the corporate/private jets of the wealthy, and spy satellites, to seismic events in one unified interface. Hook an AI agent up to have it parse through data and find previously unseen correlations. The knowledge is available to all but rarely aggregated in the open, until now. | Stars: 9,650 | 30 stars today | 语言: Python

Filingrail now ships a first-party MCP server on PyPI: pip install filingrail-mcp . One install, one config block, and Claude Desktop — or Cursor, or Continue, or any MCP-compatible client — can query SEC filings as tools. No glue code. That's worth naming directly. Most SEC-data APIs ship a REST endpoint and stop. You write the agent integration yourself: parse the response, wire up the tool schema, handle auth headers. Filingrail ships the integration as a maintained package with the same update cadence as the underlying REST API. This post covers the setup, what you can ask once it's wired in, and the honest limits. I built both the API and the MCP server — I'll be upfront about that throughout. This post covers a data API that returns SEC-registered financial information. Nothing here is investment advice. Two ways to wire it in Option 1 — pip install filingrail-mcp (recommended) Install the package, add one block to your Claude Desktop config, restart. Filingrail's endpoints appear as tools. No separate service to run, no background daemon. Option 2 — RapidAPI MCP Playground tab (no local install) The Filingrail listing on RapidAPI has an MCP tab that generates a ready-to-paste config block. Same endpoints, same auth, zero install step. Either path gives Claude the same tools. Pick the one that fits your setup. Setup — the pip install path You'll need Python 3.10+ and a RapidAPI key. 1. Subscribe to Filingrail Go to the Filingrail RapidAPI listing and subscribe. Free tier is 50 calls/day, no credit card. Copy your X-RapidAPI-Key from the RapidAPI dashboard. 2. Install the server pip install filingrail-mcp 3. Add Filingrail to your Claude Desktop config On macOS: ~/Library/Application Support/Claude/claude_desktop_config.json On Windows: %APPDATA%\Claude\claude_desktop_config.json { "mcpServers" : { "filingrail" : { "command" : "filingrail-mcp" , "env" : { "RAPIDAPI_KEY" : "your_rapidapi_key_here" } } } } 4. Restart Claude Desktop Filingrail's endpoints appear a

用 FROST 五维元模型构建可治理的多 Agent 系统：从零到一的代码教程 作者 ：FROST Team 日期 ：2026-07-09 主题 ：代码教程 | 周四轮换 项目 ：FROST + FROST-SOP 前言 2026 年，Agent 框架百花齐放——LangChain、CrewAI、AutoGen 各有各的好。但它们都有一个共同的盲区： 治理能力 。 当你的 Agent 系统从 1 个变成 10 个，从跑 Demo 变成跑生产，你会发现： 谁有权做什么？没有答案 这个决策是谁做的？无法追溯 Agent 越权了怎么办？事后才能发现 FROST（Fractal Runtime of Orchestrated Skills & Tasks）的 五维元模型 就是为解决这些问题而设计的。 今天这篇教程，我们用代码从零构建一个完整的多 Agent 治理系统。 FROST （教学框架）提供理论基础 → Gitee 仓库 FROST-SOP （工程平台）提供工程落地 → Gitee 仓库 一、五维元模型是什么？ FROST V4.0 引入了五个核心维度，每个维度解决 Agent 治理的一个关键问题： 维度 模块 解决的问题 类比 武器 Armory Agent 有哪些能力？ 武器库清单 任务 TaskRegistry 工作如何编排？ 作战计划 事件 EventCatalog 发生了什么？ 战场态势 平台 PlatformRegistry 外部资源在哪？ 后勤补给 规则 RuleRegistry 什么能做/不能做？ 交战规则 五个维度 各自独立又相互咬合 ——就像五角星的五个角，缺一个就不完整。 二、环境搭建 # 克隆 FROST 教学框架 git clone https://gitee.com/liao_liang_7514/frost.git cd frost # 安装依赖 pip install -r requirements.txt # 验证环境 python -m pytest test_core.py -v FROST 的设计哲学是 零外部依赖 ——核心只需要 Python 标准库。五维元模型的模块也遵循这个原则。 三、维度一：Armory（武器注册表） Armory 管理 Agent 所有能力的元数据。不是简单的方法注册，而是带元信息的 能力目录 。 from core.armory import Armory , SkillMetadata # 创建武器库 armory = Armory () # 注册一个技能（带完整元数据） armory . register ( SkillMetadata ( name = " summarize_text " , category = " nlp " , description = " 将长文本压缩为摘要 " , input_schema = { " text " : " string " , " max_length " : " int " }, output_schema = { " summary " : " string " }, cost_estimate = 0.002 , # 每次调用预估成本 latency_ms = 500 , # 预估延迟 tags = [ " summarization " , " compression " ] ) ) armory . register ( SkillMetadata ( name = " search_web " , category = " retrieval " , description = " 联网搜索获取最新信息 " , input_schema = { " query " : " string " , " top_k " : " int " }, output_schema = { " results " : " list[dict] " }, cost_estimate = 0.01 , latency_ms = 2000 , tags = [ " search " , " real-time " ] ) ) # 发现可用技能 nlp_skills = armory . discover ( category = " nlp " ) print ( f " NLP 技能: { [ s . name for s in nlp_skills ] } " ) # 输出: NLP 技能: ['summarize_text'] # 按标签发现 search_skills = armory . discover ( tags = [ " real-time " ]) print ( f " 实时技能: { [ s

Pytest Fundamentals for Data Engineers Data engineering code is still software, and software that doesn't have tests is a liability. If you've ever had a production pipeline silently produce wrong numbers because a downstream transformation made an assumption about a column that changed upstream, you know the pain. Testing isn't just about "does the pipeline run" — it's about verifying that every transformation, every branch of logic, and every edge case behaves exactly as you expect. This post lays the groundwork for testing data pipelines with pytest. I'll cover why testing matters, the fundamental pattern every test follows, how to write your first tests and assertions, how to verify that your code raises the right exceptions, and how to mock external dependencies so your tests stay fast and deterministic. By the end, you'll have a solid foundation for writing unit tests that catch regressions before they reach production. Testing 101 for Data Pipelines If you're new to testing or have only ever written a few scripts, the core idea is simple: a test is a small piece of code that exercises another piece of code and checks that the result is what you expected. In pytest, that looks like a function whose name starts with test_ and which contains an assert statement. The Fundamental Pattern: Arrange, Act, Assert Almost every test follows three steps: Arrange – set up the input data and any necessary context. Act – call the function or method you want to test. Assert – verify the output is correct. For a data pipeline transformation, that might be: def test_clean_amount (): # Arrange raw_data = " 1,200.50 " # what you might get from a CSV # Act result = clean_amount ( raw_data ) # Assert assert result == 1200.50 clean_amount is a pure function — no file I/O, no database calls. It just takes an input string and returns a number. That makes it straightforward to test. Assert Raises: Expecting Exceptions Data pipelines often deal with invalid data that should trigger an