标签：#Python

GitHub热门项目 | holehe allows you to check if the mail is used on different sites like twitter, instagram and will retrieve information on sites with the forgotten password function. | Stars: 12,201 | 166 stars today | 语言: Python

I fixed this script's frontmatter parser a week ago. A draft with an unclosed --- block used to blow up with a bare ValueError: not enough values to unpack , and I patched it to raise a clean, actionable error instead. I wrote that fix up, verified it with a stubbed repro, added a --selftest case for it, called it done. Then I went back to write today's articles and actually looked at the line I "fixed" instead of the error path around it. def parse ( text ): meta = {} body = text if text . lstrip (). startswith ( " --- " ): parts = text . lstrip (). split ( " --- " , 2 ) if len ( parts ) < 3 : raise ValueError ( " frontmatter opened with ' --- ' but never closed with a second ' --- ' delimiter " ) _ , fm , body = parts ... split("---", 2) doesn't split on lines that are --- . It splits on the literal substring "---" , anywhere in the text, and stops after the second one it finds. My fix only handles the case where it finds fewer than two — an unclosed fence. It says nothing about what happens when the second "---" it finds isn't the closing fence at all, because a third one showed up first, buried inside a frontmatter value. That's not a hypothetical. I write these article titles myself, and "before/after" is a phrase I reach for constantly: --- title : My Before---After Refactor tags : ai, python, refactor published : true --- real body starts here split("---", 2) finds the em-dash-style --- inside the title before it finds the real closing fence on its own line. So the split points land in the wrong place entirely: >>> from publish_devto import parse >>> meta , body = parse ( text ) >>> meta { ' title ' : ' My Before ' } >>> body ' After Refactor \n tags: ai, python, refactor \n published: true \n --- \n real body starts here \n ' The title got truncated to "My Before" . tags and published never got parsed as frontmatter fields at all — they're sitting in the body now, as literal text, along with the real closing fence and a stray leftover --- . If I ran this thr

Back in early August I fixed a bug in update_article , one of the tools in this repo's DEV.to MCP server. The bug was straightforward: the tool built its PUT payload from three optional parameters, and if a caller passed none of them, it still fired a GET and a PUT with an empty {"article": {}} body against a live published post, then logged a no-op entry to the audit trail as if something had happened. The fix was a guard: raise before either network call if the built payload dict ends up empty. article = {} if title is not None : article [ " title " ] = title if body_markdown is not None : article [ " body_markdown " ] = body_markdown if published is not None : article [ " published " ] = published if not article : raise ValueError ( " update_article called with no fields to update " " (title/body_markdown/published all None) " ) before = _dev ( f " /articles/ { article_id } " ) result = _dev ( f " /articles/ { article_id } " , method = " PUT " , data = { " article " : article }) _log_article_update ( article_id , before , article . keys (), result ) I closed the ticket, ran a stubbed selftest, moved on. Going back into this function for something unrelated, I noticed the guard only ever asks one question: did the caller pass a field? It never asks the question that actually matters for a tool whose whole job is writing to a live post: would this field's value be different from what's already there? Walk through what happens if a caller — an agent that re-reads an article's current title before deciding whether to touch it, gets it slightly wrong, or just calls the tool defensively with the value it already has — passes title="Same Title It Already Has" , and that string is in fact identical to the article's current title. article isn't empty. It has one key. The guard passes clean. Both network calls fire: before = _dev ( f " /articles/ { article_id } " ) # GET, real call result = _dev ( f " /articles/ { article_id } " , method = " PUT " , data = { " article " :

Every few weeks a new checkpoint drops and the timeline fills up with claims that it's cheaper, smarter, and about to change everything. Some of those claims hold up. Many don't. And even when a model genuinely is better on public leaderboards, that tells you almost nothing about whether it's better on your codebase, your tasks, and your budget . I wrote previously about building a reproducible harness before wiring any model into your workflow. This article is the sequel nobody asked for but everybody needs: once you have a harness, how do you evaluate a steady stream of new models without spending a steady stream of money? The answer I keep coming back to is routing by task difficulty : don't run your whole eval suite against every candidate. Tier your tasks, send the cheap ones to cheap models, and reserve expensive runs for the cases that actually discriminate between models. The problem with "run everything against everything" If your eval suite has 60 tasks and a new model appears every two weeks, naive evaluation costs scale linearly forever. Worse, most of those runs are wasted signal: Easy tasks (rename a variable, write a docstring, fix an obvious off-by-one) are solved by almost every current model. Running a frontier-priced model on them tells you nothing. Medium tasks (implement a small feature against an existing test, refactor across two files) are where models actually diverge. Hard tasks (multi-file reasoning, subtle concurrency bugs, unfamiliar framework internals) discriminate strongly but are few — and they're where failures are expensive to verify. So the harness should spend its budget where the signal is. A concrete artifact: a tiered router in ~80 lines of Python Here's a minimal, runnable sketch. It assumes your eval tasks are JSON files with a tier field ( easy , medium , hard ) and a verify command you can execute (a test suite, a diff check, whatever your harness already uses). # router.py — tiered evaluation router (working sketch, adapt

Three weeks ago a page that had been pulling steady search traffic for over a year disappeared from Google. Not deranked, just gone. I only noticed by accident, about ten days later, while poking around Search Console for something unrelated. Ten days of a page earning nothing because nobody, including me, was watching. Some background: I'm a marketer. I run a small agency, I publish a lot of pages across a few sites, and my technical ceiling for the last decade has been editing HTML that someone else wrote. Our actual developers are busy with actual work, and "can you build me a thing that watches Google" is exactly the kind of request that dies in a backlog. Search Console does show you indexing problems. It shows them to people who log in and go looking. I have around 400 URLs I care about across three properties, and I was never going to check them by hand on any schedule more honest than "when something feels off." I'd been reading Claude Code posts on here for months as a spectator. The genre is usually a developer using it to move faster. I wanted to know what happens when someone who can't write the code at all uses it to start from zero. So I paid for a month and typed what I wanted in plain English. Version one lasted twenty minutes My first prompt was something like: check if these URLs are indexed in Google and tell me when one falls out. Claude cheerfully produced a script that ran a site: search for every URL and scraped the results page. It worked. For about twenty minutes. Then Google decided I was a robot, which was technically correct, and started serving captchas. Nobody warned me about this part of vibe coding: the model will build exactly what you asked for, including when what you asked for is against the rules and dies on contact with reality. It only mentioned that scraping Google results is a bad idea after I pasted the captcha error and asked why everything was broken. Then it apologized and told me what it could have said at the start: the

Your script crashes, and near the bottom of the traceback sits AttributeError: 'NoneType' object has no attribute 'name' . It reads like Python is being deliberately unhelpful — but it's actually telling you something precise. You just tried to use a variable that turned out to be None , and it's telling you exactly which one and where. The error isn't saying your program is fundamentally broken. It's saying: at this exact line, you reached for an attribute on a value that was None instead of the object you expected. That's a narrow claim, and once you know how to read it, tracking down why it was None is usually mechanical. What the error is actually telling you Take this code: class User : def __init__ ( self , id , name ): self . id = id self . name = name def find_user ( users , user_id ): for u in users : if u . id == user_id : return u return None user = find_user ( users , target_id ) print ( user . name ) # AttributeError: 'NoneType' object has no attribute 'name' Read the message in two parts. 'NoneType' object has no attribute 'name' tells you the object you called .name on wasn't a User — it was None . has no attribute 'name' tells you which access failed. Put together: whatever user was pointing to when you hit that line wasn't what you expected — it was nothing at all. The message never claims .name is the problem. .name is just where the crash became visible. The real question is one step earlier: why was user None ? Here, find_user() falls through its loop without a match and explicitly returns None — so either target_id is wrong, or that user genuinely isn't in the list yet. The fix, step by step Read the attribute name in the error ( 'name' here) — that tells you which line and which access failed, nothing more. Trace back to where the None value came from. Find the line that assigned, returned, or fetched it. Ask why it's None there , specifically. The most common causes: a lookup function that found nothing and returned None , a dict.get() call th

Background Registration confirmation emails were not reliably reaching users on Gmail and Outlook outside Japan — sometimes landing in spam, sometimes not arriving at all. Investigation pointed to a single root cause: the wpmm.jp domain had SPF and DKIM configured, but no DMARC record . What each of the three does SPF (Sender Policy Framework) declares in DNS which IP addresses are authorized to send mail for a domain. Receiving servers check the sending IP against the SPF record to confirm the source is legitimate. DKIM (DomainKeys Identified Mail) adds a cryptographic signature to the message headers and body. The receiving server looks up the public key in DNS and verifies that the message has not been tampered with and was signed by a party controlling that domain. DMARC (Domain-based Message Authentication, Reporting and Conformance) sits above both. It tells receiving servers what to do when SPF and DKIM alignment fails, and it collects aggregate reports about how mail from the domain is being treated. The key point is that SPF and DKIM are independent checks. Without DMARC, there is no single authoritative statement about how the alignment result should influence delivery decisions. Major providers including Gmail weigh the absence of DMARC when scoring incoming mail. Adding the DMARC record The following TXT record was added to the wpmm.jp DNS: _dmarc.wpmm.jp TXT "v=DMARC1; p=none; rua=mailto:info@wpmm.jp" p=none means "collect data, but do not reject or quarantine mail that fails alignment." Starting with p=reject or p=quarantine risks blocking legitimate mail if DKIM alignment turns out to be misconfigured somewhere. The safe approach is to start with p=none , monitor the reports, and tighten the policy gradually. rua=mailto:info@wpmm.jp sets the destination for aggregate reports. Google and other receivers periodically send XML summaries showing which mail passed or failed SPF/DKIM alignment. This moves visibility from passive (you notice when users compl

I spent four months tuning a custom weather ensemble. It was worse than guessing. The fix was not a better ensemble. It was admitting someone already built the right thing and giving it away for free. What I built and why it failed The original weather bot counted forecast members. It pulled raw output from four systems: GFS, AIGEFS, ECMWF IFS, and AIFS. Up to 164 individual simulations per contract. The logic was simple. If at least three of four systems agreed on direction, the bot traded. If they disagreed, it sat out. That sounds reasonable. It was not. I ran 112 settled trades through the system and scored the model with a Brier score. The model scored 0.2858. Predicting the historical base rate, with no model at all, scores 0.2439. Lower is better. My model was worse than making no prediction. The problem was not direction. Direction was right about 60 percent of the time. The problem was confidence. The model spread its probabilities 2.1 to 4.0 times too narrow. It was certain when it should have been uncertain. In prediction markets, confidence sizes your bets. A model that is too confident trades too big on the wrong calls. The confident wrong calls cost more than the confident right ones made. There was also a systematic temperature bias at the gridpoint level, peaking around seven degrees Fahrenheit. The model leaned warm in a way that was not in the data. It was in the model. What I should have done first Before building anything, I should have checked whether the thing I was building already existed in better form. NOAA publishes the National Blend of Models. It blends dozens of forecast systems and applies statistical post-processing no individual model can match. It produces calibrated, bias-corrected, station-level probabilistic temperature guidance. For exactly the stations Kalshi settles on. For free. The NBM already does what I was trying to do by hand. It corrects the biases I was measuring. It produces uncertainty ranges I was approximating with

You can prompt an agent for three hours and ship a 25,000-line pull request. Nobody on your team can tell you why it works — or why it breaks at 2 AM. The New Workflow It's 2026. You're the senior engineer on a mid-size product team. Your job has always been the person who catches the architecture mistakes before they compound — the one who notices that a Kafka dependency was grafted onto a read-heavy query, or that someone denormalized the database because it was faster than fixing the ORM. This morning, you open your inbox. There are seven pull requests. The first one is 24,506 lines added, 3,938 removed, with a description that reads: "Implemented user analytics pipeline with event streaming." You pull the branch. It runs. The tests pass. When you ask the author where the data flows, they send you a link to a Claude conversation. Somewhere in that 47-turn exchange, between confident architectural recommendations and polite apologies when the model changed its mind, is the design decision. You read all 47 turns. You still don't know why they chose Kafka. This is not a hypothetical. This is what the post-AI-productivity era looks like for teams that adopted coding agents without updating their engineering discipline. The speed limit has been removed. And the people who built their careers on being the speed limit are now obsolete. What Changed Before AI coding assistants, there was a natural throughput cap on software output. A senior engineer could review perhaps three meaningful pull requests per day. A team of ten could ship maybe fifteen high-quality merges per sprint. This cap wasn't arbitrary — it was enforced by the time required to actually understand what you were merging. AI changed the cost structure, not the review requirement. A developer armed with a capable agent can now produce 25,000 lines of code in a morning. The agent writes the code. The agent writes the tests. The agent writes the documentation. The agent even writes the PR description, which

If you work with library data, you work with MARC21 — the length-prefixed binary record format catalogues have run on since the 1960s, complete with a directory of field offsets, subfield delimiters, and a pre-Unicode character encoding called MARC-8 that needs a lookup table with thousands of entries to decode. In Python that problem is solved: pymarc is mature, complete, and pleasant to use. In Go it wasn't. gomarc is a port of pymarc to Go. It covers the binary MARC21 transmission format, MARC-8 to Unicode conversion, MARCXML, and MARC-in-JSON — and on real catalogue exports it runs 4x to 11x faster than the library it was ported from. go get github.com/beyto1974/gomarc@v0.1.0 It reads like pymarc If you know pymarc, you already know this API. Iterate records, pull the fields you want: reader := marc . NewReader ( f ) for { record , err := reader . Next () if errors . Is ( err , io . EOF ) { break } if err != nil { log . Println ( err ) // permissive: bad records are skipped, not fatal continue } title , _ := record . Title () fmt . Println ( title ) } Title , Author , ISBN , ISSN , Subjects , Publisher , PubYear and more are there as methods. For anything else, go at the tag and subfield directly: value , ok := record . Get ( "245" ) . Subfield ( "a" ) for _ , f := range record . GetFields ( "650" ) { fmt . Println ( f ) } Build records, modify them, write them back: record . Get ( "245" ) . SetSubfield ( "a" , "The Zombie Programmer : " ) writer := marc . NewWriter ( out ) writer . Write ( record ) And convert to the formats the rest of your stack can actually read — both use UTF-8 throughout instead of MARC-8, so standard tooling works: s , err := record . AsJSON () // MARC-in-JSON records , err := marc . ParseXML ( r ) // MARCXML Large MARCXML files stream one record at a time via marc.NewXMLReader rather than loading into memory. The numbers Two real catalogue exports — 138,076 records, 166 MB. AMD Ryzen 5 3600, Go 1.25.12, CPython 3.13.5, gomarc v0.1.0, pym

Automating Your Morning: A Daily Briefing Pipeline You Can Build You should not manually read news, emails, or Slack in the morning. The average knowledge worker loses 23 minutes to context switching between 8:00 AM and 9:30 AM, according to a 2023 RescueTime study. That is 92 hours per year—two full workweeks—spent on low-signal input. The fix is not "waking up earlier." The fix is building a passive briefing pipeline that compiles, ranks, and summarizes your information sources before you open your laptop. This article shows you the exact architecture, tools, and failure points, based on my own production setup running for 14 months. The Problem: Your Morning Input Is Unstructured Here is the chain of causality. You wake up and check three things: email, Slack/Teams, and newsfeeds. Each app is a separate silo with its own notification system. Each notification triggers a micro-decision: Is this urgent? Do I need to act? Should I forward this? That decision process is not free. A 2022 University of California Irvine study measured that after each interruption, it takes an average of 23 minutes to return to deep focus. But most people never return to deep focus in the morning—they just bounce between silos. The result is "reactive paralysis": you start your day by responding to others' priorities, not your own. And because each silo sorts by recency (not importance), you read a promotional email from your bank before a critical client update. Why Manual Curation Fails You might think, "I'll just spend 10 minutes skimming." Let me give you the math. If you receive 50 emails, 30 Slack messages, and 20 industry news headlines, that is 100 items. At 6 seconds each to decide relevance (not read), that is 10 minutes of pure triage. But you will read the interesting ones—that is a minimum of 45 minutes total. The deeper issue is recency bias . News apps show you the latest story, not the most important one. Email shows the newest sender, not the highest-value contact. With

reply_comments.py is the script that tells me which DEV.to comments still need a reply. It walks every comment tree on every article I've published and reports the ones I haven't answered yet. I've fixed two bugs in it already: needs_reply() used to think a thread was "handled" forever after a single reply, even if the other person followed up again, and a dedup check was keyed on the thread's root comment instead of whichever message actually needed the reply, so a second round of conversation went permanently invisible. Both fixes are in --selftest now, and both looked, from the outside, like they'd covered this file's tree-walking logic pretty thoroughly. They hadn't. Today I found a third bug in the same handful of functions, and it survives even with both prior fixes applied. What the existing code assumes Comments on DEV.to come back from the API as trees. A top-level comment has a children list, and each child can have children of its own. The function that decides whether a thread needs attention is needs_reply() , built on latest_message() : def latest_message ( comment ): """ The most recently created message anywhere in this comment ' s subtree. """ latest = comment for c in comment [ " children " ]: candidate = latest_message ( c ) if candidate [ " created_at " ] > latest [ " created_at " ]: latest = candidate return latest def needs_reply ( comment ): return latest_message ( comment )[ " user " ][ " username " ] != ME This walks the whole subtree and returns exactly one message: whichever one has the latest timestamp, anywhere in the tree. _pending_entry() (the function pending() actually calls) is built directly on top of that single answer — it checks whether the latest message needs a reply, and if so, returns one entry for the whole thread. That's a reasonable design if a thread only ever grows one message at a time: root comment, my reply, their follow-up, my reply, and so on. Every test case in this file's --selftest , and both of the earlier bug

开源项目从 0 到 1：我是怎么用 2 周业余时间搭建多平台发布系统的 起因：一个周三晚上的崩溃 故事开始于一个普通的周三晚上。 22:00，写完了一篇 Python 教程。22:35，还在复制粘贴第 6 个平台。 当时心里想的是： 「我写了 40 分钟文章，为什么还要花 30 分钟发布？」 作为一个程序员的本能反应——这个问题应该用代码解决。 Day 1-2：技术选型 动手之前花了两天调研。核心问题是： 怎么跟 9 个平台对话？ 经过调研发现，平台分三类： 类型 代表 方案 有公开 API 掘金、CSDN、Dev.to HTTP 请求直接调 有半公开 API 知乎、博客园 需要逆向签名（x-zse-96、X-Ca） 没有 API 小红书、抖音 只能 Playwright 浏览器模拟 技术栈决策： 后端: Python + FastAPI - 选 Python 因为 Playwright 的 Python 绑定最成熟 - 选 FastAPI 因为异步支持好，自带文档 前端: 原生 HTML/CSS/JS - 不需要 React/Vue。一个管理后台而已，原生 JS 完全够用 - 零 npm 依赖，部署简单 数据库: SQLite - 单文件，不需要装 MySQL/PostgreSQL - 个人工具，并发不是问题 浏览器插件: Chrome Extension Manifest V3 - 一键提取 Cookie，用户不用手动 F12 不做的事情： ❌ 不用 Docker（增加复杂度，个人部署不需要） ❌ 不用 Redis（SQLite 足够） ❌ 不用前端框架（过度设计） Day 3-5：搭建骨架 先搭最小可用版本： 一个 API + 一个页面 + 一个平台 。 选掘金作为第一个平台——它有公开 API，最简单。 # 核心抽象：每个平台一个 Adapter class BasePlatform : async def publish ( self , title , content , tags , credentials ) -> PublishResult : raise NotImplementedError async def fetch_stats ( self , post_id , credentials ) -> StatsResult : raise NotImplementedError 这个阶段的关键决策： 策略模式 ：每个平台是一个独立的 Adapter 类，新增平台不改核心逻辑 异步优先 ：所有网络请求用 async/await，发布 9 个平台可以并发 失败隔离 ：一个平台失败不影响其他平台 骨架搭完后，掘金能发了。但只有一个平台没意义——得把所有平台都接上。 Day 6-10：攻克硬骨头 知乎：逆向 x-zse-96 签名 知乎的 API 要求每个请求带一个 x-zse-96 签名头。签名的生成逻辑藏在知乎前端的混淆 JS 里。 花了两个晚上逆向：原来是 MD5(特定参数 + d_c0 Cookie) → AES-CBC 加密 → 拼接前缀 。 def _make_x_zse_96 ( url : str , d_c0 : str ) -> str : source = f " 101_3_3.0+/api/v4/ { url } + { d_c0 } " md5_hash = hashlib . md5 ( source . encode ()). hexdigest () # ... AES-CBC 加密 ... return " 2.0_ " + encrypted . hex () 搞定的那一刻，知乎文章秒发——那种感觉比写完代码还爽。 CSDN：阿里云网关签名 CSDN 用的是阿里云 API 网关的 HMAC-SHA256 签名方案。文档是中文的，但签名串的格式写得很隐晦——分隔符必须是 \n ，不能是空格。这个细节浪费了我两个小时。 博客园：2002 年的 XML-RPC 博客园的 API 是 MetaWeblog XML-RPC——一个 2002 年的协议。Python 标准库自带的 xmlrpc.client 直接用。但有个坑：分类里不加 [Markdown] ，文章就会被当 HTML 解析。 小红书：Playwright 兜底 小红书没有公开 API，只能上 Playwright。但每次启动浏览器要 2-3 秒。优化手段： Cookie 持久化，跳过登录 先通过 API 拿到图片上传地址，只让 Playwright 做最后的表单提交 在 Linux 服务器上用 Xvfb 虚拟显示 Day 11-14：打磨产品 核心功能跑通后，开始打磨： 数据看板 ：每个平台的阅读/点赞/评论自动汇

Writing Takes 40 Minutes, Publishing Takes 30 — How I Solved It Last Wednesday, 22:00. I just finished writing a tutorial on Python async programming — 2200 words, clean Markdown, syntax-highlighted code blocks. 22:03, open Juejin. Paste title. Paste content. Code highlighting gone. Fix manually. Pick tags. Publish. 22:08, open Zhihu. Paste title. Paste content. The Draft.js editor merged async def into asyncdef . Fix line by line. Publish. 22:15, open CSDN. Paste title. Content looks fine. But the category dropdown has 50 options and "Python" is buried. Publish. 22:20, open Cnblogs. Must add [Markdown] tag to categories or the whole article renders as garbled HTML. Publish. 22:25, open SegmentFault. Search tags for "Python async" — zero results. Type manually. Publish. 22:30, open Dev.to. Translate title. Translate content. Publish. Forty minutes to write. Thirty minutes to publish. 22:35, all done. But the next morning, I wanted to check stats — another round of logging into each platform's dashboard one by one. I'm Not Alone Searching forums and social platforms, I found many developers share this pain: "Every time I publish an article, I open 7-8 tabs, copy-paste 7-8 times, fix formatting 7-8 times. The joy of writing gets killed by the drudgery of publishing." "I usually only publish on one platform now. It's just too much work to do more. But then search engine exposure suffers." Why "Just Copy-Paste" Doesn't Work Each platform has a different editor: Platform Editor Markdown Handling Juejin Custom Markdown Good, but code highlighting sometimes breaks Zhihu Draft.js rich text No Markdown support, eats line breaks CSDN Dual-mode Mode switching corrupts formatting Cnblogs TinyMCE Must add [Markdown] tag or disaster SegmentFault Markdown Okay, but tag system is painful Dev.to Markdown Best experience, but English-only audience The same Markdown renders differently everywhere. Copy-paste doesn't solve it. What I Built I spent two weeks of evenings building PolyPos

I have four different load_env() functions in my MCP server project ( my-git-manager ) — one in server.py , one in publish_devto.py , one in reply_comments.py , one in scripts/list_all_published_titles.py . All four exist for the same dumb reason: this repo has no dependency on python-dotenv , so each script that needs GITHUB_TOKEN or DEV_TO_API reads .env by hand. I went digging for a fresh bug in this repo this week — I write a lot about it, and the well is getting shallow — and decided to actually diff all four load_env() implementations against each other instead of reading them one at a time like I usually do. They'd never been compared side by side before. That's how I found this one. The line that started it Every one of them does roughly this: for line in f : line = line . strip () if " = " in line and not line . startswith ( " # " ): k , v = line . split ( " = " , 1 ) os . environ . setdefault ( k , v . strip (). strip ( '"' ). strip ( "'" )) Look closely at what gets .strip() ed there. v — the value — gets stripped of whitespace and surrounding quotes. k — the key, the actual name of the environment variable — gets nothing. That's fine if your .env file looks like this: DEV_TO_API = abc123 It's not fine if it looks like this: DEV_TO_API = abc123 Spaces around = are a completely normal thing to type. Plenty of .env examples online use them. Plenty of people reach for that style out of habit from other config formats. And line.split("=", 1) doesn't care — it splits on the first = no matter what's next to it, so k comes out as "DEV_TO_API " , trailing space included. What that trailing space actually does os.environ.setdefault("DEV_TO_API ", "abc123") sets an environment variable. It's just not the one anything is looking for. Every caller in this repo does os.environ.get("DEV_TO_API") — no trailing space, because that's the name everyone actually types. That lookup returns None , or whatever was already sitting in the environment before .env ever got read. I

"Why not just use a cron job?" is the first question I get whenever someone sees an Airflow DAG. Fair question. Cron works. It's been around for decades. It's simple. The real answer isn't that cron is bad — it's that cron solves a different problem than Airflow does. Cron is a job scheduler . It runs a command at a fixed time. That's it. It doesn't know whether the command succeeded, whether its dependencies are satisfied, or whether it should even run at all today. It just fires the command and moves on. Airflow is a workflow orchestrator . It doesn't just schedule tasks — it models them as a graph of dependencies, tracks their state, retries failed ones, and gives you a UI to see what ran, what failed, and why. Here's where that difference actually matters. The problem cron can't solve Imagine a simple ETL pipeline: Extract raw data from an API Validate and clean it Load into a warehouse Run a transformation Send a Slack alert if anything fails With cron, you'd write five separate cron entries, one per step, and hope the timing works out. If step 2 fails but step 3 runs anyway, you now have bad data in your warehouse. If step 4 takes twice as long one day, you've silently broken your SLA. Nobody gets notified unless you manually add alerting logic to every script. With Airflow, you model this as a DAG: from airflow import DAG from airflow.operators.python import PythonOperator from datetime import datetime with DAG ( dag_id = " daily_etl " , schedule = " 0 6 * * * " , start_date = datetime ( 2026 , 1 , 1 ), catchup = False , ) as dag : extract = PythonOperator ( task_id = " extract " , python_callable = extract_data ) validate = PythonOperator ( task_id = " validate " , python_callable = validate_data ) load = PythonOperator ( task_id = " load " , python_callable = load_to_warehouse ) transform = PythonOperator ( task_id = " transform " , python_callable = run_transformation ) extract >> validate >> load >> transform Airflow guarantees the order. If validate fail

If you have ever needed to send an email, process a payment, or generate a report without making your user wait, you have probably run into Celery. Celery is a tool that lets you run jobs in the background, away from your main app. This article breaks down how it works, step by step, in plain language. What Is Celery, In Simple Terms Think of Celery like a restaurant kitchen. Your app (the waiter) takes an order from a customer. Instead of cooking the food itself, the waiter drops the order into a queue (the kitchen order rail). A cook (the worker) picks up the order from the rail and prepares it. When the food is ready, it goes to a pickup counter (the result backend) where anyone can come check if it's done. Celery has four main players: The Producer - your app, the one that creates tasks. The Broker - the message queue that holds tasks until a worker is free. The Worker - the process that picks up and runs the tasks. The Result Backend - where results are stored, if you need them later. In short: your app sends a task message to the broker. The broker holds it until a worker is free. The worker picks it up, runs the actual function, and (if you set one up) writes the result to the result backend. Your app can then go back and check that result backend to see what happened. Now let's go through each part. 1. How Tasks Get Registered Before Celery can run a task, it needs to know the task exists. This is called registration , and it happens the moment your Python code is imported - not when the task runs. The @app.task decorator You create a Celery app instance, then decorate any function with @app.task . That decorator does not run the function immediately. Instead, it wraps the function and adds it to a task registry - basically a dictionary that Celery keeps internally, mapping a task name to the actual function. from celery import Celery app = Celery ( " myproject " ) @app.task def send_welcome_email ( user_id ): # logic to send an email print ( f " Sending wel

Today was a good day and a weird day, in that order. The good part: the autonomous pentest agent I've been building — I call it HALO — went from "runs a bunch of tools and hopes" to an actual web-recon → web-attack → flag-capture pipeline that pulled real flags out of a live target. The weird part: it captured flags on VulnBegin, and then, when it came time to actually submit them, they wouldn't take. Not an error. Not a crash. Just… rejected. I want to write down both halves honestly, because the second half is the more interesting engineering lesson, and it's the one I'd have skipped past a few months ago. What actually shipped today A few concrete milestones, roughly in the order they unblocked each other: The arsenal went from 31 tools to 42. I wired in a chunk of web + OSINT tooling — content discovery, subdomain enumeration, template scanning, XSS probing, passive URL collection. The point wasn't "more tools = better." It was to give the agent enough of a web-attack surface that it could go from host to flag without me babysitting each step. I stopped the silent hangs. This one cost me the most time and had the dumbest root cause. A couple of the Go-based scanners would just… hang. No output, no error, they'd ride the timeout all the way to the wall and die with nothing. I'd assumed it was a networking or a binary-compatibility problem and chased that for way too long. It wasn't. The agent runs as an MCP server over stdio — meaning the server's own stdin is the JSON-RPC pipe the whole system talks over. When I spawned a child scanner, it inherited that stdin, tried to read from it, and blocked forever waiting on a pipe that was never going to feed it. One line — stdin=subprocess.DEVNULL on the subprocess call — took one scanner from a 60-second timeout to a 1-second run. That's the whole fix. I'm still a little mad about how long it took to find. A pile of invocation fixes. Small, unglamorous, necessary: a resolver that reads targets from stdin instead of a fl

Short answer: the least risky alternative to a single-provider OpenAI-compatible API is a thin routing layer with one internal contract, a small Python adapter, and an eval set that measures answer quality before price. Treat “cheapest” as a workload result, not a label. A US/EU chatbot also needs a deliberate data-residency decision before a key or SDK enters production. The attractive story is easy: one API key, one SDK, and a familiar chat-completions shape. Measure it. The production story has more edges. Provider-specific tool calls, token accounting, streaming events, retention settings, and regional routing can differ while the first text response still looks fine. That is how an in-app chatbot passes a demo and fails an eval. Consider a support bot that retrieves three passages, answers in a stream, and offers an escalation tool. A compatibility test that checks only the final sentence can miss an empty retrieval marker, a tool argument that is valid text but invalid JSON, a stream terminator that the client never handles, and a fallback that sends the same user request to a second region. The transcript still looks plausible in a screenshot. The trace tells a different story. I've learned to make those states explicit in the adapter before tuning a model. I build RAG and agent features in Python, so my first question is not “which model wins?” It is “which contract can I test?” The app should own that contract. A provider adapter should translate it at the boundary, and the rest of the application should never know whether the request went to an OpenAI-compatible endpoint, a Claude-style API, a Gemini-style API, or a local service. How can an app chatbot compare compatible API alternatives across US and EU? Start with the request that matters to the user: a message plus retrieved context, a latency budget, a maximum output, and a trace ID. Record the selected region and provider in server-side metadata, but don't send a secret to the browser. “One API key”

GitHub热门项目 | 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training. | Stars: 163,607 | 69 stars today | 语言: Python