AI 资讯
Why Corrupted Training Data Doesn't Show Up as High Loss
Originally published at ai.bedvibe.studio . There is an assumption almost every practitioner carries without examining it: if your dataset has bad samples in it, the loss will tell you. Corrupted rows spike. Broken files stick out. Sort by per-sample loss, look at the top of the list, and there is your garbage. I believed it too. Two separate failures in my own work say it is wrong, and they fail in the same direction — quietly. The reproducible one: a dataset that cannot be learned While validating trainproof I ran a controlled fault-injection study: one base setup, a Qwen2.5-3B QLoRA, run six ways, three seeds each, eighteen runs total. Every log ships in the repo so the verdicts can be checked rather than believed. One configuration shuffled the dataset's labels into pure noise. The labels no longer corresponded to the inputs at all. This is not a hard dataset or a noisy dataset. It is a dataset that cannot be learned , because there is no relationship left in it to learn. That run reduced its loss by 62%. On its own curve it was textbook-healthy — a clean downward slope, no spike, no plateau, nothing a human or a rule would flag. It was learning nothing useful. It was memorising the statistics of noise, which any sufficiently large network will happily do. From a single run's loss curve it is indistinguishable from a real one. That is where the assumption broke for me. Not "loss is a weak signal for this." Loss is not a signal for this at all, in isolation. The production one, and what I can and cannot prove about it The second failure came from real work rather than an experiment, and it is the one I think about more. Building a text-to-speech corpus of roughly 110,000 recordings, a small number of the files were pure loud white noise. Not corrupted in the file-format sense — they opened fine, played fine, had valid headers and valid duration. They simply contained no speech. Just noise, at volume. They did not surface as high-loss outliers. Being precise about
AI 资讯
Architectural Breakdown: Can AI Remember What It Sees?
 # Can AI Remember What It Sees? The 3 AM OOM That Taught Me Everything About Visual Memory Systems At 2:47 AM, my production cluster dropped from 120 fps across 26 cameras down to absolute zero. The culprit was an unbounded `asyncio.Queue` that ballooned to 14 GB in 11 seconds. The fix was not more RAM. It was treating hardware constraints as first-class citizens in every design decision. --- ## The Core Lie: Statelessness by Design AI models forget by default. Transformers discard context once their attention window expires. CNNs process each frame in isolation with no persistence layer. **"Remembering" requires explicit memory injection.** You need RAM for short-term buffers, disk for long-term archives, and compressed embeddings for semantic recall. These are not interchangeable. Most engineers conflate them and pay the price in production. In practice, this distinction separates graceful degradation from hard crashes at the worst possible moment. The [ ShipMVP.tech ]( https://www.shipmvp.tech ) blueprint puts it plainly: **memory is a resource, not a feature.** --- ## Root Cause: The Three Sins That Killed My Pipeline ### Sin 1: Unbounded Queues python BEFORE: OOM in 11 seconds queue = asyncio.Queue() # No maxsize → infinite growth until death **Fix:** Cap queues to a hardware-derived bound. python AFTER: Hardware-bounded, fails fast on overflow self.queue = asyncio.Queue(maxsize=100) # ~1.5 MB at 224x224x3 uint8 **Failure walkthrough:** 1. Traffic spike hits 1200 fps and the queue swells to 800K frames (14 GB). 2. The kernel invokes swap thrashing until the OOM killer terminates the process. 3. **Lesson:** Derive `maxsize` from `(available_RAM / frame_size) * safety_factor`. Never guess. ### Sin 2: Redundant Allocations Each frame went through four separate copies: OpenCV BGR, Pillow RGB, NumPy
AI 资讯
How I Built a Zero-Trust Docker Sandbox for AI Coding Agents & Untrusted Repos
My vision a lightweight, permission-headache-free Docker setup for running OpenCode, uv, and untrusted Python code without risking your host OS. When contributing to unfamiliar open-source projects or letting AI coding agents (like OpenCode ) run terminal commands, there's always a slight hesitation. What if a build script touches my system Python, or a rogue command wipes host files? To solve this, I built saferun a zero-trust, disposable Docker sandbox designed specifically for Python developers and AI agent workflows on macOS and Linux. Here’s how it works, the permission nightmares I had to solve, and how you can set it up in under two minutes. The Goal I wanted a workspace that gave me: Absolute Isolation: Runtime scripts, pytest , ruff , and AI agent commands execute strictly inside a disposable Linux container. Seamless IDE Integration: Files edited inside PyCharm or VS Code on the host machine sync instantly with the container. Zero Permission Headaches: Any files generated inside the sandbox belong to my host user account—not root . Persistent Speed: Package downloads cached permanently via uv so environment startup stays millisecond-fast. Isolated Credentials: Global SSH and Git keys remain safely on the host machine. Solving the "Non-Root" Docker Nightmare The hardest part of containerized dev environments is file ownership. If you run Docker as root , any file your AI agent generates belongs to root , locking you out on your host machine. If you pass your local user ID ( -u "$(id -u):$(id -g)" ), Docker mounts non-existent directories as root:root , causing Permission Denied crashes when tools like uv try to write to cache folders. saferun solves this inside the base Dockerfile by pre-creating cache directories and granting open write permissions upfront: FROM python:3.12-slim # Install curl (needed to install OpenCode) RUN apt-get update && apt-get install -y --no-install-recommends \ curl \ && rm -rf /var/lib/apt/lists/ * # Install uv globally RUN pip
AI 资讯
Observability for AI Agents with OpenTelemetry
AI agent observability means capturing your agent's reasoning cycles, tool calls, and token usage as...
开源项目
🔥 microsoft / data-formulator - 🪄 Data Formulator is an interactive AI-powered data analysis
GitHub热门项目 | 🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data. | Stars: 16,944 | 668 stars this week | 语言: Python
开源项目
🔥 AgriciDaniel / claude-obsidian - Self-organizing AI second brain for Obsidian + Claude Code.
GitHub热门项目 | Self-organizing AI second brain for Obsidian + Claude Code. Drop any source and Claude reads, links, and files it into one connected knowledge graph of plain Markdown you own. AI note-taking, personal knowledge management (PKM), and an open-source Notion alternative. Based on Karpathy's LLM Wiki pattern. | Stars: 11,516 | 272 stars today | 语言: Python
AI 资讯
Architectural Breakdown: We fixed the eval platform we're competing on: a TypeError that crashed thr
We Fixed the Eval Platform: The TypeError That Took Down Three Benchmark Pipelines At 3 AM, Sentry lit up with TypeError: Cannot read property 'map' of undefined . Three benchmark pipelines crashed. Not a memory leak, not a segfault, but a race condition hiding behind a TypeError, turning a high-stakes eval run into chaos. Here is how we resolved it, with no fluff. The Root Cause: Async Data Meets Blind Faith in .map() The error trace pointed to evaluator.ts:42 , where .map() assumed inputData.metrics would always exist. The junior dev tested with clean data, but in production, fetchBenchmarkData() (async) and evaluatePipeline() (sync) were racing . At 100+ RPS, metrics was often undefined . The Offending Code: const results = inputData . metrics . map ( metric => computeScore ( metric )); Why It Failed: Race Condition : inputData was fetched asynchronously, but evaluatePipeline() treated it as synchronous. OOM Risk : Unbounded .map() on 10K+ metrics could exhaust 8GB RAM. Worker Starvation : No concurrency limits led to thread pool exhaustion. The Fix: Guard Clauses, Bounded Queues, and Pragmatism Step 1: Fail Fast, Fail Loud Added zero-overhead runtime checks to reject bad data early: // eval-platform/core/evaluator.ts import { isNullOrUndefined } from ' ../utils/guards ' ; async function evaluatePipeline ( inputData : BenchmarkInput ): Promise < EvaluationResult > { if ( isNullOrUndefined ( inputData ?. metrics )) { throw new Error ( ' EVAL_400: metrics missing ' ); } // Proceed only if data is valid } Why? Stops TypeError crashes immediately. Cost: 1-2 CPU cycles. Negligible. Step 2: Chunked Processing for 8GB RAM Original code processed all metrics at once, causing OOM crashes. Fixed with 100-item chunks: const CHUNK_SIZE = 100 ; // 100 items ≈ 10MB peak memory const results : number [] = []; for ( let i = 0 ; i < inputData . metrics . length ; i += CHUNK_SIZE ) { const chunk = inputData . metrics . slice ( i , i + CHUNK_SIZE ); results . push (... chunk . map
开发者
ESP32 + Python: From Microcontroller to IoT
ESP32 + Python: From Microcontroller to IoT Artcal 0: Introduction When it comes to transferring data from one place to another, things can sometimes become tricky, especially when communication happens between the hardware and software levels. In this article series, I would love to share the experience and knowledge I’ve gathered while working with ESP32 and Python. We’ll explore how these two technologies can work together, starting from the basics and gradually moving towards more interesting and practical projects. If you have any questions, suggestions, or ideas along the way, feel free to share them in the comments section below. I’d love to hear from you and discuss them with the community. So, without further ado, let’s begin! 🚀 What is ESP32? Think about Esp32 as a microcontroller with Internet facilities, consisting WiFi, Bluetooth and a own wireless data transfer protocol called ESP-NOW between ESP32 chips. Nowdays, the developers have made development boards integrading these chips for the easy use. ESP32 is a family of microcontrollers developed by Espresiff. This can read sensor inputs, process data, contol devices and specially connect to the internet. This is like Arduino but better, faster and smaller. With these information that we have, we can speak about this board as, "A powerful microcontroller that can interact with electronic components and communicate with other devices through Wi-Fi, Bluetooth, and other communication methods." Python??? We use different languages to tell the same thing but in different ways. We use programming languages to tell the computer the same thing but in different approches. Some languages can be hard to learn and some are easy. Some are well developed and some are not. Python programming language was created back in 1980s by Guido Van Rossum, with the development begining around 1989. It was publicly released in Feb, 1991. 🐍 1989 — Guido van Rossum developing Python. 🐍 1991 — The first public release. 🐍 2000 — Py
AI 资讯
The Counter That Counted a Call the Preflight Never Reached
This is a submission for DEV's Summer Bug Smash: Clear the Lineup , powered by Sentry . Project Overview I was working on a small Python component that performs a preflight check and then, if the check succeeds, invokes one synchronous operation callback. A counter records whether that callback invocation returned normally. The counter is used for diagnostics, so it must follow the control flow rather than the expected happy path. Bug Fix or Performance Improvement When a handled failure occurred, the old implementation still returned one: return 1 That value was hard-coded because the successful path was expected to invoke exactly one operation. If the preflight check failed, however, the operation was never entered and the function still returned one. An offline reproduction produced: operation_entries=0 old_count=1 The failure was handled, but the counter contradicted the actual control flow. Code Reduced to the relevant lines, the old behavior was: # Simplified pre-fix behavior def buggy_completed_calls ( * , preflight , operation ): try : preflight () operation () except Exception : pass return 1 Here is the complete fixed function from the standalone reproducer: from collections.abc import Callable Callback = Callable [[], None ] def completed_calls ( * , preflight : Callback , operation : Callback ) -> int : """ Return one only when the cooperative operation returned normally. """ try : preflight () operation () except Exception : return 0 return 1 The essential regression assertion is shown below. Both callbacks are local, so the test performs no network request: # Abbreviated test excerpt def test_preflight_failure_does_not_count_an_unentered_operation (): operation_entries = 0 def refuse_preflight (): raise RuntimeError ( " controlled preflight refusal " ) def operation (): nonlocal operation_entries operation_entries += 1 result = completed_calls ( preflight = refuse_preflight , operation = operation , ) assert operation_entries == 0 assert result == 0 My
开发者
thumb: popup images and render LaTeX directly in Vim
I made a small Vim 9.2+ plugin called thumb . Put the cursor on an image path → :Thumb → popup the image. Select LaTeX in Visual mode → :Thumb → render it as an image popup. For example:  Put the cursor on diagram.png and run: : Thumb Or select: \frac { a }{ b } = \sqrt { x ^ 2 + y ^ 2 } and run: : Thumb It uses Vim's native popup image support, with Python/Pillow for image conversion and matplotlib for LaTeX rendering. No mappings are installed, so you can add your own: nnoremap < leader > t < Cmd > Thumb < CR > xnoremap < leader > t < Cmd > Thumb < CR > GitHub: https://github.com/JosefAlbers/thumb Requires Vim 9.2+, Python 3, Pillow, and matplotlib. Feedback welcome, particularly around the popup positioning/rendering.
AI 资讯
The Model Was Fine. My Token Assumptions Weren't.
The model was never the problem, and that is exactly why the bug took three days to find. My ticket-classification service started returning the fallback label for long, non-English messages shortly after I moved the inference path to a cheaper endpoint, and every instinct pointed at the new model. The real culprit was a token-counting mismatch that silently truncated the prompt before the model ever saw the classification instruction. The Symptom The failure was remarkably consistent, which made it even more misleading. Messages under roughly two thousand characters classified correctly, while longer ones, especially in German and Japanese, fell through to a generic "other" bucket with a perfectly valid JSON response. The parser was not the issue, the prompt had not changed in weeks, and the retry logic never fired because the endpoint returned a normal 200 status. My first assumption was that the cheaper model was simply weaker at long-context reasoning, so I ran a controlled comparison using the same fifty tickets against the previous endpoint. The old path classified all fifty correctly, the new one failed on nineteen, and that result seemed to confirm the model-quality theory. What bothered me was the distribution: the failures clustered exactly where the input length crossed a threshold, and no ticket under that threshold ever failed. The Reproduction To isolate the variable, I needed a clean environment where I could swap endpoints without touching the production deployment, and MonkeyCode's free server option turned out to be a practical debugging tool. The project is open source, and its free model access let me replay the failing tickets without spending my own quota, so I spun up a disposable instance and pointed the same harness at the same prompt. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The reproduction took about twenty minutes, and the result was identical on every retry: long inputs failed, short inputs passed.
AI 资讯
Your Retry Loop Is a Token Incinerator: A Cascade Router for Mixed-Tier Endpoints
When a free endpoint returns 429, most agents do the most expensive thing possible: retry. Retrying looks harmless. A 200-millisecond request becomes a 2-second wait, then another attempt. But under peak load, that loop becomes a 30-second stall while your agent clicks refresh on an empty response. If the quota window resets during the stall, every retry burns tokens you could have spent on actual work. The retry loop assumes the failure is temporary. For rate limits, that assumption is usually wrong. Quota counters reset on a fixed schedule, not on your convenience. You are not just waiting; you are burning wall-clock time that could have gone elsewhere. The Cascade Pattern A cascade router is the alternative. It sends requests to the free endpoint, backs off on rate-limit signals, then degrades gracefully to a backup endpoint. The free tier carries the load; the backup exists only when needed. You get the cost advantage of the free tier and the reliability of the paid tier. The design has three parts: an endpoint abstraction layer, a rate-limit detector, and a circuit breaker that trips when the free endpoint fails repeatedly. Here is the core code: # cascade_router.py — free tier first, paid/self-hosted as fallback. import json import os import time import urllib.error import urllib.request from dataclasses import dataclass @dataclass class Endpoint : name : str url : str api_key : str model : str cooldown_until : float = 0.0 consecutive_failures : int = 0 def available ( self ) -> bool : return time . time () >= self . cooldown_until class CascadeRouter : def __init__ ( self , endpoints : list [ Endpoint ]): self . endpoints = endpoints def _call_one ( self , ep : Endpoint , messages : list [ dict ]) -> tuple [ int , dict ]: body = json . dumps ({ " model " : ep . model , " messages " : messages , " max_tokens " : 256 }). encode () req = urllib . request . Request ( ep . url , data = body , headers = { " Content-Type " : " application/json " , " Authorization "
AI 资讯
I Built a Python Bot That Plays Blackjack on Discord's OwO Bot 🃏
I Built a Python Bot That Plays Blackjack on Discord's OwO Bot 🃏 An automation experiment in game logic, human-like timing, and why the house still wins. ⚠️ Disclaimer first: This project is for educational purposes only . It's a coding experiment about automation, pacing, and basic blackjack strategy. I'm not promoting gambling, I'm not responsible for any losses, and self-bots can violate Discord's Terms of Service — know the rules before running anything like this. What is this thing? If you've spent time in Discord economy servers, you've probably met OwO Bot — one of the most popular Discord bots out there, with its own cash economy and gambling minigames, including Blackjack . I asked myself a fun engineering question: Can I write a Python client that plays full Blackjack sessions on its own — with human-like pacing, break cycles, and a sensible betting strategy? That experiment became GhoSty OwO BlackJack Farm — a Python-based Discord self-bot focused on OwO Bot's Blackjack, now at V2.1 . What it does 🔄 Full Blackjack automation — handles the game loop end-to-end. 💡 Smart betting — strategy-based decisions instead of random yolo bets. 😴 Smart Sleep — lifetime work/break cycles instead of 24/7 spamming. ⏱️ Dynamic gaps — randomized delays between every action. 🚨 Zero win guarantees — on purpose. More on that below. The stack (and why an old discord.py) Python 3.10+ discord.py==1.7.3 colorama Yes, 1.7.3 is ancient — deliberately. The self_bot=True pattern that this kind of client relies on was removed in newer discord.py versions, so legacy 1.7.3 is the line that still supports it. If you've never touched pre-2.0 discord.py, this project is a small time capsule of that API. The whole project is intentionally tiny: OwO-Blackjack-Farm/ ├── main.py # bot + game logic ├── config.json # your token & settings ├── requirements.txt └── README.md Setup is two steps: drop your token into config.json , then: pip install discord.py == 1.7.3 colorama python main.py Start it
开源项目
🔥 NVIDIA-NeMo / Speech - A scalable generative AI framework built for researchers and
GitHub热门项目 | A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech) | Stars: 18,257 | 45 stars today | 语言: Python
AI 资讯
I Got 28 TPS Out of Free Kaggle GPUs. Here's What It Took.
I want to be upfront about something: this whole project runs on free Kaggle T4 notebooks, an AWS EC2 t3.micro relay that costs almost nothing, and public internet. No A100s. No private datacenter network. No budget. And yet, ShardFlow v2.1 hits 28.10 TPS peak on Qwen2.5-7B across two separate cloud regions over WAN. This is the story of how that happened, and specifically the one fix in v2.1 that I did not see coming. The Problem: Running a 7B Model When You Have No Money A 7B parameter model in FP16 needs roughly 15 GB of VRAM. A single Kaggle T4 has 16 GB. Technically it fits, barely, with nothing left over for a KV cache. The solution is tensor parallelism: split the model across two machines. Node 0 (Iowa) handles layers 0 to 14. Node 1 (Oregon) handles layers 14 to 28, plus the LM head and final verification. They talk to each other through a TCP relay running on an EC2 t3.micro in Ohio. The baseline throughput with this setup and no tricks: 4.92 TPS. Usable, but not fast. Speculative Decoding: The Idea LLM inference is slow because it's sequential. You generate one token, wait, generate another, wait. Each round trip across WAN costs you ~86ms RTT. At 1 token per round trip, you're fighting the network the whole time. Speculative decoding flips this. Instead of sending one token at a time, you run a tiny draft model locally to guess the next K tokens ahead. Then you send all K guesses to the verifier in one shot. If the big model agrees with M of them, you've committed M tokens in a single round trip instead of one. ShardFlow uses Qwen2.5-0.5B as the draft model, running on cuda:1 of Node 0 while the 7B target slice runs on cuda:0. Zero VRAM contention. The drafter proposes 8 candidates, Node 1 verifies them all in parallel, and you get an average of 4.07 tokens per round trip instead of 1. With speculative decoding in eager mode: 14.3 TPS peak. 3x better. The Wall I Hit I thought 14.3 was the ceiling. The network was the obvious bottleneck: two Kaggle instan
AI 资讯
When Python is Too Slow
Python is a perfect language for Agile development, where requirements might change on the go. Especially if you are in a startup business, you will need to experiment and change things fast. However, Python is an interpreted language, and in certain situations you might need faster performance than what an interpreted language can provide. A common practice in these cases is using python-to-binary bindings, where the binary code is built with Rust, C++, or Go. In this article, I will explore bindings to Rust-based code. How do the bindings work The idea behind bindings is that you create a module with functions of a specific domain in a language that compiles to binary, and build it as a C-compatible dynamic library ( .so on Linux, .dylib on macOS, .dll on Windows). Then a Python wrapper is built as a Python package and installed together with the dynamic library, allowing you to import and use functions that pass control to the corresponding functions in the dynamic library. On some occasions, classes can be used instead of functions. If any parameters are complex, they must be serialized in the wrapper and passed to the dynamic library as a JSON string or as a set of individual primitive parameters. An experiment with benchmarks To try this Python-Rust communication, I vibe coded an experiment that reads a large CSV file and builds a new one with duplicates stripped out based on specified column indexes. In my test case, it was a 3 MB CSV file with data about European NGOs for the donation platform I am building, where I wanted to remove the NGOs that don't have website URLs listed. As benchmarked, the file was processed 4.3x faster with the Rust binding than directly with Python. Here is the repo to get a first glimpse into the code and structure. What is there to know about Rust A few things about Rust: Rust packages are built with Cargo, which is the equivalent of pip, virtualenv, and setuptools combined. A single package is called a crate, and it can be publi
开源项目
🔥 shy3130 / tick-stock-panel - TSP自托管、零运维的 A 股「选股 + 监控 + 回测」量化工作台 | 基于 TickFlow 数据源 | LLM能力
GitHub热门项目 | TSP自托管、零运维的 A 股「选股 + 监控 + 回测」量化工作台 | 基于 TickFlow 数据源 | LLM能力驱使策略定制+个股分析+复盘 | 自由接入第三方数据源与个性化扩展数据 | 个人开源 ,非TickFlow官方项目 | Stars: 3,477 | 82 stars today | 语言: Python
AI 资讯
How I built an FVG trading bot for OKX and made 99% of its signals useless on purpose
How I built an FVG trading bot for OKX and made 99% of its signals useless on purpose If you trade crypto futures, you know the drill. You're staring at the chart at 3am because you're scared to blink and miss "the perfect entry". Or worse, you get in emotionally, chase a pump, and hand back all your profit in one bad night. I got liquidated once because my stop was at -5% and the liquidation price was at -2%. Price gapped straight through my stop. That's how this project started. I built FVG Killer , a bot that trades one setup only: the ICT Fair Value Gap, on OKX perpetuals. The repo is open-source: https://github.com/Xbs950812/okx_fvg_agent 1. What it trades FVG stands for Fair Value Gap, from the ICT (Inner Circle Trader) framework. The idea is simple: one violent candle moves price fast and leaves a "vacuum" where almost nobody got filled. The theory says market makers rebalance and price tends to come back and fill at least half of that vacuum. So the bot waits for price to retrace into the gap, enters, takes profit at the 50% level (consequent encroachment in ICT-speak), and stops out outside the gap. Detection pipeline: Pre-filter: at least a 3-sigma move and 5x volume expansion Three-candle gap detection, scanning 1H and 4H It tracks the top 100 contracts around the clock, even when it holds nothing 2. The part nobody tells you: saying no Textbooks show you three candles and call it a day. Reality: a naive detector spits out dozens of signals a day and 99% of them are garbage. I built five gates to reject them. Each has a real log line from production: Freshness: gap older than ~100 candles? Drop it. [Freshness] SNXX 1H FVG 186 candles old > 24, drop ATR grade: gap width less than 0.5x ATR is a weak setup. [ATRGrade] width 0.16/ATR 0.37 = 0.43 < 0.5, weak C-grade Direction: don't long a coin that just pumped 14%, don't short one that dumped. [MoverDir] ETHFI 4H long rejected: +14.4% in 24h Depth: if the resting order is 6% off price, you're catching a falli
AI 资讯
483 tests passed, but Vestibule RAG framework wasn't installable — lessons from building with AI agents
I spent two months building Vestibule, an open-source Python framework for the boring layer of RAG ingestion — stable document IDs, a state ledger, error classification, per-vertical governance. The parts every team struggles with once the demo works and production doesn't. Most of the code wasn't typed by me. Four AI agents did the work — one wrote designs, one reviewed them, one implemented, one reviewed the code — all through real GitHub pull requests, with me signing off at every gate. The result: twelve components, three releases, 878 tests. Two moments defined the whole experience. When the process caught what I couldn't The trickiest component provisions vector indexes on first use, safely even when workers race each other. Its design was rejected and revised five times before any code existed. In the first round, the reviewer agent found a genuine race condition: a worker still inside a slow index-creation call (~390 seconds with retries) would look stale (the threshold defaulted to 300 seconds), lose its claim to a waiting worker, and now two workers create the same index. A production race, in the default configuration, spotted by one AI reading another AI's design — before a single line was written. When green tests lied to me After v0.2 shipped, I wrote a quickstart script and ran the pipeline the way a stranger would — for the first time. pip install didn't work. At all. A packaging conflict made the whole framework uninstallable, while 483 tests sat green. An hour of actually using it turned up two more: a default model name that had never once worked against the real SDK, and an import that took down an entire package when an optional dependency was absent. What went wrong wasn't the tests — it was what they measured. They proved the code agreed with itself: same working tree, same mocked seams. Nothing ever checked the world a user lives in: clean machine, real install, real SDK. Passing tests and a working product turn out to be two different claims
AI 资讯
skillcheck Update: Scorer Fixes, Cleaner Failures, Honest Token Numbers
skillcheck is a static analyzer for SKILL.md files, the format agents like Claude Code, Copilot, Codex, and Cursor use to load reusable skills. It validates frontmatter, scores description discoverability, checks file references, enforces token budgets, and flags cross-agent compatibility issues. No network calls, no LLM calls, no file mutations. Runs as a CLI, a GitHub Action, or a pre-commit hook. pip install skillcheck skillcheck skills/ Latest pass was hardening and accuracy, not features. Here's what changed and why. Description scores went up. Skills that were scoring low because the scorer was broken will now see a jump in scoring. Median across the reference corpus went from 75 to 90. --explain-score also now tells you which pattern hits or misses instead of just a number. The score exists to predict whether an agent will actually find and trigger your skill, so a scorer that under-credits good descriptions defeats the point. The fix was validated against real-world skills, and the separation held: filler still scores 28-65, well-written descriptions 85-100. Corrupt files now fail cleanly instead of crashing. Before, a bad history ledger or non-UTF-8 skillcheck.toml above the skill dumped a Python traceback. It's now a clear error naming the file and byte offset (exit code 2). Config discovery walks up the directory tree, so one bad file could break every scan under it. Now every untrusted read (ingest, history, config) goes through the same guard before parsing, so they all reject the same way. README has been corrected in regards to token estimates. Without tiktoken, expect roughly 20-30% over-estimation, so install the extra if you're near a budget limit. The offline heuristic feeds the budget checks and its accuracy had never actually been measured, just assumed. It's benchmarked against tiktoken across the full corpus now, and the documented numbers are the measured ones. pip install "skillcheck[tiktoken]" The rest of the pass is invisible on purpose: f