标签：#python

FP8 gave us a clean 1.5x on Qwen3-8B serving throughput on an RTX PRO 6000 Blackwell (1,725 to 2,597 tok/s at concurrency 32, vLLM). The uncomfortable question is always the same: did the model get dumber. This post is the exact check we ran before recommending the switch, with numbers, so you can run the same one. Why "run an eval suite" is usually the wrong first answer Standard benchmarks (MMLU and friends) are noisy instruments for quantization deltas at 8B scale. Score movement inside the error bars tells you nothing about whether YOUR prompts changed behavior. What you actually want to know is narrower: on the workload you serve, does the FP8 checkpoint produce materially different outputs than BF16, and are any of the differences wrong. That is answerable directly, cheaply, and per prompt. The method Both configurations run the same fixed workload: 20 prompts covering reasoning, code, summarization, translation, extraction, classification, math, and instruction following. Greedy decoding, temperature 0, 256-token cap, streamed. Greedy matters: it removes sampling noise, so any output difference is attributable to the numerics. Then a three-stage comparison: Byte equality. outputs_bf16[i] == outputs_fp8[i] . Anything identical is settled. Similarity triage. For non-identical pairs, difflib.SequenceMatcher.ratio() sorts near-identical wording drift from real divergence. Side-by-side review under a written rubric. Every non-identical pair gets read. The rubric asks one question: is there a factual or numerical claim that one precision gets right and the other gets wrong. Wording changes, reordering, and equally-defensible readings are recorded but not counted as regressions. The core loop is small: import difflib , json bf16 = json . load ( open ( " vllm_bf16_conc1.texts.json " )) fp8 = json . load ( open ( " vllm_fp8_conc1.texts.json " )) for i , ( a , b ) in enumerate ( zip ( bf16 , fp8 )): if a == b : print ( i , " identical " ) continue r = difflib . Sequenc

GitHub热门项目 | Open-source framework for the research and development of foundation models. | Stars: 1,923 | 214 stars today | 语言: Python

I often remember the shot I want before I remember its filename. That gap is what binquery is for. It is a local Python CLI that indexes video clips and turns a sentence into a ranked shortlist for a human to review. It deliberately stops before editing: no timeline generation, no automatic cut, and no render. The smallest reproducible trial You can test the complete installed command path without supplying footage: python3 -m venv .venv .venv/bin/pip install binquery .venv/bin/binquery demo --out /tmp/binquery-demo The demo generates a synthetic 30-second video locally, then exercises splitting, indexing, validation, and querying. The first run may download OpenCLIP model weights. This is an end-to-end pipeline smoke test, not evidence of semantic search quality on real footage. Why keep the architecture small? The current design uses: ffmpeg to sample three frames from each clip OpenCLIP ViT-B-32 to build the local visual index plain JSON and NumPy files for metadata and vectors a JSON result containing clip paths, scores, and ranking signals There is no database, vector service, or daemon to operate. Querying an existing index does not resample the footage or rebuild the full index. The trade-off is straightforward: three frames keep indexing understandable and bounded, but they can miss important content in long or visually varied clips. I would rather expose that limitation than market a synthetic demo as a quality benchmark. Ranking signals are not explanations The output includes fields such as score , gate , and reasons . Here, reasons means ranking signals recorded by the pipeline. It should not be interpreted as a reliable semantic explanation of why a clip is correct. That distinction matters because a plausible-looking explanation can create more confidence than the underlying retrieval quality deserves. The shortlist is meant to reduce what a person must inspect, not replace editorial judgment. What binquery does not do It does not build a timeline or e

Why a Local RAG Chatbot for Trading Research Most "AI trading assistant" products are black boxes: your notes, strategy docs, and market notes get shipped to a third-party API, billed per token, and stored who-knows-where. For a retail NIFTY trader or a quant researcher, that is the worst of all worlds — you pay continuously, you leak your edge, and you cannot audit what the model actually read. This guide shows how to build a Retrieval-Augmented Generation (RAG) chatbot that runs 100% locally on an Android phone using Termux + Ollama. It ingests your own research (PDFs, markdown notes, option-chain exports) and answers questions grounded only in that data. No OpenAI key. No Anthropic key. No monthly bill. No data leaving the device. OBSERVED: Running ollama run llama3.2 on a mid-range phone inside Termux is slow but usable for document Q&A (3–8 tokens/sec). On a laptop it is smooth. SOURCE: Local testing on Termux 0.118, Ollama 0.3.x, Android 14. DERIVED: For production research volumes, run Ollama on a spare x64 machine and point Termux at it over LAN. What You Will Build A four-part pipeline: Ingest — load your research docs (markdown, PDF, CSV) into chunks. Embed — turn chunks into vectors with a local embedding model. Store — keep vectors in a local file-based index (no server needed). Answer — retrieve top-k chunks and ask a local LLM to answer strictly from them. The whole thing is ~200 lines of Python. No paid APIs. Prerequisites Android phone with Termux installed (F-Droid version, not Play Store). ~2 GB free storage. Basic Python comfort. pkg update && pkg upgrade -y pkg install python clang ffmpeg -y pip install ollama numpy Install Ollama inside Termux: curl -fsSL https://ollama.com/install.sh | sh NOTE: The official install script targets Linux. On Termux you often need the community build. If the script fails, install the ollama package via a Termux-compatible binary or run Ollama on a LAN machine and use ollama serve remotely. Pull a small model and a

Free AI Tiers Bill You in Hours, Not Dollars Free model access looks like a bargain until you track the hours you spend feeding context back into a model with no memory. A zero-cost invoice hides the most expensive resource in your workflow: your own attention. My position is straightforward: treat a free tier like a metered service and measure the hidden costs before you adopt it. The token counter tells you almost nothing about the real price. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I'm using MonkeyCode's free model access and free server option as a concrete example; the measurement approach applies to any free tier. The dashboard shows tokens, not time Every free plan advertises a generous token allowance and a server that wakes up on demand. What the marketing page omits is the labor you spend reassembling context, waiting for cold starts, and double-checking output. Those costs do not appear on any invoice, but they consume your day in chunks. Four of them matter more than the token meter. Context reconstruction — Every new conversation starts from zero, so you re-explain your stack, your file layout, and your constraints. Those re-pasted tokens count against the same allowance you were trying to save. Cold-start waiting — A free server that sleeps after idle adds seconds to every call. Multiply that by a scheduled job that fires hourly and you have lost real time. Human verification — Confident output still needs a human to check it, and that check is the most expensive line item in the whole system. Attention fragmentation — A free allowance looks huge until you split it across codegen, debugging, and review. Small tasks nibble the budget faster than big ones. A ten-minute audit script The script below turns the argument into a reproducible measurement. It sends three representative prompts to any OpenAI-compatible endpoint, records wall-clock latency, and extracts token usage from the response. Run it several times du

Here's the conclusion up front: a free LLM endpoint is a moving target. You can't see the changes, but they're happening — model updates, quantization tweaks, server-side prompt rewrites. And your app will feel them, usually as a slow, invisible quality dip. I've spent weeks on this account probing free LLM servers, caching tokens, and building evaluation harnesses. The pattern I keep seeing: teams pick a free tier, wire it in, and then never look at it again. They treat it like a static API. It isn't. The fix is a nightly drift check. A small script that runs your most important prompts against the endpoint, compares the outputs to a baseline, and tells you when something changed. Not a benchmark. Not a one-time eval. A recurring alarm. This post walks through a 90-line harness you can run tonight. I'll use MonkeyCode's free server as the reference endpoint — it's an open-source project with free model access, a free server option, and, as advertised at the time of writing, a 10M token grant. The exact numbers may move, so check the repo's README before you depend on them. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Why drift is the silent killer of free-tier apps Let's be honest: free endpoints don't come with changelogs. The provider can swap the underlying model, adjust the temperature default, or add a safety filter without telling you. Your tests still pass. Your error rate stays flat. But the responses get a little shorter, a little more evasive, a little less useful. Users notice before you do. They don't file bugs for 'the bot got dumber.' They just stop using it. A drift check turns 'the bot got dumber' into a concrete signal: 'the pass rate on 12 core prompts dropped from 92% to 74% overnight.' That's something you can act on. Step 1: Define your core prompts Don't test everything. Pick 10-20 prompts that represent the actual workload your app handles. For each prompt, define what 'good' looks like. Prompt Expected beha

Polymarket Paper Trading Bot: Build One in Python A real-money trading bot is the wrong place to discover that your signal logic, order-book handling, or position accounting is broken. A Polymarket paper trading bot gives you a safer engineering environment: consume real market data, generate real signals, simulate orders and fills, and measure hypothetical performance before connecting execution credentials. The important distinction is that paper trading should simulate the execution layer , not fabricate market data. Polymarket currently exposes public market data without authentication, while its public WebSocket market channel provides real-time order-book and price updates. This article builds that architecture in Python. What You'll Learn How a paper-trading architecture differs from a live bot How to discover markets through the public API How to consume CLOB order-book data How to simulate limit-order fills How to track positions and P&L How to test arbitrage, market-making, and directional strategies How to graduate from paper trading to production safely About the Author Soulcrancerdev Contact: X: @soulcrancerdev Telegram: soulcrancerdev YouTube: YouTube channel The Architecture A useful design separates data, strategy, simulation, and accounting : flowchart LR A[Gamma Market Discovery] --> B[Market Metadata] C[CLOB REST / WebSocket] --> D[Market Data Engine] B --> D D --> E[Strategy Engine] E --> F[Paper Execution Engine] F --> G[Virtual Portfolio] G --> H[P&L / Risk Metrics] D --> I[Logger / Metrics] The key design decision is that PaperExecutionEngine should implement the same interface your live execution engine eventually uses. That means the strategy does not know whether an order is simulated or real. 1. Discover Markets Polymarket's Gamma API provides public market discovery. The current documentation exposes keyset pagination through: https://gamma-api.polymarket.com/markets/keyset Markets include fields such as conditionId , clobTokenIds , outco

First post in a build-in-public series about verdict , an MCP server that gives coding agents structured, sandboxed test feedback. The problem Watch a coding agent work and you'll see it run pytest in your shell, unsandboxed, and then push 40,000 tokens of raw output through its context window to answer one question: did my change break anything? That's three problems in one command: Token waste. The agent needs ~10 lines of signal and pays for a wall of dots, warnings, and tracebacks. No sandbox. The tests run on your machine, in your environment, with your files writable. No memory. When a test fails, the agent can't tell whether it broke it or whether it was broken before it arrived - so it either "fixes" pre-existing failures nobody asked about, or ships regressions it assumes were already there. verdict is an MCP server that replaces the pytest shell-out with four tools: tool what it returns verify(scope?) impact-selected tests, run in an ephemeral container, as a ~400-token typed verdict explain_failure(check_id) the full traceback - only on demand history(fingerprint) first seen / last seen / times seen for a failure run_checks(["ruff","mypy"]) lint & type checks, same verdict shape ▶️ Watch the 30-second demo - Claude Code fixing a bug with verdict verifying in a container. The three ideas 1. Verdicts, not output. verify returns typed JSON: counts, per-failure message + location, and nothing else. Full tracebacks live behind explain_failure . The whole verdict for a real failing run is ~400 tokens - the raw pytest output it replaces was ~40k. The design rule in the repo is blunt: nothing bulky rides in the summary, ever. 2. Fingerprints give failures identity. Every failure is hashed from its normalized signature - volatile tokens (addresses, tmp paths, ids, durations) collapsed first. Same logical failure ⇒ same fingerprint, across runs and refactors. Fingerprints are what make the third idea possible: 3. History answers "was it me?" verdict keeps a small S

Originally published at ai.bedvibe.studio . I built a text-to-speech product and kept getting burned by the same thing. On normal sentences the model sounded great. Then it would hit a number, a date, an acronym or a name, and quietly mangle it. Worse, the metric everyone reaches for — Word Error Rate — was lying to me in both directions. It flagged perfectly good audio as broken because the script said 3:30 PM and the transcript said "three thirty pee em." And it missed real failures on short tokens, where the speech recogniser is as unreliable as the TTS. So I wrote the QA framework I wished I had, packaged it as ttsproof , and then ran it as a blind study against a production TTS service so the results would be more than an opinion. The two failures WER cannot see A TTS pipeline breaks in two different ways, and a single WER number blurs both. Structural defects. The clip is empty, truncated, three times too long, stuck in a repeated-chunk loop, clipping, or has a click at the tail. These have nothing to do with pronunciation — you can catch them with no model at all, straight from the waveform. Pronunciation and content errors on the hard cases: numbers, decimals, dates, clock times, acronyms, single letters, URLs, names. ttsproof splits them apart and handles each one honestly: Structural checks, no model needed — empty or truncated audio, duration explosions, long internal silences, clipping, loop detection, end-of-clip artifacts. numpy and soundfile, nothing else. Equivalence-aware WER/CER — the expected text and the ASR transcript are both canonicalised to spoken form before scoring, so 3:30 PM against "three thirty" stops counting as an error. ASR-uncertainty quarantine — when the audio is structurally clean but the recogniser disagrees on a very short utterance, the sample is set aside for a human instead of being auto-failed. At that length the ASR is as likely to be wrong as the TTS. The study: 390 samples, and a blind human check I evaluated the method

Originally published at ai.bedvibe.studio . There is an assumption almost every practitioner carries without examining it: if your dataset has bad samples in it, the loss will tell you. Corrupted rows spike. Broken files stick out. Sort by per-sample loss, look at the top of the list, and there is your garbage. I believed it too. Two separate failures in my own work say it is wrong, and they fail in the same direction — quietly. The reproducible one: a dataset that cannot be learned While validating trainproof I ran a controlled fault-injection study: one base setup, a Qwen2.5-3B QLoRA, run six ways, three seeds each, eighteen runs total. Every log ships in the repo so the verdicts can be checked rather than believed. One configuration shuffled the dataset's labels into pure noise. The labels no longer corresponded to the inputs at all. This is not a hard dataset or a noisy dataset. It is a dataset that cannot be learned , because there is no relationship left in it to learn. That run reduced its loss by 62%. On its own curve it was textbook-healthy — a clean downward slope, no spike, no plateau, nothing a human or a rule would flag. It was learning nothing useful. It was memorising the statistics of noise, which any sufficiently large network will happily do. From a single run's loss curve it is indistinguishable from a real one. That is where the assumption broke for me. Not "loss is a weak signal for this." Loss is not a signal for this at all, in isolation. The production one, and what I can and cannot prove about it The second failure came from real work rather than an experiment, and it is the one I think about more. Building a text-to-speech corpus of roughly 110,000 recordings, a small number of the files were pure loud white noise. Not corrupted in the file-format sense — they opened fine, played fine, had valid headers and valid duration. They simply contained no speech. Just noise, at volume. They did not surface as high-loss outliers. Being precise about

 # Can AI Remember What It Sees? The 3 AM OOM That Taught Me Everything About Visual Memory Systems At 2:47 AM, my production cluster dropped from 120 fps across 26 cameras down to absolute zero. The culprit was an unbounded `asyncio.Queue` that ballooned to 14 GB in 11 seconds. The fix was not more RAM. It was treating hardware constraints as first-class citizens in every design decision. --- ## The Core Lie: Statelessness by Design AI models forget by default. Transformers discard context once their attention window expires. CNNs process each frame in isolation with no persistence layer. **"Remembering" requires explicit memory injection.** You need RAM for short-term buffers, disk for long-term archives, and compressed embeddings for semantic recall. These are not interchangeable. Most engineers conflate them and pay the price in production. In practice, this distinction separates graceful degradation from hard crashes at the worst possible moment. The [ ShipMVP.tech ]( https://www.shipmvp.tech ) blueprint puts it plainly: **memory is a resource, not a feature.** --- ## Root Cause: The Three Sins That Killed My Pipeline ### Sin 1: Unbounded Queues python BEFORE: OOM in 11 seconds queue = asyncio.Queue() # No maxsize → infinite growth until death **Fix:** Cap queues to a hardware-derived bound. python AFTER: Hardware-bounded, fails fast on overflow self.queue = asyncio.Queue(maxsize=100) # ~1.5 MB at 224x224x3 uint8 **Failure walkthrough:** 1. Traffic spike hits 1200 fps and the queue swells to 800K frames (14 GB). 2. The kernel invokes swap thrashing until the OOM killer terminates the process. 3. **Lesson:** Derive `maxsize` from `(available_RAM / frame_size) * safety_factor`. Never guess. ### Sin 2: Redundant Allocations Each frame went through four separate copies: OpenCV BGR, Pillow RGB, NumPy

My vision a lightweight, permission-headache-free Docker setup for running OpenCode, uv, and untrusted Python code without risking your host OS. When contributing to unfamiliar open-source projects or letting AI coding agents (like OpenCode ) run terminal commands, there's always a slight hesitation. What if a build script touches my system Python, or a rogue command wipes host files? To solve this, I built saferun a zero-trust, disposable Docker sandbox designed specifically for Python developers and AI agent workflows on macOS and Linux. Here’s how it works, the permission nightmares I had to solve, and how you can set it up in under two minutes. The Goal I wanted a workspace that gave me: Absolute Isolation: Runtime scripts, pytest , ruff , and AI agent commands execute strictly inside a disposable Linux container. Seamless IDE Integration: Files edited inside PyCharm or VS Code on the host machine sync instantly with the container. Zero Permission Headaches: Any files generated inside the sandbox belong to my host user account—not root . Persistent Speed: Package downloads cached permanently via uv so environment startup stays millisecond-fast. Isolated Credentials: Global SSH and Git keys remain safely on the host machine. Solving the "Non-Root" Docker Nightmare The hardest part of containerized dev environments is file ownership. If you run Docker as root , any file your AI agent generates belongs to root , locking you out on your host machine. If you pass your local user ID ( -u "$(id -u):$(id -g)" ), Docker mounts non-existent directories as root:root , causing Permission Denied crashes when tools like uv try to write to cache folders. saferun solves this inside the base Dockerfile by pre-creating cache directories and granting open write permissions upfront: FROM python:3.12-slim # Install curl (needed to install OpenCode) RUN apt-get update && apt-get install -y --no-install-recommends \ curl \ && rm -rf /var/lib/apt/lists/ * # Install uv globally RUN pip

AI agent observability means capturing your agent's reasoning cycles, tool calls, and token usage as...

GitHub热门项目 | 🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data. | Stars: 16,944 | 668 stars this week | 语言: Python

GitHub热门项目 | Self-organizing AI second brain for Obsidian + Claude Code. Drop any source and Claude reads, links, and files it into one connected knowledge graph of plain Markdown you own. AI note-taking, personal knowledge management (PKM), and an open-source Notion alternative. Based on Karpathy's LLM Wiki pattern. | Stars: 11,516 | 272 stars today | 语言: Python

We Fixed the Eval Platform: The TypeError That Took Down Three Benchmark Pipelines At 3 AM, Sentry lit up with TypeError: Cannot read property 'map' of undefined . Three benchmark pipelines crashed. Not a memory leak, not a segfault, but a race condition hiding behind a TypeError, turning a high-stakes eval run into chaos. Here is how we resolved it, with no fluff. The Root Cause: Async Data Meets Blind Faith in .map() The error trace pointed to evaluator.ts:42 , where .map() assumed inputData.metrics would always exist. The junior dev tested with clean data, but in production, fetchBenchmarkData() (async) and evaluatePipeline() (sync) were racing . At 100+ RPS, metrics was often undefined . The Offending Code: const results = inputData . metrics . map ( metric => computeScore ( metric )); Why It Failed: Race Condition : inputData was fetched asynchronously, but evaluatePipeline() treated it as synchronous. OOM Risk : Unbounded .map() on 10K+ metrics could exhaust 8GB RAM. Worker Starvation : No concurrency limits led to thread pool exhaustion. The Fix: Guard Clauses, Bounded Queues, and Pragmatism Step 1: Fail Fast, Fail Loud Added zero-overhead runtime checks to reject bad data early: // eval-platform/core/evaluator.ts import { isNullOrUndefined } from ' ../utils/guards ' ; async function evaluatePipeline ( inputData : BenchmarkInput ): Promise < EvaluationResult > { if ( isNullOrUndefined ( inputData ?. metrics )) { throw new Error ( ' EVAL_400: metrics missing ' ); } // Proceed only if data is valid } Why? Stops TypeError crashes immediately. Cost: 1-2 CPU cycles. Negligible. Step 2: Chunked Processing for 8GB RAM Original code processed all metrics at once, causing OOM crashes. Fixed with 100-item chunks: const CHUNK_SIZE = 100 ; // 100 items ≈ 10MB peak memory const results : number [] = []; for ( let i = 0 ; i < inputData . metrics . length ; i += CHUNK_SIZE ) { const chunk = inputData . metrics . slice ( i , i + CHUNK_SIZE ); results . push (... chunk . map

ESP32 + Python: From Microcontroller to IoT Artcal 0: Introduction When it comes to transferring data from one place to another, things can sometimes become tricky, especially when communication happens between the hardware and software levels. In this article series, I would love to share the experience and knowledge I’ve gathered while working with ESP32 and Python. We’ll explore how these two technologies can work together, starting from the basics and gradually moving towards more interesting and practical projects. If you have any questions, suggestions, or ideas along the way, feel free to share them in the comments section below. I’d love to hear from you and discuss them with the community. So, without further ado, let’s begin! 🚀 What is ESP32? Think about Esp32 as a microcontroller with Internet facilities, consisting WiFi, Bluetooth and a own wireless data transfer protocol called ESP-NOW between ESP32 chips. Nowdays, the developers have made development boards integrading these chips for the easy use. ESP32 is a family of microcontrollers developed by Espresiff. This can read sensor inputs, process data, contol devices and specially connect to the internet. This is like Arduino but better, faster and smaller. With these information that we have, we can speak about this board as, "A powerful microcontroller that can interact with electronic components and communicate with other devices through Wi-Fi, Bluetooth, and other communication methods." Python??? We use different languages to tell the same thing but in different ways. We use programming languages to tell the computer the same thing but in different approches. Some languages can be hard to learn and some are easy. Some are well developed and some are not. Python programming language was created back in 1980s by Guido Van Rossum, with the development begining around 1989. It was publicly released in Feb, 1991. 🐍 1989 — Guido van Rossum developing Python. 🐍 1991 — The first public release. 🐍 2000 — Py

This is a submission for DEV's Summer Bug Smash: Clear the Lineup , powered by Sentry . Project Overview I was working on a small Python component that performs a preflight check and then, if the check succeeds, invokes one synchronous operation callback. A counter records whether that callback invocation returned normally. The counter is used for diagnostics, so it must follow the control flow rather than the expected happy path. Bug Fix or Performance Improvement When a handled failure occurred, the old implementation still returned one: return 1 That value was hard-coded because the successful path was expected to invoke exactly one operation. If the preflight check failed, however, the operation was never entered and the function still returned one. An offline reproduction produced: operation_entries=0 old_count=1 The failure was handled, but the counter contradicted the actual control flow. Code Reduced to the relevant lines, the old behavior was: # Simplified pre-fix behavior def buggy_completed_calls ( * , preflight , operation ): try : preflight () operation () except Exception : pass return 1 Here is the complete fixed function from the standalone reproducer: from collections.abc import Callable Callback = Callable [[], None ] def completed_calls ( * , preflight : Callback , operation : Callback ) -> int : """ Return one only when the cooperative operation returned normally. """ try : preflight () operation () except Exception : return 0 return 1 The essential regression assertion is shown below. Both callbacks are local, so the test performs no network request: # Abbreviated test excerpt def test_preflight_failure_does_not_count_an_unentered_operation (): operation_entries = 0 def refuse_preflight (): raise RuntimeError ( " controlled preflight refusal " ) def operation (): nonlocal operation_entries operation_entries += 1 result = completed_calls ( preflight = refuse_preflight , operation = operation , ) assert operation_entries == 0 assert result == 0 My

I made a small Vim 9.2+ plugin called thumb . Put the cursor on an image path → :Thumb → popup the image. Select LaTeX in Visual mode → :Thumb → render it as an image popup. For example:  Put the cursor on diagram.png and run: : Thumb Or select: \frac { a }{ b } = \sqrt { x ^ 2 + y ^ 2 } and run: : Thumb It uses Vim's native popup image support, with Python/Pillow for image conversion and matplotlib for LaTeX rendering. No mappings are installed, so you can add your own: nnoremap < leader > t < Cmd > Thumb < CR > xnoremap < leader > t < Cmd > Thumb < CR > GitHub: https://github.com/JosefAlbers/thumb Requires Vim 9.2+, Python 3, Pillow, and matplotlib. Feedback welcome, particularly around the popup positioning/rendering.

The model was never the problem, and that is exactly why the bug took three days to find. My ticket-classification service started returning the fallback label for long, non-English messages shortly after I moved the inference path to a cheaper endpoint, and every instinct pointed at the new model. The real culprit was a token-counting mismatch that silently truncated the prompt before the model ever saw the classification instruction. The Symptom The failure was remarkably consistent, which made it even more misleading. Messages under roughly two thousand characters classified correctly, while longer ones, especially in German and Japanese, fell through to a generic "other" bucket with a perfectly valid JSON response. The parser was not the issue, the prompt had not changed in weeks, and the retry logic never fired because the endpoint returned a normal 200 status. My first assumption was that the cheaper model was simply weaker at long-context reasoning, so I ran a controlled comparison using the same fifty tickets against the previous endpoint. The old path classified all fifty correctly, the new one failed on nineteen, and that result seemed to confirm the model-quality theory. What bothered me was the distribution: the failures clustered exactly where the input length crossed a threshold, and no ticket under that threshold ever failed. The Reproduction To isolate the variable, I needed a clean environment where I could swap endpoints without touching the production deployment, and MonkeyCode's free server option turned out to be a practical debugging tool. The project is open source, and its free model access let me replay the failing tickets without spending my own quota, so I spun up a disposable instance and pointed the same harness at the same prompt. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The reproduction took about twenty minutes, and the result was identical on every retry: long inputs failed, short inputs passed.