标签：#Python

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . The Initial Goal: "Upgrade and Secure" Like many developers, I recently fell into the trap of assuming that "smaller is always better, and newer is always safer." I decided to upgrade my terminal-based web UI project, py_terminal , to the bleeding-edge python:3.15-rc-alpine Docker base image. The logic was sound: Alpine Linux has a much smaller footprint, meaning a smaller attack surface. Python 3.15 Release Candidate would give me early access to performance improvements and patches. What followed was a cascading series of build failures that taught me a valuable lesson about container architecture, Python's C-API, and what actually makes a container secure. The Descent into Dependency Hell The moment I pushed the Dockerfile update and ran docker build , the pipeline exploded. 1. The Missing Wheels The first error was abrupt: ERROR: No matching distribution found for litellm==1.93.0 Because I was combining a release candidate of Python (3.15-rc) with Alpine (which uses musl libc instead of the standard glibc ), pre-compiled binaries (wheels) simply didn't exist for several of my packages. pip was forced to download raw source code and build from scratch. 2. The Rust Compiler (Wait, Rust?) One of litellm 's underlying dependencies is fastuuid , which is written in Rust. Because pip was building from source, it attempted to download the Rust toolchain ( cargo ). It immediately failed: Error loading shared library libgcc_s.so.1: No such file or directory Because Alpine is so incredibly stripped down, it didn't even have the basic C runtime library ( libgcc ) required to run the Rust compiler. 3. Fighting the PyO3 API Determined to win, I added the heavy build tools to Alpine ( apk add build-base cargo libffi-dev ). The build got further, but then crashed while compiling tiktoken and pydantic-core . The bridge between Rust and Python is handled by a library called PyO3 . It explicitly re

What to know about python static file security when using StayPresent's web.html/markdown — directory exposure, path traversal, and URL filtering. Security Notes for Serving Static Files with StayPresent Serving a status dashboard or a rendered README with web.html() / web.markdown() is convenient precisely because it automatically picks up neighboring CSS, JS, and images with no extra configuration. That same convenience has a security dimension worth understanding clearly before you point it at a directory. This covers python static file security as it applies specifically to StayPresent: what's protected automatically, and what's still your responsibility to manage. Table of Contents The Directory-Wide Exposure Behavior Why This Is Intentional Path Traversal Protection The One-Time Directory Warning Markdown-Specific Protections: Escaping Markdown-Specific Protections: URL Scheme Filtering What's Rejected vs What's Allowed Structuring Directories Safely Full Example Best Practices Common Mistakes FAQs Conclusion The Directory-Wide Exposure Behavior When you call web.html("templates/index.html") or web.markdown("docs/guide.md") , StayPresent doesn't just serve that one file — it serves every file in that file's directory , not only the specific CSS/JS/image files actually referenced from the page. This is what makes relative asset links ( href="style.css" , src="images/logo.png" ) work automatically without any extra configuration on your part. The consequence: if a .env file, your bot's own source code, or a .git/ directory happens to sit in that same directory, it becomes downloadable by anyone who requests it by name — whether or not anything on the page actually links to it. templates/ ├── index.html ├── style.css <- intentionally public, referenced from index.html ├── .env <- NOT referenced anywhere, but still reachable Why This Is Intentional This isn't an oversight — it's what makes web.html() / web.markdown() usable with zero configuration for the overwhel

A guide to python isolated logging with StayPresent's dedicated logger — no root logger mutation, what gets logged, and how to configure it. How StayPresent's Logging Works (Without Breaking Yours) A surprisingly common way for a third-party package to quietly break your application's logging is by calling logging.basicConfig() somewhere in its own code — which mutates the root logger and can silently change formatting, duplicate output, or override handlers you already configured for your own loggers. StayPresent avoids this entirely through python isolated logging : everything it logs goes through its own dedicated logger, never the root one. Table of Contents The Problem with logging.basicConfig() StayPresent's Dedicated Logger What Gets Logged, and at What Level Adjusting Verbosity Attaching Your Own Handler Logging During Multi-Bot Runs Logging During Shutdown Full Example Best Practices Common Mistakes FAQs Conclusion The Problem with logging.basicConfig() logging.basicConfig() configures the root logger, which every other logger in your process falls back to unless it's explicitly configured otherwise. If your bot calls it once at startup, and a dependency somewhere else in your stack calls it again, whichever call happens first usually "wins" silently — no error, just unexpected formatting or duplicate log lines that are hard to trace back to their cause. A well-behaved library avoids touching the root logger at all, and instead logs through its own named logger. StayPresent's Dedicated Logger StayPresent logs exclusively through a logger named "staypresent" , configured with a single dedicated StreamHandler and logger.propagate = False . It never calls logging.basicConfig() , and it never touches the root logger in any way. This means it cannot clobber, duplicate, or reformat log output your own script has already configured for its own, unrelated loggers — StayPresent's logs and your bot's logs coexist without interfering with each other. What Gets Logged,

Key takeaways A request is judged on many layers at once — IP reputation, TLS fingerprint, HTTP/2 shape, headers, and how the browser is driven — and failing any one is enough for a 403. Your laptop passes because every layer is consistent with a real home browser; a server changes one (usually the IP) and the inconsistency is the tell. A 403 with no challenge page almost always means you were blocked at the network layer (IP/ASN reputation or TLS/JA3 fingerprint) before any HTML was served — not a credentials or rate-limit bug, so 'add a User-Agent' or 'slow down' won't fix it. The fix order that actually works: get off datacenter IPs (residential/ISP proxies), match a real browser's TLS fingerprint, and spin up a real-browser stealth setup only for the pages that truly need JavaScript — escalate, don't lead with a browser. A proxy only changes your IP; a Linux VPS still leaks a Linux-shaped TLS/JS fingerprint, so 'residential IP + datacenter everything-else' is a contradiction a real home machine never makes — which is why a proxied server can get blocked harder than your laptop. Your scraper runs perfectly on your laptop. You deploy it to a VPS or a CI runner, change nothing in the code, and suddenly every request comes back 403 . It feels like a bug — the code is identical — but it usually isn't. Anti-bot systems judge a request on many signals at once, and moving from your home machine to a datacenter flips several of them at the same time. This post breaks down exactly which signals change, how to tell which one is blocking you, and how to fix it — for authorized access to public data (we'll keep that framing honest throughout; nothing here is about defeating a protection). A request is judged in two stages It helps to know that detection happens in two stages: Stage 1 — before any HTML is served. IP reputation, your TLS handshake, your HTTP/2 settings, and header order are all inspected on the connection itself, passively and cheaply, before your request is e

On May 20, 2026, AWS ended support for DMS Fleet Advisor. Fleet Advisor answered a question every migration team asks first: what is actually in my database estate, and how hard will it be to move? It was free. It was fully managed. It was backed by the largest cloud provider on earth. It still lost. AWS's official notice says only: "After careful consideration, we decided to end support for AWS DMS Fleet Advisor." No reason given. But you don't need one — the documentation tells you. Here is what Fleet Advisor required before it would tell you a single thing about your databases: Install a standalone data collector in your local environment Create an Amazon S3 bucket Create IAM policies, roles, and users — via CloudFormation, which was the recommended path Create database users with the minimum required permissions on every source Establish network access from the collector to each database server Then you'd meet the ceilings: recommendations for up to 100 databases at a time, one-to-one target mapping only, no multitenant server support. Now picture running that gauntlet inside a bank. You are a Business Solution Architect. You have been asked to scope a migration. You do not yet have approval for the migration — that approval is what the assessment is for . And to produce the assessment, you must first request production database credentials, get an agent binary through software approval, provision an S3 bucket, and get an IAM stack past a security review. That is a six-week procurement conversation to answer a question you were hoping to answer this week. AWS's replacement recommendation is Migration Evaluator — a consulting-led engagement. Read that as the finding it is: AWS looked at self-serve migration assessment, and concluded that humans and services do it better than a product. I think they were half right. And the half they got wrong is the interesting part. The lesson: friction is a competitor, and it usually wins We talk about developer tools as if the

Why this combination keeps winning for IoT telemetry backends — not in a benchmark, but against flaky gateways, replayed data, and firmware that never quite agrees with itself. If you’ve ever built the backend for a fleet of IoT devices — sensors, gateways, industrial equipment reporting temperature, humidity, GPS, battery, signal strength — you’ve faced the same fork in the road early on: what do you build the ingestion layer with, and what do you store the data in? After building a telemetry backend from scratch for a real fleet of LoRa/BLE sensors and gateways — handling dual ingestion paths, binary and JSON payload formats, automatic recovery of lost data, and a growing set of operational dashboards — I keep coming back to the same combination: Python (FastAPI + asyncio) for the API, MQTT for device transport, and PostgreSQL for storage. Here’s why that combination holds up so well for this specific problem, not just “in general.” Full Article: https://medium.com/@jackpelorus/python-postgresql-and-mqtt-the-boring-stack-that-actually-survives-a-real-device-fleet-9297146cbe8d?sharedUserId=jackpelorus

When building local RAG (Retrieval-Augmented Generation) applications, edge agents, or serverless AI pipelines, developers usually hit a wall with standard vector stores: memory overhead. Running a dedicated vector database locally often demands hundreds of megabytes—or gigabytes—of RAM just to keep indices warm. On the flip side, lightweight local options like scanning raw JSON files or querying SQLite don't scale well when vector dimensions climb into the thousands (1536d+). We built Bitweave to solve this exact trade-off: a zero-copy, SIMD-accelerated hybrid retrieval engine in Rust (with Python bindings) that handles categorical filtering and vector search while locking its active heap footprint under 1.1 MB RSS. The Architecture: How Bitweave Achieves Sub-Millisecond Speed at <1.1 MB RAM Bitweave relies on a 3-part design to maximize search speed while keeping memory consumption negligible: [ Categorical Filters ] ---> Bit-Sliced Bitmaps │ ▼ [ Query Vector (1536d) ] --> 1-Bit SIMD Pre-Filtering (Hamming Distance) │ (Top K Candidates) ▼ [ Raw Embeddings Buffer ] -> Zero-Copy Float32 Rescoring (exact_rescore=True) │ ▼ Top-K Results Array (NumPy) Zero-Copy Memory Mapping (memmap2) Instead of deserializing index files into Python RAM or Rust heap space, Bitweave uses memory-mapped files (.bweave). The operating system's page cache handles lazy loading of index segments directly from disk into virtual address space. As a result, the active RSS memory footprint remains static around 1.1 MB, whether your index holds 5,000 or 200,000 records. 1-Bit Vector Quantization & SIMD Hamming Distance High-dimensional float32 vectors (1536d) are quantized down to 1-bit sign masks (where values > 0 map to 1 and <= 0 map to 0). During pre-ranking, Bitweave uses SIMD bitwise XOR and POPCNT operations to compute Hamming distances across candidate vectors in microseconds. Zero-Copy 2-Pass Float32 Rescoring (exact_rescore=True) Quantization speeds up initial candidate selection, but f

GitHub热门项目 | TokenSpeed is a speed-of-light LLM inference engine. | Stars: 1,745 | 53 stars today | 语言: Python

GitHub热门项目 | SGLang is a high-performance serving framework for large language models and multimodal models. | Stars: 30,917 | 73 stars today | 语言: Python

GitHub热门项目 | Deepfakes Software For All | Stars: 56,008 | 135 stars today | 语言: Python

OpenWorker shipped in late July 2026. It is MIT-licensed, runs on your own machine, and takes your API key instead of selling you inference. The pitch is narrow and worth repeating exactly: it is an agent that hands you finished work , not a chat transcript. A drafted document on disk. A Slack reply with the real numbers in it. A calendar that has actually been rearranged. There are a lot of desktop agents right now. This post is about what makes this one structurally different, what state it is actually in, and how to get it running. What it is in one paragraph OpenWorker is a desktop app: a Tauri shell around a React UI, sitting on top of a local Python agent server. You give it an outcome ("prepare a customer brief from these three files and the Jira tickets"). It decomposes that into steps, reaches into your files, terminal, and connected SaaS apps, and produces an artifact. Before anything consequential happens - sending a message, running a shell command, writing to your calendar - it stops and asks. The engine is built on aisuite , Ng's provider-agnostic LLM library. That matters more than it sounds like: OpenWorker is explicitly positioned as a reference implementation of what you can build on aisuite, so the codebase doubles as a worked example if you are building your own harness. The four things that actually distinguish it 1. There is no OpenWorker inference service You paste a key, or you point it at Ollama and use none at all. The curated list covers OpenAI, Anthropic, Google, plus OpenAI-compatible vendors like DeepSeek, GLM, Kimi, Qwen, MiniMax, Mistral, and Grok, plus open-weight models through Together and Fireworks. Roughly thirty models are marked as verified for tool-calling work; you can point it at any other model string and accept the risk yourself. The practical consequence: your cost is your provider bill, and swapping models is a dropdown, not a migration. 2. The permission model is typed, not a confirmation dialog This is the part I would

Rate limiting is one of those features every production API eventually needs. Whether you're building a public REST API, a WebSocket service, or an authentication endpoint, you'll eventually face problems like: Credential stuffing Brute-force attacks API abuse Bots scraping your endpoints Unexpected traffic spikes Most applications solve this with a simple request counter. But after building several APIs with Django, FastAPI, and Flask, I realized that production traffic requires much more than "X requests per minute." That observation led me to build drogue , an open-source Python library for rate limiting and traffic protection. The Problem Traditional rate limiting is straightforward: Allow 100 requests per minute. This works well for many cases, but real-world applications quickly expose its limitations. For example: A distributed attack can remain below the per-IP limit. A bot can rotate through proxies. WebSocket connections often require different handling than HTTP requests. Different endpoints need different protection strategies. I wanted a system that could go beyond simple request counting. Design Goals From the beginning, I focused on a few principles. 1. Clean framework integration I didn't want endpoint functions filled with framework-specific plumbing. Instead, the library should feel like a natural extension of the framework. from fastapi import FastAPI from drogue.adapters.fastapi import DrogueLimiter app = FastAPI () limiter = DrogueLimiter ( app , default_limits = [ " 100/minute " ]) @app.get ( " /users " ) @limiter.limit ( " 10/minute " ) async def users (): return { " status " : " ok " } No additional request objects. No complicated middleware configuration. Minimal boilerplate. Multiple Rate Limiting Algorithms Different applications require different algorithms. Instead of supporting only one approach, drogue includes multiple options: Token Bucket Sliding Window Fixed Window Each has different trade-offs between accuracy, burst handling, and

Automated offense has one embarrassing failure mode: it lies to you about winning. Point a tool at a target, and the naive success check is a substring match — see uid=0(root) in the response, call it a shell. But a service banner can print that. A tarpit can stream it on connect. And the moment your success signal is wrong, everything downstream inherits the lie: the report, the "which hosts are owned" state, the next move. You get a confident engine that's confidently wrong. Here's how I made mine prove it instead — what worked in a live run today, and the sharp reader feedback that already made the design better. The idea: make the target echo a secret it couldn't have guessed Borrow the oldest trick in authentication. Before each attempt, the orchestrator mints an unpredictable per-attempt nonce and injects it. The delivered command has to send that nonce back: import os def make_nonce() -> str: return os.urandom(12).hex() # unpredictable — a target can't guess it A result is only trusted if that exact nonce comes back, in a structured evidence line: HALO-EVIDENCE nonce=c609007176813c9110fccc27 level=shell uid=0 host= exit=0 _EVIDENCE = re.compile(r"HALO-EVIDENCE nonce=(\S+) level=(\S+)") def breach_confirmed(output, ok, *, nonce) -> bool: m = _EVIDENCE.search(output or "") return bool(ok and m and m.group(1) == nonce) Delivery is a ladder, because real hosts are inconsistent Proof is worthless if you can't deliver a payload. So delivery degrades gracefully — all stdlib socket: Reverse shell — target dials back to an ephemeral listener, announces the nonce, hands back /bin/sh. Bind shell — if egress is blocked, the target binds a shell and you connect in. Blind callback — if no interactive channel survives, the target just connects back and sends the nonce. That still proves code execution, with no usable shell. Each rung self-selects the first available interpreter (bash /dev/tcp, python3, perl, nc), so the same primitive works against arbitrary hosts, not one

Cloud AI coding tools are powerful, but they also come with a bill, an API key, and a quiet upload of your source code to someone else's servers. What if your AI assistant ran entirely on your own machine instead? That is FLASH CLI (Fast Local Agent SHell): an AI-powered command-line assistant that talks to local or self-hosted Ollama models and can actually run shell commands for you. No API key. No subscription. No cloud. Why FLASH is different 100% local. It connects to an Ollama server, by default on localhost, so your code and prompts stay on your hardware. No keys, no bill. Ollama needs no API key, so there is nothing to pay for and nothing to leak. Truly agentic. FLASH does not just chat. It runs a real tool loop: it inspects your system, runs commands, searches the web, and shows its reasoning as it works. Model freedom. Point it at llama3.1, qwen2.5, mistral, or any tool-capable Ollama model, and swap with one setting. Local or remote. Set OLLAMA_HOST and the same client talks to a GPU box on your network or a server behind a reverse proxy. The four main tools FLASH gives the model a tight, powerful toolset: shell: run any command, non-interactively, with a timeout. web_search: pull live results from DuckDuckGo, built in. get_os: detect the operating system so it picks the right command every time. reason: surface a line of its thinking without ending the turn. It loops through plan, act, and observe until the job is done, then answers in rendered Markdown with syntax highlighting. Configuration Optional configuration lives in ~/.flash.env: MODEL=llama3.1 OLLAMA_HOST=http://localhost:11434 What it feels like Ask a question and let it work: [Flash]> what are the biggest files here? Thinking: check the OS, then find the largest files Retrieving operating system information Executing shell command: du -ah . | sort -rh | head -3 … Run a command yourself with the ! prefix, no AI in the loop: [Flash]> !git status On branch main nothing to commit, working tree cle

This is the architecture of Shibui Finance , an MCP server that gives Claude direct SQL access to 64 years of US stock market data. About 10,000 symbols, 31 million daily price records, quarterly financials back to 1990, 56 pre-computed technical indicators, and 6.4 million SEC filing records. Free to use. Stack: Python, PostgreSQL, dbt, DuckDB, FastMCP, Caddy. Runs on a single VPS. Data pipeline Three stages: ingest into PostgreSQL, transform with dbt, export to DuckDB. Data APIs / SEC EDGAR / FRED | Python ETL (Polars, ADBC) | PostgreSQL clean_* schemas (~50 raw tables) | dbt (27 models) staging -> integration schema (17 analytical tables) | DuckDB export (daily, ~14 GB file) | FastMCP server (read-only, streamable-http) | Caddy (TLS) -> mcp.shibui.finance Multiple sources feed the pipeline: commercial data APIs for prices, fundamentals, valuations, and estimates. SEC EDGAR for filing metadata and insider transactions (bulk historical + a 5-minute Atom feed for near-real-time). FRED for FX rates to normalize non-USD fundamentals. Public registries for ticker classification. The ETL is a Python CLI organized by data source. Each module has its own fetcher, loader, and CLI. A single all command runs everything in fixed sequence. You can't refresh 10,000 tickers daily without hitting rate limits, so the ETL rotates: each run refreshes the stalest 5% of tickers. Full universe cycles in about 20 runs. Recent prices always refresh on every run. Every table write is a single transaction. DROP + CREATE inside a transaction, rollback on failure. The database never serves partial data, and dbt always sees complete tables even when ingest jobs overlap. The dbt layer 27 models in two tiers. The process layer handles standardization: enriching symbols with security types and exchange mappings, linking SEC amendment filings to their originals, repairing filer date typos. The integration layer produces the 17 tables that Claude actually queries. This is where raw normalized tabl

GitHub热门项目 | Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills. | Stars: 27,552 | 180 stars today | 语言: Python

GitHub热门项目 | "OpenSpace: The Skill Management Layer for AI Agents" -- https://open-space.cloud/ | Stars: 7,129 | 91 stars today | 语言: Python

GitHub热门项目 | A curated list of awesome libraries, packages, strategies, books, blogs, tutorials for systematic trading. | Stars: 9,199 | 113 stars today | 语言: Python

Every tool in the East Africa coordination infrastructure stack started from the same scaffold. Here's exactly how to build and publish one yourself. What You're Building An MCP server is a Python package that exposes tools to AI assistants. When a user installs it and connects it to Claude, the AI can call your tools as naturally as answering a question. pip install your-mcp-server # Then Claude can: # "Check NHIF coverage for outpatient surgery" → calls your tool → returns structured result Step 1: Set Up the Project (2 min) your-mcp-server/ ├── src/ │ └── your_package/ │ ├── __init__.py │ └── main.py ├── pyproject.toml ├── README.md └── .github/ └── workflows/ └── publish.yml mkdir your-mcp-server && cd your-mcp-server mkdir -p src/your_package touch src/your_package/__init__.py src/your_package/main.py Step 2: Write Your Tool (10 min) # src/your_package/main.py from __future__ import annotations from typing import Annotated from fastmcp import FastMCP mcp = FastMCP ( name = " your-mcp-server " , instructions = " Describe what your server does in one paragraph. " , ) @mcp.tool ( description = ( " What this tool does in plain language. " " Include the Western parallel if applicable. " " Note if it uses DEMO data. " ) ) def your_tool ( param1 : Annotated [ str , " Description of param1 " ], param2 : Annotated [ int , " Description of param2 " ] = 0 , ) -> dict : # Your logic here return { " result " : f " Processed { param1 } " , " note " : " DEMO — replace with real data source in production " , " source " : " your-mcp-server " , } def main (): mcp . run () if __name__ == " __main__ " : main () Step 3: Configure pyproject.toml (3 min) [build-system] requires = ["setuptools> = 61.0 "] build-backend = "setuptools.build_meta" # ← exact string, no variation [project] name = "your-mcp-server" version = "0.1.0" description = "One-line description" authors = [{name = "Your Name" , email = "you@example.com" }] license = { text = "MIT" } readme = "README.md" requires-pytho

LangGraph isn't cheaper than LangChain — unless you opt out of its defaults Cost-audit series, episode 4. This series began with an AI agent that burned 136M tokens overnight → . When LangChain deprecated ConversationBufferMemory (the subject of episode 1 in this series), the official migration path was LangGraph. The pitch: explicit state management, you control exactly what flows where. More expressive, more controllable. It is — but only if you reach for the controls. The default state model in LangGraph has the same unbounded-growth problem as the memory it replaced. Teams migrating to escape ConversationBufferMemory's cost curve often land on an identical curve, with new graph complexity on top. This audit shows exactly where the default grows, what it costs, and what opt-outs exist. The default: MessagesState + add_messages The quickstart in LangGraph's own docs uses this pattern: from langgraph.graph import StateGraph , MessagesState def my_node ( state : MessagesState ): messages = state [ " messages " ] response = llm . invoke ( messages ) # sends ALL messages to the LLM return { " messages " : [ response ]} graph = StateGraph ( MessagesState ) graph . add_node ( " agent " , my_node ) MessagesState is a TypedDict with a single key, messages , backed by the add_messages reducer. Here's what that reducer does: # langgraph/graph/message.py — add_messages (def at line 18; merge loop below) def add_messages ( left : Messages , right : Messages ) -> Messages : # ... (coerces left/right to lists of BaseMessage) ... left_idx_by_id = { m . id : i for i , m in enumerate ( left )} merged = left . copy () ids_to_remove = set () for m in right : if ( existing_idx : = left_idx_by_id . get ( m . id )) is not None : if isinstance ( m , RemoveMessage ): ids_to_remove . add ( m . id ) else : merged [ existing_idx ] = m # same id → update in place else : merged . append ( m ) # new id → APPEND (the list grows) merged = [ m for m in merged if m . id not in ids_to_remove ] retu