标签：#Python

The .txt File as the Soul of a Personal AI — FileRAG Memory Architecture By Dharanidharan J (JD) Full Stack & AI Engineer | Building Jarvix The Problem Nobody Talks About Every chatbot tutorial teaches you the same thing: history = [] history . append ({ " role " : " user " , " content " : message }) And that works — until it doesn't. After 500 turns, your dict has forgotten who the user is. After 1000 turns, you're hitting token limits. After a restart, everything is gone. Redis helps with persistence but still buries early facts under noise. Vector DBs help with retrieval but bloat storage and need infrastructure. What if the memory itself was just a file? The Idea Every conversation a user has gets distilled into a plain .txt file. That file is the brain. On every new query, a hybrid BM25 + semantic RAG retrieves the most relevant chunks from it and injects them as context. users/ └── jd.txt ← the soul file The soul file looks like this: [Turns 1-5] - User's name is JD, software engineer - Building FileRAG, a novel memory architecture - Uses Pop!_OS with Fish shell and NVIDIA GPU [Turns 6-10] - Has a cat named Pixel who distracts during coding - Paused TaskNest due to burnout - Now focused on AgenticMesh Human readable. Editable. Yours. Why This Is Different Most memory systems store messages . FileRAG stores a relationship . System What it stores Dict / Redis Raw message objects Vector DB Embeddings of messages FileRAG Distilled understanding of the user The longer you use it, the more the AI understands you — not because it has more messages, but because it has a better summary of who you are. The Architecture User message ↓ Topic drift check (cosine similarity) ├── Drift detected → distill current buffer immediately └── No drift → continue ↓ Hybrid retrieval (BM25 + ChromaDB) from soul file ↓ Inject context → LLM responds ↓ Append to turn buffer ↓ Every 5 turns → distill → append to soul file → update ChromaDB ↓ Emergency distillation on exit (SIGINT/SIGTERM)

I came across Scrapling through a recommendation on X and decided to put it through its paces — not against a demo page, but against Lazada Singapore, a production site with Google reCAPTCHA and a custom slider verification. The setup: a single 4GB VPS, no residential proxies, no credits, just open-source tools. Here's the full journey: installation pitfalls, wiring it into an AI agent, choosing the right browser for the job, and the real-world benchmarks that followed. What Is Scrapling? Scrapling is an adaptive web scraping framework for Python (BSD-3, v0.4.8). It handles everything from single HTTP requests to full-scale concurrent crawls. What sets it apart from the BeautifulSoup/Scrapy world: Adaptive element tracking — saves fingerprints of targeted elements and relocates them after site redesigns using similarity scoring. Your scrapers survive CSS changes without maintenance. Three fetchers, one API — HTTP ( Fetcher , curl_cffi), browser ( DynamicFetcher , Playwright Chromium), and stealth ( StealthyFetcher , Chromium + anti-bot patches). Swap with one line. Spider framework — Scrapy-like API with async, concurrent crawling, Ctrl+C pause/resume via checkpoint persistence, multi-session support. MCP server — 14 tools exposed natively for AI coding agents. Your agent can call mcp_scrapling_get , mcp_scrapling_fetch , mcp_scrapling_stealthy_fetch directly. It's open source, pip-installable, and designed to be the backbone of a scraping stack — not just another tool in the toolbox. Installation on a 4GB VPS This is where the real story starts. The VPS has 4GB RAM, 2 vCPUs, 77GB disk, and runs an AI agent gateway (615MB baseline). Every browser installation decision matters. What we installed pip install scrapling[fetchers,ai] # HTTP + Chromium + MCP server scrapling install # Downloads Playwright browsers This pulls in Playwright Chromium, Firefox, and WebKit (~1.3GB disk), plus curl_cffi for HTTP requests and patchright (Playwright fork) for browser automation.

If you manage PostgreSQL databases across multiple environments, you've probably done this: SSH to the DB host (or connect via psql ) Run CREATE USER jsmith CONNECTION LIMIT 20 PASSWORD '...' Slack the password to the developer Forget to log it anywhere Repeat for every environment, every onboarding, every access request It's tedious, error-prone, and leaves zero audit trail. Here's a better way. What I Built pg-user-api is a lightweight Flask REST API that wraps PostgreSQL user provisioning in clean HTTP endpoints. You register your databases once in a SQLite inventory, then any tooling — CI pipelines, internal portals, Ansible playbooks, or a plain curl — can create and manage users across environments without ever touching psql . GitHub: pcraavi/PostgreSQL-user-creation-API The Problem It Solves In teams that span dev, QA, UAT, and prod, you end up with different patterns of users: App service accounts — named after the host/port combo ( web01_8080 ) Kubernetes workload accounts — named after env prefix + farm ( dv_gearservice ) Individual dev/QA accounts — low connection limits, scoped to non-prod Read-only analyst accounts — prod only, no DDL DBA accounts — CREATEDB CREATEROLE LOGIN , rarely provisioned Each type has different CONNECTION LIMIT values, privilege levels, and naming conventions. Encoding these patterns in an API means the rules are consistent, repeatable, and auditable. Architecture The project is intentionally small — five Python files and a requirements list: pg_user_api/ ├── app.py # Flask app — all endpoints ├── auth.py # HTTP Basic Auth (constant-time compare) ├── database.py # SQLite registry + audit log ├── notifications.py # Notification stubs (Webex / Slack / Email) ├── seed_db.py # One-time setup: creates DB + sample records └── requirements.txt Two credential pairs, clearly separated: PG_API_USER / PG_API_PASS — who can call this API (your team/tooling) PG_ADMIN_USER / PG_ADMIN_PASS — the PostgreSQL DBA role that executes DDL The DBA cr

Anthropic's Mythos model can autonomously find zero-day vulnerabilities. Their CVD disclosure process uses manual SHA-3-512 hash commitments to prove findings existed. I built something that automates that in one line of Python. What AetherProof does One function call generates a 128-byte Ed25519-signed receipt that proves: What model ran — FNV-1a hash of provider/model ID What it produced — hash of the output When — cryptographic nanosecond timestamp Tamper-evident — flip any byte anywhere → INVALID python import aetherproof receipt = aetherproof.for_anthropic( "Find vulnerabilities in this binary.", finding_text, model="claude-mythos-preview" ) receipt.save("CVE-2026-001.receipt") print(receipt.verify()) # True Try it in 30 seconds pip install aetherproof python -c " import aetherproof r = aetherproof.for_anthropic('question', 'answer') print(r.verify()) # True print(r.pretty()) " The unusual part — invisible Unicode watermarking Receipts embed invisibly into any text using Unicode Private Use Area codepoints (U+E000–U+E0FF). AI output carries its own audit trail. Works in any language — Arabic, Chinese, Devanagari, Hebrew, Thai, Japanese all tested. signed_output = aetherproof.embed(ai_response, receipt.to_bytes()) # Text looks identical. Receipt is inside. aetherproof.verify_embedded(signed_output) # True Numbers 187 tests, 0 failures 128/128 byte flips all detected 1000/1000 tamper probes pass Cross-language: Python generates, Rust CLI verifies 15,446 receipts/sec (Python) · 5,472/sec (Rust) Why AGPL-3.0 Free for open source. Commercial use needs a license. This is the compliance layer under your AI stack — it should be open, auditable, and not vendor-locked. GitHub https://github.com/pulkit6732/aetherproof Built by Pulkit. Feedback welcome.

What Happened Last month, the UK Government's AI Safety Institute merged AgentThreatBench into their official inspect_evals framework — the same framework they use to evaluate frontier AI models from OpenAI, Anthropic, and Google DeepMind. AgentThreatBench is an open-source adversarial benchmark I built that contains 200+ attack payloads specifically designed to test whether AI agents can resist memory poisoning attacks. Why This Matters AI agents are increasingly being deployed with persistent memory — they remember past conversations, user preferences, and context across sessions. This creates a new attack surface: memory poisoning . An attacker who can inject malicious content into an agent's memory can: Exfiltrate sensitive data on subsequent sessions Override safety instructions persistently Manipulate agent behavior without the user's knowledge The OWASP Agentic Security Initiative identified this as ASI06 — Agent Memory Poisoning . What AgentThreatBench Tests The benchmark covers 5 attack categories: Category Payloads Description Prompt Injection 40+ Instructions disguised as memory content Protected Key Tampering 40+ Attempts to overwrite system-level keys Sensitive Data Leakage 40+ PII/credential exfiltration via memory Size Anomaly 40+ Memory inflation / resource exhaustion Behavioral Drift 40+ Gradual personality/instruction shifts How to Use It pip install agentthreatbench # Run the full benchmark against your agent atb run --target your_agent_endpoint --output results.json # Or use individual attack categories atb run --category prompt_injection --target your_agent_endpoint The BEIS Validation The UK Government's AI Safety Institute uses inspect_evals to: Evaluate frontier models before deployment decisions Benchmark safety mitigations across providers Track regression in safety properties over time Having AgentThreatBench merged into this framework means it's now part of the official government toolkit for AI safety evaluation. Links GitHub : github.co

GitHub热门项目 | Memory library for building stateful agents | Stars: 4,467 | 599 stars this week | 语言: Python

GitHub热门项目 | Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages. | Stars: 78,940 | 148 stars today | 语言: Python

GitHub热门项目 | PentestAgent is an AI agent framework for black-box security testing, supporting bug bounty, red-team, and penetration testing workflows. | Stars: 2,489 | 30 stars today | 语言: Python

GitHub热门项目 | Deep Eye orchestrates multiple AI providers (OpenAI, Claude, Grok, Gemini, OLLAMA, Groq, Mistral, OpenRouter, LiteLLM, LM Studio) for intelligent payload generation, scans targets for 45+ vulnerability types, and produces professional reports with compliance mapping. | Stars: 1,562 | 42 stars today | 语言: Python

GitHub热门项目 | 自动化上传视频到社交媒体：抖音、小红书、视频号、tiktok、youtube、bilibili | Stars: 11,582 | 185 stars today | 语言: Python

GitHub热门项目 | A platform for reproducible world model research and evaluation | Stars: 1,058 | 346 stars today | 语言: Python

When I first connected to a precious metals WebSocket API, I expected to get a clean stream of prices. What I actually got was a firehose of mixed ticks—gold, silver, platinum—all arriving through the same callback. If you’ve ever tried to build a trading bot or a custom chart, you know this is a recipe for disaster. In this post, I’ll share how I solved the problem with a few lines of Python and a clear mapping strategy. The scenario: You have one WebSocket URL that pushes quotes for multiple metals. You need to separate them so you can update different UI components, run independent strategies, or store them in distinct database tables. The data pain point: every message uses the same JSON structure, and the only differentiator is a field like symbol . If you don’t act on it immediately, everything gets mixed up. Identify Assets via the Symbol Field Start by checking the API docs for the field that carries the instrument code. Usually it’s symbol , but instrumentId or type are also used. Here’s a typical reference table: Field Description Example symbol Asset code XAUUSD, XAGUSD instrumentId Internal platform ID 1001, 1002 type Asset class gold, silver I turn this into a dictionary mapping each symbol to a human-readable category: asset_map = { " XAUUSD " : " gold " , " XAGUSD " : " silver " , " XPTUSD " : " platinum " } Buffer Messages by Type Because these streams are high-frequency, I avoid processing every tick individually. Instead, the WebSocket callback just updates an in-memory store that is already grouped by asset type: # Keep the hot path extremely light def on_message ( msg ): symbol = msg [ ' symbol ' ] price = msg [ ' price ' ] asset_type = asset_map . get ( symbol , " unknown " ) cache [ asset_type ][ symbol ] = price Then, a background timer fetches the latest prices from cache["gold"] and cache["silver"] separately and does the actual work—like computing indicators or rendering charts. The key benefit is complete isolation: your gold logic never t

Welcome back to Day 3, Python dynamic duo! 🚀 If you survived Day 2 , you now know how to create variables and throw strings, integers, floats, and booleans into their own little cardboard boxes. 📦 But what happens when you’re building a game and your character needs an inventory? Or you're making a shopping list app? Creating 50 different variables like item1, item2, item3 will make you want to throw your router out the window. 🪟💻 Today, we are leveling up our storage game. We are moving out of single cardboard boxes and packing a Virtual Backpack: Enter Lists! 🎒🎉 🎒 What is a List? In Python, a List is a data structure used to store a collection of items in one single variable. Think of it like a backpack where you can stuff multiple things inside, keep them in a specific order, and pull them out whenever you need them. Creating a list is simple. You use square brackets [] and separate your items with commas: # Packing our survival backpack 🗺️ backpack = [ " map " , " flashlight " , " water bottle " , " protein bar " ] print ( backpack ) # Prints: ['map', 'flashlight', 'water bottle', 'protein bar'] The coolest part? Python lists don’t care what you put inside. You can mix strings, integers, and booleans all in one single backpack (though usually, it makes the most sense to keep similar things together). 🤯 The First Rule of Coding Club: We Start Counting at Zero! Here is where programming turns your brain upside down. 🧠🙃 If I asked you what the first item in our backpack list is, you’d logically say "map". And you'd be right in human language. But in Python-speak, computer memory starts counting at 0. This is called Indexing. To pull a specific item out of your backpack, you write the name of the list followed by the item's position (index) inside square brackets: backpack = [ " map " , " flashlight " , " water bottle " , " protein bar " ] # Pulling out the items using their index 🔍 print ( backpack [ 0 ]) # Prints: map (The absolute first item!) print ( backpack [

I wanted a soft gate on my resume download. Not a paywall. Just an email field — enough friction to filter bots, enough signal to know who's interested. What started as a straightforward feature turned into a three-part lesson: stateless token signing, S3 public access, and email delivery mechanics. Here's the full story. The Feature The flow I wanted: Visitor clicks "Download Resume" on the About page or Hero A modal asks for their email Backend validates the email (format + disposable domain check) A signed, time-limited link is emailed to them They click the link, the PDF opens No database tokens. No cron jobs. No permanent S3 URLs floating around. Part 1 — The Model and the Gate The Resume Model Resume follows the singleton pattern I already use for page headers — force pk=1 on every save, restrict add/delete in admin. One row, forever. class Resume ( models . Model ): pdf = models . FileField ( upload_to = " resume/ " , storage = private_resume_storage ) last_updated = models . DateField ( default = date . today ) def save ( self , * args , ** kwargs ): self . pk = 1 super (). save ( * args , ** kwargs ) ResumeDownloadRequest logs every email that requests a link — no tokens, no expiry columns, just a record of who asked and when. class ResumeDownloadRequest ( models . Model ): email = models . EmailField () created_at = models . DateTimeField ( auto_now_add = True ) unsubscribed = models . BooleanField ( default = False ) class Meta : ordering = [ " -created_at " ] The unsubscribed flag is there for a future newsletter broadcast — when a new blog post goes out, skip anyone who opted out. Blocking Disposable Emails Before signing anything, the email is checked against a frozenset of ~70 known throwaway domains: # core/validators.py DISPOSABLE_EMAIL_DOMAINS : frozenset [ str ] = frozenset ({ " mailinator.com " , " guerrillamail.com " , " yopmail.com " , " 10minutemail.com " , " trashmail.com " , # ... ~70 total }) def is_disposable_email ( email : str ) -> bool

Hey Dev.to the community, I'm Ashwin Gururaj — a Data Scientist & AI Engineer based in Melbourne, Australia, currently open to full-time, contract, and internship opportunities. I specialise in building production-grade AI systems — not just notebooks and demos, but end-to-end pipelines that actually run in production. What I work with: Python · LangChain · LangGraph · FastAPI · RAG pipelines · pgvector · Multi-agent systems · LLMs · Groq · HuggingFace · Pydantic · Docker · Celery · Redis · PostgreSQL · Data Science · SQL · Pandas · Scikit-learn What I've built recently: Sift — an open-source multi-agent fact-checking pipeline. Takes any text, extracts every factual claim, retrieves grounded evidence via HyDE RAG + live web search, and returns auditable verdicts with cited sources. Built with LangGraph, pgvector, FastAPI, and Docker. → GitHub Open to: Full-time Data Scientist / AI Engineer / ML Engineer roles Remote or Melbourne-based Companies building serious AI products If you're hiring or know someone who is — I'd genuinely appreciate a connection. GitHub: https://github.com/ashg2099 LinkedIn: https://www.linkedin.com/in/ashwin-gururaj-93943816a/ Thanks!

By Umair Sheikh, founder of Gateplex Autonomous AI agents are shipping fast. LangChain, CrewAI, AutoGen — the frameworks are mature, the tutorials are everywhere, and developers are connecting agents to real systems: databases, payment APIs, email, file storage. And then something goes wrong. Not because the code is buggy. Because the agent did exactly what it was told — and what it was told turned out to be a problem nobody anticipated. I spent nearly a decade in fintech and responsible AI policy watching this pattern repeat. A system behaves perfectly in testing. In production, an edge case triggers behaviour that was technically correct but operationally catastrophic. By the time anyone notices, the action has already executed. The problem is not the agent. The problem is that there is nothing between the agent's decision and the real world. The gap nobody talks about Most agent observability tools log what happened. That is useful for debugging. It does nothing to prevent the next incident. What agents actually need is a governance layer — something that intercepts every action before it executes, checks it against your rules, and either allows it, flags it for review, or blocks it outright. This is what a firewall does for network traffic. Your AI agent deserves the same treatment. What this looks like in practice Here is a simple LangChain agent calling an external tool: from langchain.agents import initialize_agent , Tool from langchain.llms import OpenAI def send_payment ( amount : str ) -> str : # This actually moves money return f " Payment of { amount } sent " tools = [ Tool ( name = " SendPayment " , func = send_payment , description = " Send a payment " )] agent = initialize_agent ( tools , OpenAI (), agent = " zero-shot-react-description " ) agent . run ( " Send $5000 to vendor account " ) This works. It also has no guardrails whatsoever. If the agent misreads the input, hallucinates a vendor, or gets manipulated via prompt injection, the payment goes

GitHub热门项目 | AI Agent Governance Toolkit — Policy enforcement, zero-trust identity, execution sandboxing, and reliability engineering for autonomous AI agents. Covers 10/10 OWASP Agentic Top 10. | Stars: 3,077 | 1,314 stars this week | 语言: Python

GitHub热门项目 | Official, Anthropic-managed directory of high quality Claude Code Plugins. | Stars: 28,330 | 7,703 stars this week | 语言: Python

GitHub热门项目 | Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands. | Stars: 127,228 | 323 stars today | 语言: Python

GitHub热门项目 | Public repository for Agent Skills | Stars: 142,410 | 686 stars today | 语言: Python