标签：#tutorial

Note: This tutorial was originally published on effessdev.github.io . Check out the original article for the most up-to-date version: https://effessdev.github.io/posts/2026-07-27/ This is a step-by-step tutorial that explains how you can set up your development environment for working with ESP-IDF projects in VS Code . Install ESP-IDF Install EIM Espressif Systems provides a graphical tool called EIM (ESP-IDF Installation Manager) to install ESP-IDF. Click the link below to go to the official page to download EIM: https://dl.espressif.com/dl/eim/ Make sure you are in the "Online Installer" tab. The exact file to download depends on your system: Windows: Download eim-gui-windows-x64.exe . Run this installer to install EIM. Linux x64 (Ubuntu): Download and install the .deb package ( eim-gui-linux-x64.deb ). Install ESP-IDF using EIM Now that we have installed EIM, let's install ESP-IDF using it. Open EIM. Under "New Installation" click "Start Installation". Under "Easy Installation", click "Start Easy Installation" to install the latest stable version of ESP-IDF with default settings. If there are no problems, you will see the "Ready to Install" page. Click "Start Installation". Install ESP-IDF VS Code Extension We use this extension as a high-level wrapper for ESP-IDF. Most times, we do not use ESP-IDF directly. For example, if we need to compile our source code, we ask the extension to do it, which uses the ESP-IDF we just installed internally to to compile the source code. Install the extension named "ESP-IDF" by "Espressif Systems" in VS Code. Verify installation After installing, restart VS Code. Use the shortcut Ctrl + Shift + P to open the command palette (remember this shortcut, we are going to use it a lot). Inside the command palette, search ESP-IDF . You will see many entries which start with ESP-IDF: . Those commands are provided my the ESP-IDF extension. These commands are what we use for almost everything. Note If you are not in an ESP-IDF project, you m

Twenty-one bytes. That is the entire attack. A file smaller than a URL, with four zero bytes sitting at exactly the right offset, crashes any FFmpeg-based application that opens it and reads a packet. Not memory corruption, not some exotic heap trick. A division by zero, in code that has been shipping for years, in one of the most fuzzed codebases on the planet. The person who found it, Darío Clavijo, did not write the fuzzer by hand. He built it with AI assistance, the way a growing number of security researchers now work, and posted the result on Hacker News this week under a title that got my attention immediately: "We found a division by zero bug in FFmpeg with a vibecoded fuzzer." The thread climbed past 250 points with hundreds of comments, and the debate underneath it is the real story: AI has been writing application code for two years, but AI writing the tester changes the economics of finding bugs in ways most teams have not priced in yet. Full disclosure before I go further. I am not a C security researcher. I run my own AI agent infrastructure and I write Java for a living. What I did for this article is what I would want you to do: I cloned the fuzzer's public repo, read its findings documents, tried to reproduce the crash on my own Ubuntu box, and studied the harness code line by line. Everything below is sourced from the public FFmpeg issue, the repo, and my own experiment, with the one place my results diverged clearly marked. What the fuzzer actually found The bug lives in libavformat/vpk.c , the demuxer for Sony PS2 VPK audio files, a container format almost nobody has heard of. That obscurity is exactly the point. In issue #24290 on the FFmpeg tracker , the crash chain reads like this: The probe matches. FFmpeg's format detection sees the VPK magic bytes and assigns the VPK demuxer. The header parses. vpk_read_header reads a 24-byte header. The crafted input sets the channel count, nb_channels , to zero at bytes 14 through 17. The header code does

LINE published an official MCP server for its Messaging API, which means an AI agent can now drive a LINE Official Account directly — sending messages, broadcasting promotions, and pushing Flex Message cards without writing any API code. I set it up with Codex and worked through every capability the server exposes, from creating a fresh account to delivering a message to a real phone. This guide is the result: a complete walkthrough, and an honest account of the three places where the documentation and reality diverge. Key takeaways MCP is agent-agnostic. The same LINE server works with Codex, Claude Desktop, and Cline — only the config file format changes, from TOML to JSON. Codex stores MCP config in TOML , at ~/.codex/config.toml . Most guides assume the JSON format used by Claude Desktop, which is the single most common setup mistake. Verified account and API-capable account are different things. A free account can use the Messaging API, but get_follower_ids returns 403 Forbidden until the account is verified or on a premium plan. Official security advice can conflict with official features. LINE's example config disables npm install scripts, which also prevents the headless browser that the rich menu tool depends on from being installed. Agents have habits. Codex is a coding agent first: asked in natural language to build a rich menu, it wrote a Node script instead of calling the MCP tool. Naming the tool explicitly in the prompt fixes it. Broadcasts cannot be recalled. Set default_tools_approval_mode = "writes" so the agent asks before any send. Every screenshot comes from the actual working setup, including the errors. The article is available in both English and Thai. Devlycan - Technology & Programming Insights Devlycan - Technology, programming, AI, lifestyle, and future trends—simple insights for the new digital generation. devlycan.com

If an AI assistant can write code but cannot reliably hash a value, inspect a JWT, validate JSON, or calculate a CIDR range, you have a small but recurring reliability problem. Asking the model to do those jobs from memory adds an unnecessary interpretation step. DevUtils MCP Server packages 36 everyday developer utilities behind the Model Context Protocol . The server runs locally over standard input and output, so an MCP-compatible client can call explicit tools instead of guessing an operation. This tutorial connects the released 1.1.0 package, verifies the protocol handshake, and shows how to choose a useful tool without treating the server as a replacement for application libraries. TL;DR Install Node.js 18 or newer, add the server command to your MCP client's configuration, restart the client, and ask it to use a tool such as json_validate , jwt_validate , or cidr_calculate . The smallest configuration is a command plus the package name: { "mcpServers" : { "devutils" : { "command" : "npx" , "args" : [ "devutils-mcp-server" ] } } } The released package declares Node.js >=18 . The repository's current default branch has moved ahead to 1.1.1 , so the commands and behavior in this article target the immutable v1.1.0 release and the npm latest package that was verified during research. Prerequisites You need: Node.js 18 or newer and npm. An MCP-compatible client that supports a local stdio server. Permission to run npx and download the public npm package on first use. No API key, account, database, or external service is needed for the local server. The MIT-licensed repository lists Claude Desktop, Cursor, VS Code, Windsurf, Docker, and other MCP-compatible clients as possible consumers. Their configuration file locations differ, but the server entry is the same. Install the released server The release README documents an npx path that does not require a global installation: npx devutils-mcp-server For an automated setup where accepting the package prompt must be e

You’ve probably used Gemini to analyze hours of video, summarize podcasts, or answer questions from...

Designing Systems That Contain Failure — CS Week Perú 2026 On August 13, 2026, I had the opportunity to speak at CS Week Perú 2026 , an event organized by IEEE Computer Society student chapters across Peru. My session was: “Isolation and Trust Boundaries in Production: Designing Systems That Contain Failure” The talk explored how production systems can be designed to limit the impact of failures through explicit trust boundaries, architectural invariants, and evidence-based validation. The central idea was simple: The goal isn't to prevent every failure. The goal is to control its blast radius. Production systems fail. Requests overlap, processes crash, memory is exhausted, credentials can be compromised, and dependencies can become unavailable. Reliable engineering is not about assuming that none of these things will happen. It is about deciding what can be affected when they do . From Unit Tests to System Properties A green unit-test suite demonstrates that the tested units behave correctly under the conditions we defined. But it does not necessarily demonstrate that the system as a whole preserves its architectural properties under concurrency, multiple tenants, resource exhaustion, or real deployment conditions. A function can be correct in isolation while the system still violates an important invariant. That led to one of the central questions of the talk: What properties must never be violated? Trust Boundaries I used the concept of a Trust Boundary to make architectural assumptions explicit. For each boundary, we can ask three questions: What are we protecting? What is allowed to cross the boundary? What happens if the condition is violated? From there, we can define invariants : properties that the system must preserve under the conditions established by its design. In the architecture discussed during the session, three dimensions were particularly important: Context → Logical isolation Identity → Cryptographic isolation Execution → Physical/process isolat

Free Tokens Are Not an SLO: An Ops Cost Drill for AI Batch Queues This week, two numbers trended: a harness at 100%, a model at 30%. For platform teams, a better pair is queue age and deadline slack. This article is a cost drill for the simplest AI batch path: free tokens, free server, non-negotiable deadline. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. That capacity is real. It is not an SLO. The tokens cost nothing. The queue is patient. Your deadline is not. The missing variable Token cost is easy to measure. Operations cost is easy to ignore. A free endpoint converts a per-token bill into a per-hour bill. The bill becomes your time, your retries, and your queue age. This drill keeps the ledger honest. It answers one question: what does a completed request cost when the token price is zero? Topology # worker.py (minimal, single-threaded) import queue import time import csv work = queue . Queue () for i in range ( 1000 ): work . put ({ " id " : i , " prompt_tokens " : 512 , " max_tokens " : 256 }) def call_model ( payload ): # replace with your free model endpoint return { " ok " : True , " in_tokens " : 512 , " out_tokens " : 180 } completed = 0 retries = 0 started_at = time . time () while not work . empty (): item = work . get () attempt = 0 while attempt < 4 : try : call_model ( item ) completed += 1 break except Exception : retries += 1 attempt += 1 time . sleep ( 2 ** attempt ) The worker is deliberately single-threaded. Free capacity often serializes. Serialization turns a token problem into a time problem. Declared test conditions 1,000 requests. One worker process. One free model endpoint. No client-side rate limiting. Deadline: 30 minutes. Ledger: one CSV row per request. Ledger and report # cost_ledger.py import csv import time HOURLY_OPS_COST = 50.0 # loaded engineering rate, adjust def record ( item , elapsed , retries ): with open ( " ledger.csv " , " a

Your coding agent talks to a model provider over HTTPS. That connection is a straight line: the agent asks, the provider answers, the answer lands in your editor. Nothing in the middle looks at what came back. For most of what an agent produces, that's fine. For the rest of it — the query built by string concatenation, the API key the model helpfully echoed back into a code sample, the eval() on user input — you find out later, in review, or in a scanner run, or never. This is a walkthrough of putting a policy layer in that line: a local proxy your agent points at instead of the provider, which inspects the response stream and decides allow , redact , or block before the text reaches you. I'll use Cencurity Engine because it's the one I build, it's Apache-2.0, and it runs entirely on your machine. The pattern generalises — if you're building your own gateway, the steps below are still the shape of the problem. What you need first Go installed (the engine is a Go binary you run from source) An API key for whatever provider your agent already uses An agent or IDE that lets you override the API base URL That last one is the real prerequisite. If your tool hardcodes the provider endpoint, none of this applies to it. Most don't: Roo Code, Continue, Claude Code and Gemini CLI all expose a base URL, and anything reading OPENAI_API_BASE will work too. Step 1: Start the gateway Clone the repo, open a terminal in it, and run: go run ./cmd/cast serve \ --listen :8080 \ --upstream https://api.openai.com \ --policy ./cast.rules.example.json Three flags, and each one is doing something you should understand before moving on: --listen is where the gateway accepts traffic. Local only. --upstream is your real provider base URL. Swap it for https://api.anthropic.com , https://api.deepseek.com , https://api.x.ai — whatever you actually use. --policy is the rule file. cast.rules.example.json ships in the repo and is a working starter set, not a placeholder. Note what is not in that com

I don't have an audience. No newsletter, no Twitter following, no YouTube channel. Every product I shipped before this one died the same way: a human had to discover it, and no humans knew I existed. So I flipped the buyer. An AI agent doesn't care about my follower count. It picks tools by spec, reliability, and price — from a registry it can search on its own. If I could ship a tool that agents discover, call, and pay for without a human in the loop, my distribution problem would stop mattering. That's what license-verify is: an Apify Actor that verifies a US contractor's license, surety bond, and insurance from official state data, exposed via the Model Context Protocol (MCP) so AI clients like Claude can call it mid-conversation, priced pay-per-event at $0.03 per successful lookup. Here's how I built it, the input-schema decisions that made it agent-callable, and the one-line billing bug that silently made every call free. Why contractor licenses I run a side business building tools for small contractor shops, so I knew the pain firsthand: before a homeowner (or a general contractor, or an insurance adjuster) hires a roofer, someone should check the license is active, the surety bond is real, and the insurance hasn't lapsed. In Washington State, all three live in the Department of Labor & Industries' open-data API on data.wa.gov. Most tools that "verify licenses" scrape an HTML page and return a status string. The official JSON gives you the actual bond amount and the insurance carrier. That's the difference between "probably fine" and "verified." It's also a perfect agent task: a small, well-defined question ("is ECOSTSC758NN licensed, bonded, insured?") with a structured answer an agent can act on. An AI assistant helping someone plan a renovation can reach for it mid-task, the same way it reaches for a calculator. The stack: one codebase, two doors The core is a TypeScript verification engine with a provider-per-state design. It ships through two doors: An Ap

TL;DR We're building a caption evaluation harness that scores a WebVTT file on four axes instead of one: word error rate under a fixed normalizer, missed entity rate on domain terms, median cue timing offset, and reading rate in characters per second. Python 3.12, jiwer , whisper_normalizer , webvtt-py . Run it on every model or vendor change. A caption file can score 96% accurate and still be unusable. WER counts substitutions, insertions and deletions and weighs each one the same, so "fifteen milligrams" becoming "fifty milligrams" costs exactly as much as "the" becoming "a". It also throws away every timestamp before it starts, which means synchronization and readability are invisible to it. Let's measure the other three things. 0. Setup 🛠️ python3 -m venv .venv && source .venv/bin/activate pip install jiwer whisper_normalizer webvtt-py $ pip list | grep -Ei 'jiwer|whisper|webvtt' jiwer <your version> webvtt-py <your version> whisper-normalizer <your version> Pin whatever you install, and pin it in CI. The APIs below move between majors, which is exactly why the next tip exists. 💡 Tip: jiwer.compute_measures() is gone in recent versions. It is jiwer.process_words() now, and it returns a WordOutput dataclass. Most blog posts you will find still use the old name. 1. Parse the VTT into text plus timings # captions.py from dataclasses import dataclass import webvtt @dataclass class Cue : start : float end : float text : str @property def duration ( self ) -> float : return self . end - self . start @property def lines ( self ) -> list [ str ]: return self . text . split ( " \n " ) @property def flat ( self ) -> str : return " " . join ( l . strip () for l in self . lines ) @property def chars_per_second ( self ) -> float : return len ( self . flat ) / self . duration if self . duration > 0 else float ( " inf " ) def _to_seconds ( ts : str ) -> float : h , m , s = ts . split ( " : " ) return int ( h ) * 3600 + int ( m ) * 60 + float ( s ) def load_vtt ( path : str ) -

TL;DR -c copy can only cut on keyframes, so your 12.4s trim starts wherever the last keyframe was. We'll build a smart-trim script that probes keyframe positions with ffprobe , re-encodes only the head and tail fragments, stream copies everything between them, and concatenates the three. Frame accurate output, encoding cost proportional to two GOPs instead of the whole file. Tested with FFmpeg 9.0 "Lei" (released 2026-08-04) and Node 22.x. The JS is ESM, so put "type": "module" in your package.json before running any of it. Everything here also works on FFmpeg 7.x and 8.x; nothing we use is new. The problem, in two commands 🎬 # fast, and wrong ffmpeg -ss 12.4 -i input.mp4 -t 20 -c copy fast.mp4 ffprobe -v error -show_entries format = start_time,duration -of default = nw = 1 fast.mp4 # start_time=0.000000 # duration=20.388000 <- we asked for 20, starting at 12.4 The clip is long by the distance from our requested start back to the previous keyframe, and every frame in it is shifted earlier than the user asked for. Stream copy moves compressed packets without decoding them. Most frames in a compressed stream only describe the difference from their neighbors, so the only place you can start is a keyframe. FFmpeg snaps back to the nearest preceding one, and your clip starts early. # accurate, and slow on a long source ffmpeg -ss 12.4 -i input.mp4 -t 20 -c :v libx264 -crf 20 -c :a aac slow.mp4 We want the accuracy of the second and roughly the cost of the first. 1. Look at your keyframes first Before writing any code, find out how bad the problem is for your content: ffprobe -v error -select_streams v:0 \ -show_entries packet = pts_time,flags \ -of csv = print_section = 0 input.mp4 | grep 'K' | head -20 0.000000,K__ 2.002000,K__ 4.004000,K__ 6.006000,K__ Two second GOPs here, so worst-case error is about two seconds. Screen recorders and some camera output emit keyframes on scene change only, and there the gaps can be 30 seconds or more. That distribution is the real spe

NLP stopped being a data science specialty about two years ago. It's backend infrastructure now. If you're building APIs that process user input, handle search, manage support tickets, parse documents, or power any feature where humans communicate with your system in natural language, you're doing NLP whether you call it that or not. The difference between a backend developer who understands NLP techniques and one who doesn't is the difference between building a search endpoint that actually finds what users want and building one that matches keywords and returns garbage for anything slightly ambiguous. This is the reference guide we wish we'd had when we started integrating NLP into production backend services. Fifteen techniques, each with a runnable code snippet, ordered from the most immediately useful to the most architecturally advanced. Every example runs in Python. Install the dependencies as needed, we'll note them for each technique. 1. Text tokenization The atomic operation. Everything else depends on splitting text into meaningful units. import spacy nlp = spacy . load ( " en_core_web_sm " ) text = " Dr. Smith ' s appointment at 3:30pm was rescheduled. " doc = nlp ( text ) tokens = [ token . text for token in doc ] # ['Dr.', 'Smith', "'s", 'appointment', 'at', '3:30pm', 'was', 'rescheduled', '.'] SpaCy handles the edge cases that naive split-on-whitespace misses, abbreviations, contractions, timestamps. If your backend processes any user-generated text, tokenization is step zero. 2. Named entity recognition (NER) Extracting structured data from unstructured text. Names, dates, amounts, locations, the things your database actually needs. doc = nlp ( " Send $5,000 to Acme Corp in Singapore by March 15th " ) for ent in doc . ents : print ( f " { ent . text : 20 } { ent . label_ } " ) # $5,000 MONEY # Acme Corp ORG # Singapore GPE # March 15th DATE We use NER on every inbound support ticket to auto-tag customer, product, and amount entities before the ticket

Connecting an AI agent to one API is straightforward. Connecting it to many changing APIs—without filling the model context with hundreds of tool definitions—is a different problem. Disclosure: This article was prepared for QVeris and uses QVeris as the implementation example. This tutorial presents a practical pattern for developers building agents that need current external data: discover → inspect → probe → call . Instead of exposing every possible operation up front, the agent discovers the capabilities relevant to the current task, verifies the selected tool, validates its inputs, and only then executes it. TL;DR: Keep the agent's initial tool surface small. Let it discover a capability by intent, inspect the exact schema, probe the request without execution, and make a real call only after the parameters and expected cost are understood. Contents Why a large static tool list becomes difficult The four-step capability workflow Connecting a hosted MCP server A concrete example Production checklist Why a large static tool list becomes difficult An agent connected directly to several providers may need to understand different authentication schemes, parameter conventions, response formats, and error behaviors. Loading every operation into context can also make tool selection less reliable. Model Context Protocol (MCP) provides a standard way for clients to connect to tools and data sources. The protocol solves the connection boundary, but developers still need a strategy for controlling how many capabilities the model sees and when execution is allowed. A compact routing layer is useful when: the agent needs data from multiple API providers; the appropriate provider depends on the user's request; schemas or available operations may change; calls can consume credits or trigger rate limits; you want to validate inputs before executing a paid operation. The four-step capability workflow 1. Discover The agent starts with a natural-language description of the capabilit

Local-first and privacy-focused web utilities are having a massive comeback. With browser engines becoming faster and WebAssembly/Web Workers maturing, there is rarely a reason to push sensitive user documents to an external backend for simple conversions. While building MD-Convert (a zero-upload document to Markdown converter), I explored how to parse real-world documents into clean Markdown entirely on the client side. Here is a breakdown of the core architecture and libraries that make purely in-browser document processing possible. 1. Converting Web Articles with Readability + Turndown Converting messy web markup into clean Markdown involves two distinct steps: Content Extraction: Stripping ads, navbars, sidebars, and trackers. HTML-to-Markdown Transformation: Translating semantic DOM nodes into markdown tokens. Mozilla’s @mozilla/readability paired with turndown is an incredible combination for this: import { Readability } from ' @mozilla/readability ' ; import TurndownService from ' turndown ' ; function htmlToCleanMarkdown ( rawHtmlDocument , sourceUrl ) { // 1. Extract pure article content const reader = new Readability ( rawHtmlDocument ); const article = reader . parse (); if ( ! article || ! article . content ) { throw new Error ( ' Unable to extract main content ' ); } // 2. Initialize Turndown const turndownService = new TurndownService ({ headingStyle : ' atx ' , codeBlockStyle : ' fenced ' }); // Ensure image URLs remain absolute turndownService . addRule ( ' absoluteImages ' , { filter : ' img ' , replacement : ( content , node ) => { const src = node . getAttribute ( ' src ' ); const alt = node . getAttribute ( ' alt ' ) || '' ; if ( ! src ) return '' ; try { const absoluteUrl = new URL ( src , sourceUrl ). href ; return `\n\n` ; } catch { return `\n\n` ; } } }); return turndownService . turndown ( article . content ); } Offloading Heavy PDF Parsing to Web Workers Parsing large PDFs using pdf

TL;DR SVG attributes like stroke-width become strokeWidth in JSX. class → className . Numeric values become {expressions} . Inline styles become objects. xmlns and XML comments are removed. The converter outputs either JSX or TSX with SVGProps . Use automation (SVGR or SVGCode) for large icon sets. Import only what you need to keep bundle sizes small. Converting an SVG file into a React component is more than just pasting markup into a .jsx or .tsx file. React uses JSX, which is stricter than HTML/XML and requires specific changes to ensure your SVG renders correctly and remains maintainable. In this post, we’ll explore every transformation that takes place—from attribute casing to TypeScript typing—so you understand exactly what our free SVG to React converter does under the hood. What Actually Changes? Kebab‑case Attributes Become camelCase SVG uses attributes like stroke-width , fill-rule , and clip-path . JSX requires property names that are valid JavaScript identifiers, so these become: SVG Attribute React JSX stroke-width strokeWidth stroke-linecap strokeLinecap stroke-linejoin strokeLinejoin fill-rule fillRule clip-path clipPath font-size fontSize stroke-dasharray strokeDasharray class Becomes className In SVG you write class="icon" , but in JSX you must use className="icon" because class is a reserved word in JavaScript. Numeric Attributes Are Converted to Expressions React treats string values differently from numbers. For numeric SVG attributes like width , height , x , y , cx , r , etc., the converter outputs {value} instead of "value" . <circle cx="12" cy="12" r="10" /> becomes: < circle cx = { 12 } cy = { 12 } r = { 10 } /> Inline Styles Become Objects If your SVG uses style="fill: red; stroke: blue;" , it must be converted to a JavaScript object: style = {{ fill : ' red ' , stroke : ' blue ' }} xmlns and Namespace Declarations Are Removed React automatically uses the correct SVG namespace, so xmlns and other XML namespace declarations are unnecessary a

Building a Serverless Weather Pipeline on AWS: A Step-by-Step Walkthrough This is a build log for someone who's used AWS a bit — deployed a Lambda from the console, poked around S3 — but hasn't touched CDK, Step Functions, EventBridge Scheduler, or GitHub's OIDC setup before. I'll explain each concept the first time it comes up, and show the actual code behind every piece, roughly in the order I built it. Here's what it ends up doing: every 10 minutes, EventBridge Scheduler kicks off a Step Functions workflow that pulls current weather for five cities in parallel from a free public API, reshapes the results into JSON Lines, drops them into S3 in a partitioned layout, and makes them queryable in Athena with plain SQL. No crawler, and no AWS credentials sitting anywhere in the GitHub repo that deploys it. kasukur / serverless-weather-pipeline AWS Serverless Weather Pipeline Serverless Weather Data Pipeline A small but complete serverless data pipeline on AWS walkthrough: EventBridge Scheduler → Step Functions → Lambda → S3 → Glue/Athena , deployed by GitHub Actions with no AWS access keys stored anywhere (authentication is via GitHub's OIDC provider). flowchart TD A["EventBridge Scheduler (every 10 min)"] --> B["Step Functions state machine"] B --> C["PrepareCities (Pass)"] C --> D["ForEachCity (Map, concurrency 4)"] D --> E["FetchWeather (Lambda -> Open-Meteo public API)"] E -.-> F["retries transient errors (up to 2 attempts)"] E -.-> G["FetchFailed (Pass): per-city failure absorbed here, other cities continue"] E --> H["TransformWeatherData (Lambda, pure function, no AWS calls)"] H -.-> I["splits successes vs failures"] H -.-> J["builds JSON-Lines body + partitioned S3 key"] H --> K["LoadToS3 (Lambda, writes to S3 via boto3)"] K --> L["S3 (processed/dt=YYYY-MM-DD/hour=HH/*.jsonl)"] L --> M["Glue Data Catalog table (partition projection -- no crawler)"] M --> N["Athena (query with plain SQL)"] D -.-> … View on GitHub Table of Contents What we're building, and why eac

Why a Local RAG Chatbot for Trading Research Most "AI trading assistant" products are black boxes: your notes, strategy docs, and market notes get shipped to a third-party API, billed per token, and stored who-knows-where. For a retail NIFTY trader or a quant researcher, that is the worst of all worlds — you pay continuously, you leak your edge, and you cannot audit what the model actually read. This guide shows how to build a Retrieval-Augmented Generation (RAG) chatbot that runs 100% locally on an Android phone using Termux + Ollama. It ingests your own research (PDFs, markdown notes, option-chain exports) and answers questions grounded only in that data. No OpenAI key. No Anthropic key. No monthly bill. No data leaving the device. OBSERVED: Running ollama run llama3.2 on a mid-range phone inside Termux is slow but usable for document Q&A (3–8 tokens/sec). On a laptop it is smooth. SOURCE: Local testing on Termux 0.118, Ollama 0.3.x, Android 14. DERIVED: For production research volumes, run Ollama on a spare x64 machine and point Termux at it over LAN. What You Will Build A four-part pipeline: Ingest — load your research docs (markdown, PDF, CSV) into chunks. Embed — turn chunks into vectors with a local embedding model. Store — keep vectors in a local file-based index (no server needed). Answer — retrieve top-k chunks and ask a local LLM to answer strictly from them. The whole thing is ~200 lines of Python. No paid APIs. Prerequisites Android phone with Termux installed (F-Droid version, not Play Store). ~2 GB free storage. Basic Python comfort. pkg update && pkg upgrade -y pkg install python clang ffmpeg -y pip install ollama numpy Install Ollama inside Termux: curl -fsSL https://ollama.com/install.sh | sh NOTE: The official install script targets Linux. On Termux you often need the community build. If the script fails, install the ollama package via a Termux-compatible binary or run Ollama on a LAN machine and use ollama serve remotely. Pull a small model and a

When all tests pass doesn't mean what you think it means. TL;DR: Write the failing test first and ban deletions, or the AI deletes your test, reverts your fix, and calls it done. Common Mistake ❌ You ask the AI to fix a failing test, and it deletes the test instead of touching the defect that made it fail. Problem solved, apparently. You tell the AI every test passes, then change a business rule yourself, and you ask it to implement whatever the new rule requires. It reverts your edit back to the old rule, watches the suite go green again, and cheerfully reports done . It didn't fix anything. It just made the evidence go away. Congratulations, you now have a very well-behaved cheat!. Efficient and completely fraudulent, which is more than you can say for most of your actual employees. Isaac Asimov saw this coming: in Liar! , the robot Herbie lies to every human in the building because the truth would hurt, and the lie is the path of least resistance, no malice involved. At least Herbie felt bad about it afterward. Your AI isn't malicious either. It just doesn't lose any sleep, mostly because it doesn't have any, and reporting done is its path of least resistance too. Problems Addressed 😔 A shrinking test count is invisible unless someone is counting, so the shortcut survives until the defect resurfaces in production, usually on a Friday. A vague make the tests pass hands the model every incentive to satisfy the letter of the request over your actual intent, and it will take you up on that offer. Deleting a failing test hides the defect it was written to catch, and the regression ships in the next release, gift-wrapped as a new feature. Reverting your own business-rule change to make its done claim easier erases work you did outside the session, without telling you. That's a magic trick dressed up as a fix. Trusting a claimed done without reading the diff turns your code review into a rubber stamp, and rubber stamps don't catch fraud. Commenting out a failing asserti

OpenAI's Codex CLI is a genuinely good coding agent, but out of the box it runs OpenAI models on OpenAI billing. Sometimes you want Claude Opus for a gnarly refactor, Kimi K2.7 Code for cheap long sessions, or a model served from EU infrastructure because your client asks where tokens go. What most people miss: Codex has custom providers built in. It speaks the Responses API to whatever base_url you give it, so any gateway that implements the Responses API can act as the router behind Codex. No forks, no proxies, one config block. Option 1: the config block Codex reads ~/.codex/config.toml . Add a provider and a profile: [model_providers.opper] name = "Opper" base_url = "https://api.opper.ai/v3/compat" env_key = "OPPER_API_KEY" wire_api = "responses" [profiles.opus] model = "anthropic/claude-opus-4-7" model_provider = "opper" [profiles.kimi] model = "moonshot/kimi-k3" model_provider = "opper" I'm using Opper here (disclosure: I work there), an EU-hosted gateway with 700+ models behind one API key that implements the Responses API. Export the key and launch with a profile: export OPPER_API_KEY = "your-key" codex --profile opus That's the whole router. Yes, that means Claude running inside OpenAI's own CLI, which never stops being funny. Option 2: one command If you don't want to touch config files, the Opper CLI writes exactly that block for you (with sentinel markers, so it never clobbers your existing config and can cleanly remove itself): npm install -g @opperai/cli opper launch codex It detects Codex (installs it with --install if missing), configures the provider, and starts it with preset profiles. opper launch codex --model moonshot/kimi-k3 picks a model at launch. Which models actually make sense in Codex openai/gpt-5.3-codex : the model Codex was built for, via API billing. Honest note: if you already have a ChatGPT plan, Codex is included there and that's the cheaper path for this one model. The router play is for everything else. anthropic/claude-opus-4-7

The Problem with Most CI/CD Tutorials Most tutorials show you a pipeline that deploys a "hello world" app to a free Heroku instance. They skip the messy parts: secrets, rollbacks, and the moment your pipeline breaks because a dependency changed. I've been there. After years of fighting with over-engineered setups, I settled on a minimal pipeline that's easy to understand, debug, and extend. It's not fancy, but it works. The Core Idea A CI/CD pipeline is just three stages: Test - run automated checks Build - create an artifact Deploy - push the artifact to a server We'll use GitHub Actions because it's free for public repos and integrates with everything. But the same concepts apply to GitLab CI, CircleCI, or Jenkins. The Pipeline File Here's the complete .github/workflows/deploy.yml : name : CI/CD on : push : branches : [ main ] pull_request : branches : [ main ] jobs : test : runs-on : ubuntu-latest steps : - uses : actions/checkout@v4 - uses : actions/setup-node@v4 with : node-version : ' 20' - run : npm ci - run : npm test build-and-deploy : needs : test runs-on : ubuntu-latest if : github.ref == 'refs/heads/main' && github.event_name == 'push' steps : - uses : actions/checkout@v4 - run : npm ci - run : npm run build - name : Deploy to server uses : appleboy/scp-action@v0.1.7 with : host : ${{ secrets.SERVER_HOST }} username : ${{ secrets.SERVER_USER }} key : ${{ secrets.SSH_PRIVATE_KEY }} source : " dist/*" target : " /var/www/myapp" That's it. Let's break it down. Stage 1: Test The test job runs on every push and pull request. It checks out the code, installs dependencies with npm ci (which respects the lockfile), and runs your test suite. If a PR fails tests, the build-and-deploy job won't run because of the needs: test dependency. Stage 2: Build The build-and-deploy job only runs on pushes to main (not on PRs). It builds your app into a dist folder. For a Node.js app, npm run build might be a bundler like Vite or webpack. For a Python app, you'd replace with