标签：#rag

Base Power is skipping the PJM's troubled interconnection queue by placing its batteries at people's homes, offering backup services in exchange.

MCP + RAG: Why I Stopped Building Complex RAG Systems After MCP Changed Everything Honestly, I've spent the last four years building increasingly complex RAG systems. Chunking strategies, embedding models, vector databases, rerankers, hybrid search... you name it, I've probably wasted a weekend trying it. I had this 1,800-hour knowledge base project called Papers — six years of notes, articles, bookmarks, everything. I built RAG version after RAG version, each time thinking "this time it'll be perfect." Spoiler: It never was. Then I added MCP (Model Context Protocol) support. And I realized something that completely changed how I think about knowledge retrieval: MCP makes traditional complex RAG obsolete for most use cases. Let me explain what I learned the hard way. The RAG Trap I Was Stuck In If you've built a RAG system, you know the drill: Chunking : Should you use fixed-size, semantic, recursive, or something fancy like LLM-powered chunking? Embeddings : OpenAI text-embedding-3-large vs Cohere vs nomic-ai vs your fine-tuned model? Vector Database : Pinecone vs Weaviate vs PGVector vs Qdrant vs Chroma? Retrieval : Top-k how many? Hybrid search with keywords? Reranking? Prompt Compression : How do you fit all the retrieved chunks into the context window? I went through every iteration. At one point, my RAG system was over 2,000 lines of code. I had configurable chunkers, multiple embedding providers, caching layers, hybrid search... it was impressive. It also didn't work that well. Here's what bothered me the most: I kept throwing more complexity at the problem, but the fundamental issue never went away. I was trying to make my knowledge base smart, but AI already got smart. Why was I reimplementing all this understanding logic when the AI can already do it better than me? How MCP Changed the Game When I added MCP support to Papers, I started with the simplest possible approach: Expose two tools: search_notes and get_note_content Search is just basic text matchin

SeaweedFS is an open-source, distributed object storage system with an S3-compatible API, a filer for POSIX-style hierarchical access, and a small footprint. This guide deploys SeaweedFS using Docker Compose with the master, volume, filer, S3, and admin services behind Traefik for automatic HTTPS on separate admin and S3 domains. By the end, you'll have SeaweedFS serving S3-compatible object storage securely at your domains. Prerequisite: Two DNS A records pointing at the server — storage.example.com (admin dashboard) and s3.storage.example.com (S3 API). AWS CLI installed on your local machine for testing. Set Up the Directory Structure 1. Create the project directory: $ mkdir seaweedfs && cd seaweedfs 2. Generate an access key and a secret key (run twice and save both): $ openssl rand -hex 16 3. Create the environment file: $ nano .env STORAGE_DOMAIN = storage.example.com LETSENCRYPT_EMAIL = your-email@example.com ADMIN_PASSWORD = yourpassword 4. Create the S3 identities file: $ nano s3-config.json { "identities" : [ { "name" : "admin" , "credentials" : [ { "accessKey" : "YOUR_ACCESS_KEY" , "secretKey" : "YOUR_SECRET_KEY" } ], "actions" : [ "Admin" , "Read" , "Write" , "List" , "Tagging" ] } ] } Deploy with Docker Compose 1. Create the Compose manifest: $ nano docker-compose.yml services : traefik : image : traefik:v3.7.0 container_name : traefik restart : unless-stopped ports : - " 80:80" - " 443:443" volumes : - /var/run/docker.sock:/var/run/docker.sock:ro - ./letsencrypt:/letsencrypt command : - --providers.docker=true - --providers.docker.exposedByDefault=false - --entrypoints.web.address=:80 - --entrypoints.websecure.address=:443 - --entrypoints.web.http.redirections.entrypoint.to=websecure - --entrypoints.web.http.redirections.entrypoint.scheme=https - --entrypoints.web.http.redirections.entrypoint.permanent=true - --certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL} - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json - --

Web MCP, DevTools per agenti e Modern Web Guidance: meno hype, più strumenti e metodo. Negli annunci recenti di Chrome è emersa una cosa interessante: al netto delle novità “appariscenti”, ciò che resta più utile per il lavoro quotidiano è quello che migliora workflow, diagnosi e decisioni tecniche . Tre filoni, in particolare, disegnano una direzione chiara: Web MCP , DevTools per agenti e Modern Web Guidance . Di seguito una sintesi ragionata di cosa significano, perché contano per il frontend, e come prepararsi a sfruttarli. 1) Web MCP: il ponte tra agenti e Web (senza incollaggi fragili) Se stai lavorando con assistenti/agentic workflow, oggi il collo di bottiglia è quasi sempre lo stesso: far sì che un agente capisca e usi le capacità del browser e delle app web in modo affidabile. Web MCP punta a risolvere questo punto creando un linguaggio/protocollo comune per esporre “capacità” (capabilities) e strumenti (tools) che un agente può invocare in modo strutturato, invece di basarsi su prompt lunghi, scraping o integrazioni ad hoc. Perché è importante per chi fa frontend Automazioni più robuste : meno script fragili che si rompono al primo refactor del DOM. Integrazioni più standard : se più strumenti parlano lo stesso “dialetto”, il costo di collegare agenti e applicazioni scende. Esperienze utente nuove : assistenti che completano task complessi dentro l’app (es. compilazioni, ricerca guidata, operazioni amministrative) con maggiore affidabilità. Implicazione pratica Inizia a ragionare sull’app come su un insieme di azioni esplicite (es. “crea ordine”, “esporta report”, “filtra dataset”), non solo come UI. Questa mentalità ti rende pronto a esporre capacità in modo sicuro e controllato, quando lo stack lo renderà semplice. 2) DevTools per agenti: debugging e performance nell’era dell’automazione Se Web MCP è il “ponte”, DevTools per agenti è la cassetta degli attrezzi per controllare quel ponte: osservabilità, diagnosi e iterazione rapida su flussi in cui non è

A few months ago I was demoing my RAG-powered support bot to a colleague, feeling pretty confident about it. Then it confidently told her our refund policy was “30 days, no questions asked.” Our actual policy is 14 days, with conditions. The bot didn’t hedge. It didn’t say “I’m not sure.” It just made it up and said it with the same calm tone it uses for everything else. That demo stung. RAG was supposed to fix hallucinations, not just relocate them. Here’s what I learned debugging it, roughly in the order I learned it. 1. My chunks were too big, and too dumb I was splitting documents by character count, 1000 chars with slight overlap. It felt efficient. It wasn’t. A single chunk often contained unrelated sections. For example, the end of a “Shipping Policy” and the start of a “Returns Policy” could sit together in the same block. So when the retriever saw a query about returns, it would grab that chunk and the model would blend both sections into one confident but wrong answer. Fix: I switched to semantic chunking based on headings and paragraphs instead of raw character limits. More work upfront, but it stopped feeding the model Frankenstein context. 2. I trusted top-k similarity way too much My retriever was pulling the top 3 chunks by cosine similarity and passing them straight into the prompt. The problem: “similar” is not the same as “relevant.” A chunk can be semantically close to the query but still not actually contain the answer. The model doesn’t know that, it just assumes everything in context is true. Fix: I added a reranking step using a cross-encoder and started logging retrieval scores properly. That alone made it obvious when the system had no real answer but was still trying to act confident. 3. I never told the model it was allowed to say “I don’t know” My prompt was basically: “Use the context to answer the question.” That’s it. No instruction on what to do when the context is insufficient. So the model did what LLMs do when under-specified: it f

Why a simple string match beat Apple's NLEmbedding for local RAG how apple's nlembedding drove me crazy and how i built my own hybrid search engine recently, while working on my personal ai agent (pheronagent), i was focused on perfecting its memory and retrieval system. everyone is talking about that famous acronym: rag (retrieval-augmented generation). the system is simple: i feed the agent my documents, it converts them into vectors (embeddings), and when i ask a question, it finds the most similar vectors and answers me. sounds perfect on paper, right? so, like any loyal apple ecosystem developer, instead of downloading massive models from external sources (or burning money on apis), i decided to use nlembedding—the native capability of the operating system that runs directly on-device. after all, apple had embedded this into the os; it was both fast and privacy-focused. but real life, as it turns out, doesn't progress as smoothly as wwdc presentations... where have i worked? - the first explosion it all started with a very innocent question. i had uploaded my cv to the system. while chatting with my agent, i casually asked: "where have i worked?" i expected the agent to fire up the metal cores in the background within seconds, find my cv, and list the companies for me. instead, the agent stared blankly. i opened the logs to see what the hell the search engine was doing behind the scenes. the shocking scenario was exactly this: cosine similarity between the query and my actual cv text: 0.587 the threshold i set for relevance: 0.60 it missed it by a hair! "no worries," i thought. "we can just lower the threshold a bit, make it 0.55, and call it a day." but then i saw the truly terrifying thing just one line below. for the exact same query, guess what score a completely irrelevant, junk record in the system—a list of files containing .ds_store—got? 0.59 - 0.60! wait a minute... my detailed, multi-page resume gets a score of 0.587 just because it doesn't contain th

Introduction Large Language Models (LLMs) like ChatGPT have transformed how we interact with AI. They can write code, answer questions, summarize documents, and generate creative content. However, they have one major limitation - they only know what they were trained on and can sometimes generate incorrect or outdated information. So, how do modern AI applications answer questions about your company's private documents, recent news, or knowledge that wasn't part of the model's training? The answer is Retrieval-Augmented Generation (RAG). In this blog, we'll explore what RAG is, how it works, its architecture, benefits, challenges, and real-world applications. What is RAG? Retrieval-Augmented Generation (RAG) is an AI architecture that combines a retrieval system with a Large Language Model (LLM). Instead of relying only on the model's internal knowledge, RAG first retrieves relevant information from an external knowledge source and then uses that information to generate a more accurate response. Think of it like an open-book exam. Instead of answering from memory, the AI first searches for the most relevant pages and then writes the answer based on those pages. Why Do We Need RAG? Traditional LLMs have several limitations: Knowledge becomes outdated. They cannot access private company data. They may hallucinate (generate incorrect facts). Retraining models is expensive and time-consuming. RAG solves these problems by allowing the model to retrieve fresh and domain-specific information before generating an answer. RAG Architecture A typical RAG pipeline consists of the following components: User Query Embedding Model Vector Database Retriever Prompt Builder Large Language Model Final Response Step-by-Step Workflow * Step 1: * User asks a question Example: "What is our company's leave policy?" Step 2: Convert the question into embeddings The query is transformed into a vector representation using an embedding model. Example: "What is leave policy?" ↓ [0.12, -0.45, 0.7

How to Build Systems That Actually Know Your Data (Not Hallucinate About It) Introduction: The Story Begins 👦 Nephew: Uncle, I keep hearing "RAG this, RAG that" in tech interviews. When I ask what it means, people throw around words like "Retrieval-Augmented Generation" and I just nod like I understand. But honestly? I'm lost. 👨‍🦳 Uncle: (laughing) That's the best honest question I've heard all week. Let me ask you something first. If I gave you a question right now - "What year did India win the World Cup?" - how would you answer? 👦 Nephew: Well... I'd pull up Google, search for it, read the answer, then tell you. 👨‍🦳 Uncle: Exactly. You don't answer from memory alone. You go fetch the information first, then answer based on what you found . That's RAG in real life. And that simple idea - fetch first, answer after - fixes almost every problem we face with AI today. 👦 Nephew: But uncle, AI can remember things from its training. Why does it need to fetch? 👨‍🦳 Uncle: Ah! That's where we land in trouble. Come, sit... SECTION 1: RAG FUNDAMENTALS - The Core Concept The Problem We're Actually Solving 👨‍🦳 Uncle: Imagine you're hiring for a tech company. You receive 500 resumes for a Senior React Developer role. Now tell me - how would you actually process them? 👦 Nephew: I'd... probably make a spreadsheet? List all the candidates with key skills? 👨‍🦳 Uncle: Right. But here's the catch - you can't read all 500 resumes deeply. So what do you really do? 👦 Nephew: Skim for keywords like "React", "JavaScript", "5 years"? 👨‍🦳 Uncle: Exactly. You skim and hope you don't miss anyone good. Now, here's the problem: what if a candidate wrote "React.js" instead of "React"? Your eyes might still catch it. But a dumb computer doing exact string matching? It says "no match". What if someone wrote "Built real-time user interfaces with the React framework"? The candidate clearly knows React, but the word "React" appears nowhere in that sentence. The computer misses them. This is exactly wh

AI Agent Orchestration: Proxmox Automation, OpenAI Data Agents & Azure Serverless Runtime Today's Highlights Today's highlights focus on practical AI agent applications and robust deployment strategies. We delve into building a secure AI admin for Proxmox, explore OpenAI's internal data analyst agent, and examine Azure Functions' new serverless runtime for agents. I didn't trust an AI with my Proxmox cluster — so I built one that can't surprise me (Dev.to Top) Source: https://dev.to/john-broadway/i-didnt-trust-an-ai-with-my-proxmox-cluster-so-i-built-one-that-cant-surprise-me-2k9l This article details a practical, hands-on approach to building a reliable AI agent for managing a Proxmox virtual environment. The author sought an agent capable of performing critical tasks like creating VMs, fixing storage issues, and tailing container logs, but with an emphasis on predictable and safe operations. The core idea is to create an AI that operates within defined boundaries, ensuring it doesn't perform unexpected or destructive actions. This tackles a crucial challenge in AI agent development: achieving trust and control in automated workflows. The implementation likely involves careful prompt engineering, tool use, and possibly a custom execution environment or validation layers to ensure commands are executed as intended and within pre-approved parameters. This project exemplifies how developers can apply AI agent orchestration principles to real-world IT automation, moving beyond simple information retrieval to true task execution, while maintaining human oversight and preventing 'surprises' common with less constrained AI systems. It's a blueprint for anyone looking to build robust, trustworthy AI-powered RPA solutions for system administration. Comment: A brilliant take on building AI agents for critical infrastructure. The focus on 'can't surprise me' highlights the need for robust control and guardrails, crucial for production workflow automation. This is what practic

Imagine a user trying to upload a 100MB video or a high-resolution photo to your app. If you use the standard Rails file upload, that file travels from the user's browser to your Rails server, and then your server sends it to S3 or Google Cloud. This is a terrible way to do it. While that 100MB file is transferring, your Rails worker (Puma) is frozen. It can't handle other users. If three people upload large files at once, your whole app will stop responding. In 2026, the professional way to handle this is Direct Uploads . With Direct Uploads, the file goes directly from the user's browser to your cloud storage (S3, R2, etc.). Your Rails server only handles a tiny bit of metadata. It is faster for the user and much safer for your server. Here is how to set it up in Rails 8. STEP 1: Configure Your Storage First, make sure you aren't using the local disk for production. You need a cloud provider like AWS S3 or Cloudflare R2. In your config/storage.yml : amazon : service : S3 access_key_id : <%= ENV['AWS_ACCESS_KEY_ID'] %> secret_access_key : <%= ENV['AWS_SECRET_ACCESS_KEY'] %> region : us-east-1 bucket : my-app-uploads # Crucial for Direct Uploads! public : true Note: You must configure CORS in your S3/R2 dashboard to allow requests from your domain. If you don't do this, the browser will block the upload. STEP 2: The Rails Form Rails makes the backend part incredibly easy. You just add one attribute to your file field: direct_upload: true . <!-- app/views/users/_form.html.erb --> <%= form_with ( model: user ) do | f | %> <div class= "field" > <%= f . label :avatar %> <%= f . file_field :avatar , direct_upload: true %> </div> <%= f . submit "Save Profile" %> <% end %> When you add direct_upload: true , Rails automatically includes a JavaScript library that handles the "handshake" with S3. STEP 3: Adding a Progress Bar (The UX Win) Direct uploads can take a few seconds. If nothing happens on the screen, the user will think your app is broken. We can use the built-in Ac

Your retriever returned the right documents. The similarity scores look fine. The answer is still wrong. If you've shipped RAG, you've seen this — and it's the failure that survives every retrieval upgrade. What everyone tries Reranker. Higher top-k. Hybrid search. A better embedding model. All of these chase the same goal: documents more similar to the query. They help when the right document wasn't being retrieved. They do nothing when the right document was retrieved and the answer is still wrong. Why it doesn't work Similarity answers "is this chunk about the same topic?" It does not answer "does this chunk contain the facts needed to support the answer?" Those come apart constantly. A chunk can be highly similar — same vocabulary, same subject — and contain nothing that actually grounds the answer. Hand the model a pile of on-topic text and it will produce a fluent, plausible, even cited-looking answer. The grounding is cosmetic: the text was nearby, not load-bearing. High similarity with a wrong answer isn't a contradiction. You asked retrieval to find related text. It did. Nobody asked whether the text was enough. The one shift Stop treating retrieval output as evidence. Treat it as candidate material that has to pass an explicit evidence check before it can support an answer. Put a step between retrieval and generation: does the retrieved set actually contain the facts this answer requires? If not, abstain. When the documents don't contain the facts, the system should return nothing rather than a confident guess. Relevant context in, only sufficient evidence allowed through. That's the line between a RAG demo and a RAG system you can trust in production. I write about the three boundaries where production RAG dies — query, evidence, output — from the angle of shipping under security and model constraints. Read the full version on my blog , where this connects to the practical RAG Failure Diagnosis Kit for teams debugging production RAG.

The suggestion every RAG app ignores If you've shipped a retrieval-augmented assistant, you've written some version of this line in your system prompt: "If the answer isn't in the provided context, say you don't know. Do not make things up." And you've watched the model cheerfully ignore it under pressure. A confident-sounding question comes in, retrieval returns something adjacent , and the model stitches together an answer that's plausible, fluent, and wrong. Telling a language model not to hallucinate is a suggestion — and suggestions lose to the model's overwhelming prior toward being helpful. I got tired of fighting this with prompt wording, so I tried a different framing while building MCP SDK Docs Assistant , an assistant for the Model Context Protocol TypeScript SDK. The framing: don't ask the model to refuse — remove its ability to fabricate. Refusal as code, not as a prompt The core idea is that the model can only hallucinate if you hand it material to hallucinate from. So the refusal decision lives in the retrieval tool, before the model ever sees anything. If nothing clears a confidence bar, the tool returns an empty result set, and the model is left with no source text to spin into an answer. In practice, the tool looks roughly like this: const candidates = await hybridSearch ( query , { version , limit : 12 }); if ( ! hasConfidentMatch ( candidates )) { // best cosine sim < 0.45 return { relevant : false , results : [] }; // model has nothing to work with } const results = await rerank ( query , candidates , 6 ); return { relevant : true , results }; The model isn't asked to behave. The system is shaped so that the only coherent next move, when results come back empty, is to say "the docs don't cover this." Refusal stops being a personality trait you're hoping for and becomes a property of the architecture. Why this particular SDK needed it There's a second failure mode this assistant had to solve, and it's specific to fast-moving libraries. The MCP Ty

LLM Fallback in Production, Agentic eCommerce, and GitHub Copilot for Parallel Agents Today's Highlights This week highlights practical applications and architectural considerations for AI frameworks, focusing on robust LLM deployments and agent orchestration. We cover building resilient multi-provider LLM systems, leveraging agents for dynamic e-commerce, and GitHub's new desktop app for managing parallel AI agent workflows. How I built a 3-provider LLM fallback system in production (and what actually broke) (Dev.to Top) Source: https://dev.to/ayush_notsogreat_b673d5/how-i-built-a-3-provider-llm-fallback-system-in-production-and-what-actually-broke-46jk This article details the implementation of a robust LLM fallback system designed for production environments, addressing the common challenge of provider reliability and API rate limits. The author shares practical insights gained from building Socra, an application reliant on multiple LLM providers. The core of the system involves orchestrating requests across three different LLM APIs, ensuring that if one fails or encounters issues, the system seamlessly switches to an alternative without disrupting the user experience. The piece delves into the specific architectural decisions made to achieve this, including strategies for managing API keys, handling varying response formats, and implementing intelligent retry mechanisms. It also transparently discusses unexpected failures and critical lessons learned during the system's deployment, offering invaluable advice on anticipating real-world production issues beyond theoretical design. This practical guide provides a blueprint for developers seeking to build more resilient and fault-tolerant LLM-powered applications, crucial for maintaining high availability and consistent performance in AI workflows. Comment: Implementing multi-provider LLM fallbacks is essential for production-grade reliability; this article provides practical architecture and lessons from real-world

Claude LLM Execution Harnesses, RAG Rerank, & Browser-based Edge AI Today's Highlights This week's top stories delve into advanced LLM orchestration with Anthropic's execution harnesses, highlight rerankers as a critical RAG pipeline upgrade, and explore practical browser-based AI for sign language recognition without cloud dependencies. Anthropic Explains How Claude Builds Its Own Execution Harnesses (InfoQ) Source: https://www.infoq.com/news/2026/06/claude-code-harnesses/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=global This InfoQ article provides a deep dive into Anthropic's sophisticated orchestration system designed for managing multi-step processes with large language models (LLMs) like Claude. It details how the AI company constructs "execution harnesses" that enable Claude to chain together various operations, handle complex tasks, and recover from errors, going beyond simple prompt-response interactions. The system effectively functions as an internal agentic framework, showcasing advanced patterns for LLM workflow automation and robust production deployment. Understanding these internal mechanisms offers valuable insights for developers and architects aiming to build more resilient and capable AI agents that can tackle intricate, real-world workflows, from dynamic task planning to adaptive execution. It highlights the importance of modularity, self-correction, and tool integration in scaling LLM applications for enterprise use, providing a blueprint for building sophisticated AI agent orchestration layers. Comment: This is a fantastic look behind the curtain at how a leading LLM provider tackles agent orchestration at scale. It underscores that robust LLM applications require sophisticated workflow management, not just better models. RAG Rerank: the Highest-Leverage Upgrade to Your Retrieval Pipeline (Dev.to Top) Source: https://dev.to/dev48v/rag-rerank-the-highest-leverage-upgrade-to-your-retrieval-pipeline-7o5 This Dev.to artic

Pipeline & Prompts | Byte size guides on DevOps, Cloud and AI AI in the Stack #2 ⚡ Byte Size Summary RAG inserts a retrieval layer between your existing runbooks and an LLM — answers come from your documentation, not generic training data, with source citations included. This article builds a complete FastAPI service with /ingest , /query , and /health endpoints, using OpenAI embeddings and Chroma as the vector store. Everything is cloneable from GitHub. The goal is not to replace your runbooks. It is to make them queryable at the moment an incident is happening. I have never met a platform team with bad runbooks. I have met plenty of platform teams where the runbooks exist, are reasonably well written, are stored somewhere sensible — and are still completely useless at 2am when something is on fire. Not because the content is wrong. Because nobody can find the right one fast enough. The search in Confluence returns fourteen results and none of them are titled the way the engineer is thinking about the problem. The person on call is junior and doesn't know the runbook exists. The runbook was written for a slightly different version of the service and nobody updated it. The runbook problem is not a writing problem. It is a retrieval problem. That is exactly the problem RAG was built to solve — and it is one of the highest-ROI first applications of AI in a platform engineering context. Not because it is technically impressive. Because it closes a gap that costs your team hours every month. This article builds a working pipeline. By the end you will have a FastAPI service that takes a natural language question — "why is my pod stuck in CrashLoopBackOff after a config change?" — and returns an answer grounded in your actual runbooks, with the source document cited. Everything is in the GitHub repo agentic-devops What RAG Is — Without the Hype RAG stands for Retrieval-Augmented Generation. Instead of asking an LLM a question and hoping its training data contains the answ

Optimizing RAG Pipelines, Migrating AI Agents, and LLM-Powered Troubleshooting Today's Highlights This week's highlights cover advanced strategies for building and maintaining robust AI systems, from fine-tuning RAG pipelines to orchestrating agent migrations. We also explore practical, real-world LLM application in IT operations. A Cognitive Benchmark for Code-RAG Retrieval: Part 2 — Why Model Rankings Depend on the Pipeline (Dev.to Top) Source: https://dev.to/miftakhov/a-cognitive-benchmark-for-code-rag-retrieval-part-2-why-model-rankings-depend-on-the-pipeline-12a4 This article delves into the critical but often overlooked aspect of RAG (Retrieval Augmented Generation) performance: how the entire pipeline, not just the underlying LLM, dictates retrieval efficacy, especially in code-RAG scenarios. It introduces a cognitive benchmark for code retrieval, moving beyond simple keyword matching to evaluate how well a RAG system understands developer intent when querying unfamiliar codebases. The core insight is that model rankings are highly dependent on the complete RAG pipeline design, including chunking strategies, embedding models, and retrieval algorithms, rather than solely on the base LLM's capabilities. For developers building code-centric RAG applications, this implies a need for holistic pipeline optimization. The article emphasizes that focusing on individual components in isolation may lead to suboptimal results. It encourages a structured approach to benchmarking that reflects real-world developer queries and challenges, such as understanding system behavior rather than just file names. This technical perspective is crucial for anyone looking to deploy robust and performant RAG systems for code generation, search augmentation, or automated code understanding. Comment: This is a crucial read for anyone moving beyond basic RAG demos. It highlights that success in production RAG systems, particularly for code, is all about the pipeline engineering , not just

"Chat with your PDF / your notes / your docs" is everywhere. Today we build it from scratch and you'll see it's just three moves : retrieve, then generate — with one prompt trick that stops the hallucinations. This is Day 46 of TechFromZero. Yesterday (Day 45) we built the retrieval half with pgvector. Today we add the answer half and host it on Supabase. RAG in one line Find the relevant chunks of your documents, paste them into the prompt, and tell the model to answer using only those. That's Retrieval-Augmented Generation. The "augmented" part is just stuffing real context into the prompt so the model isn't guessing from memory. 1. Storage: Supabase is Postgres, so pgvector is one click Supabase is hosted Postgres with an auto-generated API. Because it's just Postgres , vector search needs no separate database: create extension if not exists vector ; create table documents ( id bigserial primary key , content text , embedding vector ( 384 ) ); -- one RPC the app calls to get the closest chunks create function match_documents ( query_embedding vector ( 384 ), match_count int ) returns table ( id bigint , content text , similarity float ) language sql stable as $$ select id , content , 1 - ( embedding <=> query_embedding ) as similarity from documents order by embedding <=> query_embedding limit match_count ; $$ ; 2. Ingest: chunk → embed → store Split your docs into paragraph-sized chunks, embed each with a free local model (all-MiniLM-L6-v2 via Transformers.js — no key, nothing leaves your machine), and insert the row + vector: const embedding = await embed ( chunk ); // 384 numbers await supabase . from ( " documents " ). insert ({ content : chunk , embedding }); Chunk size matters: too big buries the answer in noise, too small loses meaning. A few hundred characters is a good start. 3. Retrieve + Generate (the payoff) Embed the question with the same model, ask Supabase for the closest chunks, then hand them to the LLM: const query_embedding = await embed ( que

Extends an earlier model-selection benchmark to three model families (Japanese / Western / Chinese) on a Japanese RAG task. Repo + raw results: https://github.com/elvisyao007/eval-driven-llm/tree/main/reports/model-selection-v2 An earlier post benchmarked local models for a Japanese RAG task and settled on selecting by constraint rather than raw capability. This post widens the field to three families — Japanese-tuned, Western open, and Chinese — and the result forces a distinction that matters more than any single score: model capability and deployment eligibility are two different questions, and conflating them is how people get model selection wrong. Same Japanese RAG task, same judge protocol, same discriminating golden set (oracle 87.5%, only 11% of questions answered by all models — it actually separates the field). hit@5, 8B class unless noted: Model Family hit@5 Swallow-8B Japanese-tuned ~0.53 Nemotron-9B-JP Japanese-tuned ~0.62 ELYZA-JP-8B Japanese-tuned ~0.40 deepseek-r1-8b Chinese ~0.51 Llama-3.1-8B Western ~0.22 Mistral-7B Western ~0.18 gemma4-31b Western (31B) ~0.62 Three things fall out of this, and they don't all point the same direction. 1. At 8B, Japanese fine-tuning is decisive — and generic Western models just aren't competitive The Western 8B models cratered: Llama-3.1-8B at 0.22, Mistral-7B at 0.18, against a Japanese-tuned average around 0.52. That's not a small gap; it's the difference between usable and not. This answers a question people sometimes ask skeptically — why do Japanese-specific models exist when Llama is right there? At the 8B scale, on a Japanese retrieval-grounded task, a generic Western model without Japanese fine-tuning is not in the running. The Japanese tuning is doing decisive work. One honest qualifier on the table: gemma4-31b (0.62) is the one Western model that holds up — but it's 31B, not 8B. It earns its score with 4× the parameters, not with Japanese optimization. So read the table in two tiers: within the 8B class,

Most RAG demos answer "what's the right chunk?" Very few can answer the two questions a regulator or an auditor will actually ask: Replay this decision — show me the exact, complete record of how this answer was produced. Reconstruct the past — what did your system know at the moment it answered, not what it knows now? I got tired of hand-waving at both, so I shipped two pre-registered, deterministic benchmarks alongside JAMES , my local-first, audit-native Graph-RAG. Pre-registered means the metrics, scenarios, and decision rules were locked before the numbers came in — no post-hoc story-fitting. RAB — Replayable-Audit Benchmark RAB measures whether your audit trail is good enough to replay a decision, with three deterministic metrics: Metric What it checks EU AI Act AC — Audit Completeness Is every decision-relevant event logged? Art. 10 RF — Replay Fidelity Can you re-derive the answer from the log alone? Art. 12 PC — Provenance Coverage Does every claim trace to a source? Art. 19 The three metrics map verbatim to EU AI Act Articles 10, 12, and 19 — record-keeping obligations that apply from 2026-08-02 (per Article 113). Scenario S1 result: AC RF PC JAMES 1.000 1.000 1.000 Baseline-0 0.275 0.000 0.000 (vanilla default-logging) The gap is the whole point. "We have logs" (AC 0.275) is not the same as "we can replay the decision" (RF 0). Default application logging gets you a partial event trail and zero replay/provenance — which is exactly the failure mode an Article 12 audit would surface. LRB — Lifecycle Retrieval Benchmark RAG facts go stale. A policy is superseded, a price changes, a spec is revised. LRB asks: when you query as of a point in time, do you retrieve the fact that was valid then , or whatever overwrote it? Three systems compared: V — Vanilla : no time handling. N — Naive-supersede : newest fact wins. J — JAMES : validity-window retrieval ( reconstruct_graph_at(t) ). The R@1 ordering V < N < J holds across 4 model families × 4 scale points (a 12.5×

AI Agents Level Up Workflows: Terraform MCP, WebMCP, Pinecone Integrations Today's Highlights This week showcases significant advancements in AI agent orchestration and workflow automation, with new tools enabling AI to manage infrastructure, interact with the web, and leverage enterprise data. These developments highlight the growing maturity of applied AI frameworks for real-world production use cases. Terraform MCP Server Enables AI Assistants to Interact with Terraform Infrastructure (InfoQ) Source: https://www.infoq.com/news/2026/06/terraform-mcp-server-ga/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=global The Terraform MCP (Machine Code Platform) Server has recently achieved general availability, marking a significant step forward in the integration of AI assistants with infrastructure-as-code paradigms. This new server allows AI agents to directly interpret and execute operations on Terraform-provisioned infrastructure, providing a robust and standardized interface for AI-driven automation. Instead of relying on complex scripting or indirect API calls, AI assistants can now receive natural language instructions, translate them into appropriate Terraform commands, and manage resources like virtual machines, networks, and databases across various cloud providers. This capability introduces unprecedented potential for advanced workflow automation within DevOps environments. Teams can leverage AI for tasks ranging from autonomous resource provisioning based on demand surges to intelligent incident response that dynamically scales or reconfigures infrastructure. The MCP Server acts as a crucial middleware, ensuring secure and controlled interaction between intelligent agents and critical infrastructure, thereby reducing manual operational burdens and enhancing system resilience through automated, intelligent responses. This direct interaction paves the way for a new era of self-managing, AI-orchestrated cloud environments. Comment: This i