标签：#rag

Ever looked at a pile of medicine bottles and wondered, "Is it actually safe to take these together?" Polypharmacy—the simultaneous use of multiple drugs—is a significant challenge in modern healthcare. Misunderstanding Drug-Drug Interactions (DDI) can lead to severe side effects or reduced efficacy. In this tutorial, we are building an AI Pharmacist Assistant , an automated engine that uses Optical Character Recognition (OCR) to scan drug labels and Retrieval-Augmented Generation (RAG) to cross-reference a drug database. By leveraging AI healthcare automation and sophisticated LLM reasoning , we can create a safety net that identifies potential contraindications in seconds. The Architecture 🏗️ The system follows a linear pipeline: capturing raw image data, converting it to structured text, retrieving medical facts from a local SQLite-based knowledge base, and finally, using an LLM to reason about the interactions. graph TD A[Drug Packaging Image] -->|Tesseract OCR| B(Extract Drug Names) B --> C{Search SQLite DB} C -->|Found Interaction Data| D[Context Construction] D --> E[LLM Reasoning Engine] E --> F[Safety Report & Warnings] C -->|Not Found| G[Web Search/LLM General Knowledge] G --> E Prerequisites 🛠️ To follow along, you'll need the following tech stack: Python 3.10+ Tesseract OCR : For extracting text from images. SQLite : To store our curated DrugBank-style interaction data. RAG Pattern : To provide the LLM with ground-truth medical data. OpenAI SDK : For the final reasoning step. Step 1: Extracting Labels with OCR 📸 First, we need to turn those pixels into text. We use pytesseract to handle the OCR process. import pytesseract from PIL import Image def extract_drug_names ( image_path ): # Pre-processing could be added here (grayscale, thresholding) text = pytesseract . image_to_string ( Image . open ( image_path )) # In a real scenario, use an LLM or Regex to pull specific # active ingredients from the raw text print ( f " Detected Text: { text } " ) return t

Companies including Tesla and Base Power are vying for a piece of the rapidly growing market for home batteries. One technology has made it all possible.

OpenAI has cut prices on its frontier models again. If you're running any production workload on the API, your cost assumptions from six months ago are probably stale. The Real Impact of a Pricing Halving A 50% price cut sounds like pure good news, but it changes the calculus on decisions you already made. Projects you shelved because the token costs didn't pencil out deserve a second look. Architectures you built around cheaper, less capable models to save money may now be false economies - the cost gap between "good enough" and "best available" just got smaller. The more interesting shift is for teams running retrieval-augmented generation (RAG) pipelines - systems that pull relevant documents from a database at query time and feed them into the model as context. RAG workflows tend to be token-heavy because every retrieved chunk counts against your input token bill. At the old pricing, teams were aggressively trimming context windows and limiting retrieved chunks to stay within budget. At half the cost, you can retrieve more, keep longer context, and let the model reason over richer information - without changing a line of retrieval logic. Real Example Here's a simplified cost check you can drop into any project that calls the OpenAI API: import openai # Approximate pricing per 1M tokens (check platform.openai.com for current rates) INPUT_COST_PER_1M = 2.50 # update to current figure OUTPUT_COST_PER_1M = 10.00 # update to current figure def estimate_cost ( input_tokens : int , output_tokens : int ) -> float : return ( input_tokens / 1_000_000 * INPUT_COST_PER_1M + output_tokens / 1_000_000 * OUTPUT_COST_PER_1M ) # Example: a RAG call with 3,000 input tokens and 500 output tokens print ( f " Estimated cost per call: $ { estimate_cost ( 3000 , 500 ) : . 5 f } " ) # Run this across your monthly volume to see the real delta Multiply that per-call number by your actual monthly call volume and compare it against what you budgeted. For many teams, the difference will jus

We started building OliverGraph to give teams and their AI agents shared context across GitHub, Slack, docs, and the other places where work happens. At first, we thought the main problem was retrieval. Company knowledge is scattered across GitHub, Slack, docs, tickets, and people so we could connect those systems and get the right context to the agent when needed. But we ran into another problem. Agent runs were becoming another place where important context lived. An agent also gets context directly from the engineer using it. An engineer might tell an agent that the team tried something before, that a customer depends on a certain behavior, or that there's a constraint that isn't documented anywhere else. The agent uses that context while doing the work, but when the run ends, it can disappear with it. The next engineer's agent may see the resulting code without knowing what context was given to the previous agent. This gets messier during outages. Several engineers might be investigating at once with their own agents. One agent rules out a recent deploy while another discovers an issue with a database query. Those findings are now spread across separate agent sessions, and another agent might spend time investigating something that was already ruled out. Humans already deal with this Companies have fragmented context. One engineer remembers an old outage and another engineer remembers that the team already tried an approach but abandoned it. So we ask each other. Who worked on this? Why is this here? Didn’t we try this already? You might not know the answer, but you know John worked on that part of the system. John remembers the PR and the PR points to an incident. People slowly build a mental map of where all that context lives in the company. Agents don’t have that. As everyone starts using more agents, it becomes harder for humans too. My agent may be changing onboarding while your agent is modifying authentication. Another teammate’s agent may have just disc

This is the data appendix for Posts 1–4 . The narrative and takeaways live in the main posts. This page is pure measurement. Eval set: 22 questions (P1) → expanded to 31 questions (P2 onward) Corpus evolution: P1/P2: ~3,700 docs (raw threads + code chunks) P3/P4: 16,315 docs (distilled threads + bursts + code) Quick comparison (same 31-question set) Metric Vector P2 Hybrid P2 Vector P3 Hybrid P3 Hybrid + Rerank (P4) recall@1 0.68 0.61 0.52 0.39 0.87 recall@3 0.84 0.65 0.71 0.65 0.94 recall@10 0.90 0.90 0.81 0.94 0.94 MRR 0.77 0.67 0.63 0.57 0.90 Takeaway: Hybrid alone never beat pure vector on this corpus. Hybrid + LLM rerank is the first clear win. P1 — Naive vector baseline Corpus: 3,000 raw issue threads + 687 code chunks Embeddings: BGE-M3 (1024d), max_seq_length=1024, HNSW cosine Numbers (22 questions) Metric Score recall@10 1.00 (22/22) recall@3 0.95 recall@1 0.77 (17/22) Main k=1 misses Exact error pastes ( TypeError: Object of type int64... , AttributeError: 'Depends'... ) — ranked 4–5 instead of 1 jsonable_encoder code chunk outranked by issues about the function API key header implementation (code vs similar issues) Paraphrase questions (dependency injection outside routes, custom 404) Pattern: Dense search is strong on recall@10 but weak when the query has a sharp lexical signal. Ops notes Ingest wall time ~40 min (GitHub API is the bottleneck) BGE-M3 OOM on Apple Silicon fixed by capping max_seq_length=1024 Python 3.13 + uv editable install issue fixed by pinning 3.12 P2 — Hybrid (vector + FTS + RRF) Corpus: Same size as P1, with better comment pagination and symbol-based code IDs Eval set: Expanded to 31 questions (added exact error pastes + rare identifiers) Numbers Metric Vector FTS Hybrid recall@1 0.68 0.42 0.61 recall@3 0.84 0.48 0.65 recall@10 0.90 0.65 0.90 MRR 0.77 0.47 0.67 Headline: Hybrid is not a strict win over vector-only. Where hybrid helped Exact error pastes (e.g. TypeError: int64 is not JSON serializable ) → moved from rank 5 → 1 Near-d

This English version is an AI translation of my original article on Qiita (in Japanese) . Background I've been experimenting with Zenoh via its Elixir bindings, Zenohex , not for its usual pub/sub use case but for its put / get storage feature. It mostly worked, except every so the state I picked back up was one step behind. Digging into why turned into a fun rabbit hole, so here's the writeup. Reproducing it To keep things simple, strip out the GenServer part entirely and just loop put immediately followed by get on the same key: { :ok , session_id } = Zenohex . Session . open ( config ) Enum . each ( 1 .. 2000 , fn i -> payload = Integer . to_string ( i ) :ok = Zenohex . Session . put ( session_id , key , payload ) { :ok , replies } = Zenohex . Session . get ( session_id , key , 3_000 , consolidation: :latest ) case Enum . find ( replies , & match? (% Zenohex . Sample {}, &1 )) do % Zenohex . Sample { payload: ^ payload } -> :ok % Zenohex . Sample { payload: other } -> IO . puts ( "stale! put #{ payload } but got #{ other } " ) nil -> IO . puts ( "no reply at all" ) end end ) Out of 2000 iterations, a small fraction print stale! — about 78 (3.9%) in one run. The interesting part: querying again immediately afterward almost always returns the correct value (the fastest I measured was a single extra get about 1ms later). So it's not that the value disappears — there's just a small window of lag before the write is actually visible. Why Zenohex.Session.put/4 is a thin Rustler wrapper around zenoh-rust's put . Looking at the NIF implementation : fn session_put ( ... ) -> rustler :: NifResult < rustler :: Atom > { ... publication_builder .apply_opts ( opts ) ? .wait () // <- only waits for the local publish to be queued ... Ok ( rustler :: types :: atom :: ok ()) } .wait() only waits for the local session to finish handing the message off — not for the remote side (the zenohd router backing the storage) to actually receive and apply it. session_get , on the other hand,

"We have memory. We're using RAG." You have retrieval. Those aren't the same thing, and the gap between them is where agents quietly go wrong. Day 2. RAG finds documents that look relevant to your question. That's it . That's the whole job. It's a very good search engine bolted to a very good writer. But consider what it can't do. Your customer changed their pricing tier in March. The old contract is still in the index. The new one is too. RAG doesn't know which one is true, it just knows both are relevant. It hands the model two answers and lets it guess. Memory would know one of those facts replaced the other, and when. That's the difference. Retrieval finds. Memory concludes. Retrieval asks "what documents match?" Memory asks "what do I actually believe, what changed my mind, and when did that happen?" One is lookup. The other is position. This matters because a system that only retrieves can never be wrong and it can never be right either. It has no beliefs to correct. Every contradiction in your data is a contradiction it will faithfully pass along, forever, with total confidence. Day 3: if memory means concluding things, then something has to decide what gets remembered. Right now, in most systems, nothing does. We at AlphaNimble are building Memuron , a memory system for AI agents. This series is thinking behind it, in the open.

I built a RAG assistant, then found out my architecture change made it worse, and I'm glad it happened I recently built a hybrid RAG (retrieval-augmented generation) support assistant for a fictional B2B SaaS platform, "Helix," designed to answer customer-success questions grounded in a 100-document knowledge base of product docs, runbooks, and resolved support tickets. It cleared production-readiness evaluation thresholds comfortably: 0.939 faithfulness and 0.775 context precision on a 50-query RAGAs test set, against required floors of 0.70 and 0.60. But the most useful thing that came out of the project wasn't the passing score. It was a hypothesis that turned out to be wrong, and what I did after finding that out. The setup The pipeline ingests a mixed-format 100-document corpus (Markdown product docs, PDF runbooks, HTML support tickets) into a Pinecone vector index, retrieves relevant context, and generates a grounded, citation-backed answer with an explicit confidence rating via an LCEL chain. Structured output is enforced with Pydantic ( answer , sources , confidence ), using gpt-4o-mini at temperature=0 , because a support assistant answering the same question against the same context should give the same answer every time. Determinism mattered more than creative variation here. Chunking wasn't one-size-fits-all. Three formats needed three strategies: Markdown docs were split by header first, so a chunk never crosses a topic boundary, with a recursive splitter as a fallback for long sections. PDF runbooks (no header structure to exploit) got a straight recursive character split. HTML tickets were kept as one whole chunk per ticket whenever possible, because a resolution often only shows up in the final turn of the conversation, and splitting a ticket risks separating the question from its answer. 5 scanned PDFs with no extractable text layer were detected and skipped gracefully rather than OCR'd, a conscious call I'll come back to. Result: 95 of 100 document

"We don't have access to the data on the hardware/servers," Iron Mountain told Ars.

A hands-on test of BGE-M3 + Qwen3 (RAG vs. direct-context answering) on a real research paper and a full-length book including a retrieval bug hiding in a footnote, and one surprisingly good model behavior. I wanted to answer a simple question: when you feed a document to an AI model, is it actually reading it or just pattern-matching to whatever text happens to look similar to your question? So I built a small open-source pipeline to test this directly. For any document and question, it generates two separate answers: RAG answer: BGE-M3 finds the most relevant chunks of the document, and Qwen3 answers using only those chunks. Direct answer: Qwen3 reads the raw document text directly, no retrieval involved. Both run on a free Google Colab GPU. I kept the retrieval side deliberately "vanilla" fixed-size chunking, plain cosine similarity, no reranking, no fancy tricks so I could see exactly where the basic version breaks before adding any fixes. Before running my first real test, I already knew one thing to guard against: reference lists. Early experimentation (not covered here) showed that a paper's bibliography, once chunked like any other text, can get retrieved as if it were real content a citation for a paper about "text embeddings" can look deceptively similar to a generic question about a document's topic. So going in, my pipeline already strips everything after a References/Bibliography heading before chunking. With that fix in place, I ran two real tests. Test 1: A research paper on Nepali legal machine translation First document: a SIGUL 2024 workshop paper on a bidirectional English-Nepali machine translation system for the legal domain. Question: "What is this paper about?" RAG answer: This paper presents the first transformer-based bidirectional machine translation system for the English-Nepali legal domain, using a custom-built parallel corpus of 125,000 sentences. It achieves encouraging BLEU scores and addresses the scarcity of domain-specific legal tr

The eval that killed the temporal knowledge graph asserted one thing: at time T, the agent should report the state that was true at T. It failed 41% of the time. The graph had the right facts. It just handed the agent the wrong one. That number is what saved us from shipping. Every static retrieval metric looked fine. The graph answered "what is the status of Node A" with a confident, well-formed response. Trouble is, "what is the status" is a temporal question wearing a static question's clothes, and nothing in our test suite had noticed the difference until we wrote a test that actually asked about time. What I expected The pitch for a temporal knowledge graph (TKG) is genuinely good. You store facts as quadruples instead of triples: (subject, predicate, object, timestamp) or, better, (subject, predicate, object, valid_from, valid_to) . Now your agent memory isn't a flat pile of embeddings, it's a structured record of what was true and when. This is the natural next step past pure vector recall, and it slots neatly into the decay-based thinking I've written about before in Eviction Without Deletion . Instead of letting old facts fade by activation weight, you make validity windows explicit. My hope was that the graph would fix the exact failure mode that plagues flat vector memory: the agent confidently recalling a stale fact because it's semantically close to the query. With valid_from and valid_to on every edge, staleness becomes a filter, not a guess. Ask for the state at time T, filter edges where T falls inside the window, done. On paper it's cleaner than a decay curve because there's no fuzziness. A fact is either valid at T or it isn't. Schema-wise, it was simple enough. In a property graph it looks like this: // A temporal fact: Node A was in maintenance for a fixed window MATCH ( n: Server { name: 'node-a' }) CREATE ( n ) - [ :HAS_STATE { status: 'maintenance' , valid_from: datetime ( '2026-07-20T02:00:00Z' ), valid_to: datetime ( '2026-07-20T04:30:00Z' )

Short answer: for semantic search over messy B2B catalog PDFs, I would spend the latency budget during ingestion, preserve page-level evidence, and keep the query path to one embedding plus one vector search; if a catalog must become searchable immediately after every upload, I would choose simpler deterministic chunks and defer enrichment. The decisive constraint is not the PDF parser or the language of the upload service. It is the quality-versus-latency boundary: descriptions often separate a product name, dimensions, compatibility notes, and exclusions across headings or pages, while a buyer expects one coherent result. A fast pipeline that loses those relationships produces plausible but unauditable answers. A sophisticated pipeline that blocks publication for too long fails a different operational requirement. For a Node.js RAG service, I treat the upload worker, embedding adapter, and Postgres repository as replaceable components. The durable contract is the evidence record. Each record needs a stable document version, a stable chunk identity, normalized text, page bounds, catalog identifiers, and the embedding configuration that produced its vector. That is the smallest design I trust for retries, reconciliation, and citations. How should Node.js RAG handle PDF upload chunking metadata and citations? The Node.js boundary should accept an upload, hash the original bytes, write an immutable document version, and enqueue ingestion under an idempotency key derived from the tenant, catalog, and file hash. Parsing and embedding can happen asynchronously. Search should read only a version whose ingestion status was committed as complete; otherwise a retry can expose half a catalog, which is especially awkward when two chunks describe the same SKU differently. Chunking comes after extraction, not during transport. Keep page boundaries from the parser, normalize repeated headers and whitespace without rewriting the source, then group adjacent blocks around product st

Form Energy has landed Google and Crusoe as customers. Now, it has raised $750 million to expand manufacturing to deliver its massive, 100-hour batteries.

A semantic search system has already crossed its security boundary before generation begins: if retrieval admits another customer's chunk, no prompt can make that access legitimate afterward. Short answer: in a multi-tenant ask-your-docs SaaS, derive the customer identity from authenticated server context, bind both the embedding namespace and the mandatory metadata filter inside one retrieval interface, and make the resulting decision reconstructable from an audit record. This architecture decision record treats similarity search as data access, not as authorization. The application may accept a question and optional within-tenant search preferences, but it must never accept the authoritative tenant identifier, namespace, or base filter from the request body. The design aims for an exactly-once effect under retries, explicit failure boundaries, and evidence that can support reconciliation without copying sensitive document text into a second store. How should a Node.js SaaS bind each customer namespace and metadata filter for RAG? The Node.js edge should authenticate the principal, resolve one internal tenant identifier from trusted claims, and construct an immutable request context before calling ingestion, retrieval, reranking, caching, or generation. A client field such as customer_id is merely untrusted data. Even if it happens to equal the authenticated tenant, promoting that field to authority creates a contract that a later handler, worker, or administrative path can misunderstand. The central invariant is compact: every operation that can expose document-derived information requires trusted tenant context. That context selects a coarse namespace, contributes an unavoidable tenant_id metadata predicate, scopes cache and rate-limit keys, and appears in the audit event. User-selected filters may narrow the authorized set by document type, effective date, or label, but they cannot remove or replace the base predicate. Defense in depth matters here — a namespace

Before you start: This picks up where Part 1 left off. From part 1, you would've learned how to setup a Bedrock Knowledge Base in the console. In addition to that, you should have a general understanding of how the ingestion and query pipeline works. Introduction & Motivation I started this project with a singular goal: to build a comprehensive Terraform module that allows developers to deploy the entire infrastructure for a "Chat with PDF" application faster. When Amazon Bedrock was first unveiled in April 2023 , I jumped in immediately. Like many of you, I built several proof-of-concepts (PoCs) through the AWS Console. The UI is amazing for building quick pocs, but once I moved into experimentation, I realized it would be best to quickly setup and tear down the infra. An example use case was testing if there were any cost savings in using S3 Vectors vs OpenSearch and how much cost savings exactly. None of the Terraform modules I found on GitHub ( at the time ) seemed to cover the end-to-end pipeline I was looking for, so I decided to build mine. I'm also big on learning so why not. What Are We Building? A couple of terraform modules to automate everything we clicked through manually in Part 1. One terraform apply brings up the full stack: S3 Bucket : your document store. Encrypted at rest, versioning on, zero public access. OpenSearch Serverless : the vector database. Stores the embeddings Bedrock generates during ingestion. Bedrock Knowledge Base : orchestrates the chunking, embedding, and storage of documents, and retrieval at query time. Ingestion Lambda : triggered automatically when you upload a file to S3. Starts a Bedrock ingestion job so documents are chunked, embedded, and indexed without ClickOps. Query Lambda : accepts a natural language question, calls RetrieveAndGenerate , and returns an answer with source citations. Full source code + ReadMe: Bedrock Project . If you run into issues or want to extend the module, feel free to open an issue. Architectu

Four days after I published a piece arguing LLMs can't make the jump, OpenAI announced that an...

Base Power’s $1 billion round will help the startup ramp production of its home batteries.

If you're running local models through Ollama and tired of Python's overhead, check out langchain-rust . It's a full LLM framework in pure Rust that works great with local models: Ollama support — first-class integration with tool calling, vision, and streaming 9 vector store backends — InMemory, SQLite, Qdrant, ChromaDB, Redis, PGVector, MongoDB, Pinecone, FileVectorStore BM25 keyword search — with Chinese/English tokenization, no external dependency Hybrid retrieval — BM25 + Vector with RRF fusion for better recall GraphRAG — Knowledge graph construction + community detection, all local CorrectiveRAG — Self-correcting retrieval with hallucination detection Code Interpreter — LocalSandbox (subprocess), E2B cloud, or WASM sandbox LocalEmbeddings — Run embeddings without calling an API Plus: LangGraph workflows, MCP client/server, 7 memory types, guardrails, and 12+ built-in tools. Single binary, no virtualenv, no pip conflicts. Just cargo add langchainrust and go. GitHub: https://github.com/atliliw/langchainrust Docs: https://docs.rs/langchainrust

That's a rebuild I shipped. The system: a RAG assistant for fraud analysts — ask it "how do we handle card testing followed by a successful auth?" and it should answer from the team's own SOPs and case history. The complaint: the answers were wrong, therefore the model must be dumb, therefore procurement should buy a bigger model. The model was fine. It was answering perfectly — from garbage context. Walk the forensic trail with me, because every step is checkable on your own system this week. Exhibit A: the chunking was destroying meaning before anything was embedded The ingestion split SOP documents every N characters, mid-sentence. Which means half the vectors in the index encoded fragments like this: chunk_147 = " ...ing to a freight forwarder. In these cases, do NOT " chunk_148 = " cancel the order immediately. First verify the customer via " The policy — don't cancel, verify first — exists in no single chunk. An embedding can't encode a meaning that isn't in its input. Retrieval was being asked to find semantics the pipeline had already shredded. Fix one: chunk on structure (sections, paragraphs), never on character counts, with enough overlap that no rule straddles a boundary. Exhibit B: dense-only retrieval, bimodal queries Fraud analyst queries split into two populations: pattern questions ("high-value order, new account, rushed shipping") and identifier questions ("what's the SOP for decline code 4863?", "rule VEL-013 rationale"). The system was dense-only — and embeddings treat a rare token like 4863 as noise, so identifier queries retrieved similar-feeling chunks instead of the literal match. Half the query population was structurally doomed regardless of model quality. Fix two: hybrid retrieval — BM25 for the identifiers, embeddings for the patterns, reciprocal rank fusion to merge. Exhibit C: nobody could see any of this, because quality was a rumor No golden dataset. No retrieval metric. The system's accuracy was whatever the loudest anecdote said it

We live in an era where our wrists track every heartbeat, step, and sleep cycle. Yet, most of this "Quantified Self" data sits rotting in massive .xml or .json export files that are impossible to read. What if you could simply ask your AI, "How did my resting heart rate trend during the week I was stressed about the product launch?" In this tutorial, we are building a Quantified Self RAG (Retrieval-Augmented Generation) pipeline . We will take fragmented health data from Apple HealthKit and Google Health Connect, process it using DuckDB , and vectorize it into Pinecone using LangChain . By the end of this guide, you’ll have a production-grade Health Data RAG system capable of high-performance natural language queries over your personal biometrics. The Architecture: From Raw Logs to Vector Insights Handling health data at scale requires a robust ETL (Extract, Transform, Load) process. Vectorizing every single heart rate measurement (which can occur every few seconds) is inefficient and expensive. We need to downsample and summarize before embedding. graph TD A[Apple Health/Google Health] -->|Export XML/JSON| B[Raw Data Storage] B --> C{DuckDB Processing} C -->|Cleaning & Downsampling| D[Structured Parquet/JSON] D --> E[LangChain Document Loader] E --> F[OpenAI Embeddings] F --> G[Pinecone Vector Database] H[User: 'Why was my sleep poor last Tuesday?'] --> I[LangChain RAG Chain] G --> I I --> J[LLM Contextual Answer] Prerequisites 🛠️ To follow along, you'll need: Python 3.10+ Tech Stack : Pinecone , LangChain , DuckDB , OpenAI , and Pandas . An export of your health data (Apple Health export.xml or Google Takeout). Step 1: Efficient Data Crunching with DuckDB Apple Health exports are notoriously large XML files. Loading them directly into memory with standard Python is a recipe for a crash. We use DuckDB for its blazing-fast analytical capabilities to filter and downsample our data. import duckdb # Load and parse the XML (simplified logic) # Note: In a real scenario,