Local Embeddings vs. API Embeddings — Why I Chose sentence-transformers

Every RAG pipeline needs to convert text into vectors. The question is where that conversion happens. You have two options: run an embedding model locally on your own hardware, or call an API that runs the model on someone else's hardware. Both work. The right choice depends on your constraints — and understanding the tradeoffs is more useful than a recommendation. This article is about why I chose local embeddings with sentence-transformers/all-MiniLM-L6-v2 for this pipeline, and when I'd switch to an API. What Embeddings Actually Do Before the tradeoffs, a quick grounding on what's happening. An embedding model takes text and converts it into a fixed-size vector of floating-point numbers — a list of 384 numbers in the case of all-MiniLM-L6-v2 . That vector encodes the semantic meaning of the text in a way that allows mathematical comparison. Two pieces of text with similar meaning produce vectors that are close together in the 384-dimensional vector space. "Authentication failed" and "login was rejected" are semantically similar — their vectors will be close. "Authentication failed" and "quarterly revenue report" are semantically distant — their vectors will be far apart. This is what makes retrieval work. When you embed a query and search for the nearest chunks, you're finding chunks that are semantically similar to the question — not just chunks that contain the same keywords. The embedding model determines the quality of this semantic matching. A better model produces vectors where semantic similarity maps more accurately to vector proximity. The Local Embedding Choice My pipeline uses sentence-transformers/all-MiniLM-L6-v2 via ChromaDB's SentenceTransformerEmbeddingFunction : from chromadb.utils.embedding_functions import SentenceTransformerEmbeddingFunction embedding_fn = SentenceTransformerEmbeddingFunction ( model_name = " sentence-transformers/all-MiniLM-L6-v2 " ) This runs entirely on your local CPU. No API key, no network request, no cost per embedding,