What Actually Happens When You Call an LLM API
you've felt it. you type a prompt, hit send, and the response starts streaming in under a second....
找到 256 篇相关文章
you've felt it. you type a prompt, hit send, and the response starts streaming in under a second....
This is part of my build-in-public series where I document everything honestly — the problems I face,the observations I make,and what I'm trying to build. There's a moment that happens in almost every class. The teacher finishes explaining something. Looks around the room. And asks: Everyone clear? And the entire class says "yes." Including me. Even when I understood absolutely nothing. The Loop Nobody Talks About Here's what actually happens — at least for me and I suspect for a lot of you reading this: Teacher is explaining a concept. I'm trying to follow. Somewhere in the middle, I lose the thread. Maybe the explanation was too fast. Maybe the concept needed something I didn't know yet. Maybe I just zoned out for 10 seconds and missed the part that made everything else make sense. Now I have two choices: Option A: Raise my hand. Ask the question. Risk looking like I wasn't paying attention or worse ask something that makes me look stupid in front of everyone. Option B: Stay quiet. Nod. Say "yes" when the teacher asks. And hope it makes sense later. I always pick Option B. And then "later" comes — sitting alone at home, textbook open, trying to study for a test and I have no idea where to even begin. The concept is still missing. The gap is still there. But now there's no teacher, no classroom, no one to ask. So I either text a friend (who's also confused), scroll YouTube for 40 minutes looking for the right explanation, or just… close the book and tell myself I'll figure it out tomorrow. I never figure it out tomorrow. This Isn't Just a "Me" Problem I'm an engineering student in Pakistan. Maths, Physics, Chemistry — subjects where one missing concept breaks everything that comes after it. And I genuinely believe most of my classmates feel exactly the same way. We just don't say it out loud. Because saying "I don't understand" in a classroom full of people takes a kind of courage that most of us don't have. So we all nod together. And we all go home confused toget
Series: AI, Ego & Regret — Bonus Chapter Editor's Note: While compiling the old series for the...
As programmers, we love data. We track our commits, our uptime, and our deployment frequencies. But what about our most important "server"—our heart? 💓 The "Quantified Self" movement has led to an explosion of wearable data. However, if you've ever tried to analyze raw heart rate CSVs (often sampled every few seconds), you'll quickly realize that standard relational databases or even pure Pandas can get sluggish once you hit that 100k+ row mark. In this tutorial, we are going to build a high-performance Quantified Self Dashboard . We will leverage DuckDB —the "SQLite for Analytics"—to perform vectorized execution on heart rate data, paired with Streamlit and Plotly for a slick, interactive frontend. We’ll focus on Python data engineering , time-series analysis , and fast SQL processing . Why DuckDB? 🦆 Traditional databases are row-based, which is great for transactions but terrible for analytical queries. DuckDB is a columnar-vectorized query engine . This means it processes data in chunks (vectors) and utilizes modern CPU instructions (SIMD) to crunch numbers at speeds that make standard Python loops look like they're standing still. The Architecture Here is how our data pipeline flows from raw pixels (well, raw CSV rows) to actionable insights: graph TD A[Raw Heart Rate CSVs] -->|Direct Ingestion| B(DuckDB Engine) B -->|Vectorized SQL Execution| C{Data Aggregation} C -->|Moving Averages/Outliers| D[Streamlit App State] D -->|Plotly| E[Interactive Visualization] E -->|User Input| D Prerequisites 🛠️ Ensure you have the following stack installed: Python 3.9+ DuckDB : For the heavy lifting. Streamlit : For the UI. Plotly : For the beautiful charts. pip install duckdb streamlit plotly pandas Step 1: Ingesting 100,000+ Data Points in Milliseconds One of the coolest features of DuckDB is its ability to query CSV files directly without a formal "import" step. This is a game-changer for developer productivity. import duckdb import pandas as pd # Let's assume 'heart_rate.cs
👋👋👋👋 Looking back on your week -- what was something you're proud of? All wins count -- big or small 🎉 Examples of 'wins' include: Getting a promotion! Starting a new project Fixing a tricky bug Found a new song so good it fixed your whole mood 🎵 Happy Friday!
Most developers use AI like a smarter Stack Overflow . Type a question. Get an answer. Go do the work yourself . That's fine but it's the slow way 😩 There's a faster mode, and most people haven't switched to it yet. Diff: Asking & Delegating When you ask an AI : "How do I write tests for my auth module?" You get a nice explanation. Then you write the tests yourself. You're still doing the work 🥸 When you delegate to an AI agent: "Write tests for /src/auth.py . Cover login, logout, and invalid token cases. Run them. If any fail, fix the code until they pass. Tell me what you changed." The agent opens your files, writes the tests, runs them, reads the failures, fixes the code, and comes back to you with a working test suite. You review the result. You didn't do the work. That's the shift 🙂↔️ It sounds small. The time difference is huge . How to write a good delegation Every delegation that works has four parts . Think of it like giving a task to a new team member: Goal: what should it produce? Scope: which files or area of the codebase? Success condition: how do we know it's done correctly? Report back: tell me what you changed and why. Here's what that looks like in practice: Debugging: "Here's the error and the stack trace. Find the root cause, fix it, and explain what was broken." Why this works: You're not asking what the error means. You're handing over the whole problem, find it, fix it, explain it 😎 Refactoring: "Refactor this file. Max two levels of nesting. No single function longer than 30 lines. Update every call site in the codebase." Why this works: The constraints are clear and checkable . The agent knows exactly when it's done 🧐 Database migration: "Write a migration script for this schema change. Make it idempotent. Run it against a local test database and confirm it succeeds." Why this works: You gave it a way to verify its own work before coming back to you 🤔 PR review: "Read this PR diff. Find anything that could fail in production. Write the tests
For a long time, I thought programming wasn't for people like me. Not because I wasn't interested in technology. Not because I didn't enjoy solving problems. But because I kept hearing the same thing over and over again: "You need to be good at math to become a programmer." The more I heard it, the more I believed it. Whenever I saw developers building websites, apps, or cool projects, I assumed they were all math experts. 🧮 I imagined them solving complex equations all day while I struggled with basic math concepts. So before I even wrote my first line of code, I had already convinced myself that programming probably wasn't for me. And honestly, I think many beginners feel the same way. 🤔 The Fear Was Bigger Than The Reality When I finally started learning programming, I expected math to be my biggest challenge. It wasn't. My biggest challenge was understanding why things weren't working . I spent hours trying to figure out: Why isn't this button working? 🖱️ Why is this variable undefined? 🤨 Why did this code work yesterday but not today? 😅 Why did fixing one bug create three new bugs? 🐛 Very quickly, I realized that programming wasn't testing my math skills nearly as much as it was testing my patience and problem-solving ability. Most of the time, the challenge wasn't: "Can you solve this equation?" It was: "Can you figure out what's causing this problem?" 🧠 Logic Matters More Than Most People Think One of the biggest lessons I learned is that math and logic are not exactly the same thing. Yes, math uses logic. But you don't need to be a math genius to think logically. Programming is often about breaking a big problem into smaller, manageable pieces. For example: If a user clicks a button, what should happen next? If data is missing, what should the application do? If an error occurs, how should it be handled? That's logic. You're constantly thinking: "If this happens, then what should happen next?" And honestly, that's a huge part of software development. Some of
TL;DR Welcome back to Dev Opportunity Radar. This is a weekly series where I share opportunities,...
I'm a professional developer, and AI has significantly increased my output—I'd say by maybe 30 or 40 percent. GitHub Copilot has significantly changed the way I work with code. However, I take pride in producing high-quality code quickly, which is why my rates are high. Using AI helps me increase my output while maintaining that level of quality. My take on AI is that it is not going to replace humans anytime soon. It is, however, putting significant pressure on the economy. Previously, setting up a functional, decent-quality project without much complexity took time—at least weeks. Now, such tasks are incredibly fast and easy; anyone can set them up in a few minutes using AI, even without any coding knowledge. Success in most fields, however, is not just a measure of how fast you can build; it's also about how well you can execute. Current AI can offer advice, but it still cannot execute for you. Market success requires sensitivity, context, and adaptability. AI can help significantly if you know how to ask the right questions. But the economy is made of people, not AI (yet). To earn money, someone must give you money because they value what you offer. The arrival of LLMs hasn't changed this. I feel the pressure. The corporation I work for is pushing for AI adoption, and the initial drawbacks and realizations are already becoming apparent. First point: Customers, at best, don't care about your AI. They don't want it. Second point: AI succeeds at making developers more productive but fails with higher complexity—though not for the reason people usually think. With the right prompt, GPT-5.4 can create fairly complex solutions, even more complex than many corporate business processes. The real reason is that, at a certain level, complexity lies not in the total amount of information in the system, but in how the human aspect of the business translates when you try to formalize higher-level context. This is something most developers don't see (or care about). For examp
The code runs. That's not the question. There's a failure mode with AI-generated code...
Picture this: You just spent weeks building an awesome new feature. It's fully tested and ready to go. But when you hit "Deploy," your entire application goes down for 5 minutes, and your users are met with a blank loading screen. Not a good look. In the world of DevOps and Cloud Infrastructure, how you roll out updates matters just as much as the code itself. AWS Elastic Beanstalk gives us 5 distinct deployment policies to handle this smoothly. Let's break them down from simplest to most robust so you know exactly which one to pick for your next project. 1. All at Once (The Speed Demon) This is the simplest method. Elastic Beanstalk takes all your existing servers, shuts them down, deploys the new code, and boots them back up simultaneously. [ Old App ] ─> SHUTDOWN ALL ─> DEPLOY NEW ─> [ New App ] The Good : It’s incredibly fast. The Bad : Your app goes completely offline during the deploy. When to use it : Only in development environments where downtime doesn't matter. Never use this in production! 2. Rolling (The Line Worker) Instead of updating everything at once, Beanstalk splits your servers into batches (e.g., 2 at a time). It takes the first batch offline, updates them, brings them back online, and then moves to the next batch. Batch 1: [ Updating... ] Batch 2: [ Running Old App ] Batch 1: [ Running New App ] Batch 2: [ Updating... ] The Good : No total downtime! Your app stays online. The Bad : While a batch is updating, your overall server capacity drops. Plus, users might experience a "mixed state" where refreshing switches them between the old and new version. When to use it : Production environments that can handle a temporary dip in bandwidth. 3. Rolling with Additional Batch (The Safe Substitution) To fix the capacity problem of standard rolling, this policy launches a brand new batch of instances first. Once those new servers are healthy and running the updated code, Beanstalk starts rolling the update through the old servers. [Old Servers: 100% Capa
I started my career in the late 2010s, and I have had a front-row seat to the growth of the industry that has given me everything: software engineering. Looking back over the last decade, I have mixed feelings about some of the calls I made. And I am seeing the same patterns play out again now. So for engineers who are confused about where this is headed and how to navigate it, here is how I think about it. Generalist SWEs were a product of cheap money The late 2010s, I saw an huge amount of startup funding, globally. Flipkart, Snapdeal, Jugnoo, and hundreds of others were scaling hard and one hiring pattern I saw was that: everyone wanted generalist software engineers. People who could easily get upto speed across the stack.- backend, frontend, infra, deployment and simply ship. Building software was expensive. Automation was still low. Kubernetes had just gone mainstream. Shipping still meant a surprising amount of manual work: SSH-ing into servers, copying artifacts around, running mvn builds by hand, debugging deployments straight in production, duct-taping infrastructure that today you would never touch. Companies fought over engineers who maximized feature throughput. Breadth was a premium, because every extra engineer increased the rate at which software got built. It helped because the money was also free and VCs rewarded growth over efficiency, and hiring software engineers in bulk was the easiest way to spend it. Pull up a resume from an engineer who started around that time and you will usually see the same shape: a long list of technologies and frameworks, broad and adaptable, but rarely deep in any one thing. There was no incentive to go deep. LLMs Changed The Dynamics LLMs did not kill software engineering. It compressed the cost of implementation. The work that got hit first was the work that was already standardized: CRUD apps; API integration and glue code; Framework-heavy backend work; Frontend scaffolding; Standard architectural patterns. What use
We talk a lot about what AI can build. Code generation. Faster prototypes. Automated debugging. One-shot apps. Entire products created in hours. And yes, AI is powerful. But there is a quieter cost we are not talking about enough: AI hype is starting to weaken the motivation to learn core engineering deeply. That should worry us. 1. The "Why Bother?" Mindset When the dominant narrative says AI can generate code instantly, many engineers start asking: Why should I spend months mastering frameworks, architecture, databases, networking, or system design? At first, that sounds practical. If a tool can help, why not use it? But there is a difference between using AI to move faster and using AI to avoid understanding. Core engineering is not just about writing code. It is about knowing why something works, where it breaks, how it scales, and how to fix it when the generated answer is wrong. If we skip that learning, we create engineers who can prompt systems but cannot reason deeply about systems. That is a dangerous tradeoff. 2. The Funding and Praise Monopoly Right now, AI gets most of the attention. Budgets move toward AI. Leadership praises AI initiatives. Teams are pushed to add AI features even when the fundamentals are still weak. Meanwhile, excellent core engineering often goes unnoticed. The people improving reliability, performance, developer experience, infrastructure, security, and maintainability are still doing high-impact work. But in many places, that work is being treated as less exciting simply because it is not branded as AI. This creates pressure. Engineers feel they must pivot to AI, not always out of interest, but out of fear. Fear of being left behind. Fear of being replaced. Fear that their existing expertise is no longer valued. That is not innovation. That is anxiety disguised as progress. 3. The "AI-First" Discount There is another subtle problem. When someone builds something impressive today, the reaction is often: AI probably generated that.
Introduction I'm Ahmer. I'm 19, I'm doing a 4-year Software Engineering degree at a fairly...
Complete preparation breeds complacency. What is seen every day no longer raises suspicion. The...
I applied to a YC W25 startup the normal way. Filled out the form, wrote a decent cover letter, hit submit. Silence. While waiting, I found their open-source repo on GitHub. Read through the codebase out of genuine curiosity I wanted to understand what they were actually building. Found a bug. Fixed it. Opened a PR. It got merged in 2 days. They still hadn't replied to my application. Here's what that taught me about job hunting in 2025: A cover letter tells someone what you claim you can do. A merged PR shows them. One of those gets read. The other gets filed under "maybe later" -which is just "no" with extra steps. I'm not saying cold applications are dead. I'm saying they're the last resort, not the first move. If a company has a public repo, you have a backdoor that most applicants don't even think to try. Read the code deep and find something small but real. Fix it and Open a PR. Now you're not a stranger in their inbox you're someone who already ships for them. The reply came eventually, by the way. But by then, the maintainers already knew my GitHub handle. That matters more than you think. Have you ever landed something through a contribution instead of an application? Drop it in the comments curious how many people have done this.
This is the third piece in a sequence. The first asked whether Sloan had flagged anyone else — it...
What Are the 36 Stratagems? If you've heard of The Art of War, think of the Thirty-Six...