标签：#t

How I Stopped Burning Cash on Token Limits — A CTO's Field Notes Three months ago, I was staring at our monthly AI bill wondering where it all went wrong. We'd built what I thought was a pretty elegant LLM pipeline. Production-ready, observability wired up, the whole nine yards. Then the invoices started arriving, and I realized I had built a money furnace. Our token consumption was spiking 3x week over week, the 429s were everywhere, and our latency had become a meme inside the company. This is the post I wish I'd had six months ago. If you're a technical founder or a CTO running LLM workloads at scale, bookmark this. I'm going to walk you through the exact architecture decisions, the exact numbers, and the exact code that took us from "this bill is going to kill us" to "oh, this is actually manageable." The Real Problem Nobody Talks About Here's the dirty secret about running LLM-powered products: token limit errors aren't really about token limits. They're a symptom of a much deeper architectural problem. When your app throws "context length exceeded" at 2am, what it's really telling you is that you didn't think hard enough about prompt design, document chunking, model selection, and cost routing on day one. I learned this the hard way. My team was defaulting to GPT-4o for everything because, honestly, it works and the API is reliable. We were paying $2.50 per million input tokens and $10.00 per million output tokens. For a startup processing millions of documents a month, that math is brutal. We were essentially funding OpenAI's next training run with our Series A. The wake-up call came when I ran the actual numbers. Our average request was burning through maybe 8K input tokens and producing 2K output tokens. At our volume, we were spending more on inference than on two senior engineers. That is not a sustainable burn rate for a 12-person company. The Architecture Decision That Changed Everything The first question I asked myself wasn't "which model is cheapest?

Web MCP, DevTools per agenti e Modern Web Guidance: meno hype, più strumenti e metodo. Negli annunci recenti di Chrome è emersa una cosa interessante: al netto delle novità “appariscenti”, ciò che resta più utile per il lavoro quotidiano è quello che migliora workflow, diagnosi e decisioni tecniche . Tre filoni, in particolare, disegnano una direzione chiara: Web MCP , DevTools per agenti e Modern Web Guidance . Di seguito una sintesi ragionata di cosa significano, perché contano per il frontend, e come prepararsi a sfruttarli. 1) Web MCP: il ponte tra agenti e Web (senza incollaggi fragili) Se stai lavorando con assistenti/agentic workflow, oggi il collo di bottiglia è quasi sempre lo stesso: far sì che un agente capisca e usi le capacità del browser e delle app web in modo affidabile. Web MCP punta a risolvere questo punto creando un linguaggio/protocollo comune per esporre “capacità” (capabilities) e strumenti (tools) che un agente può invocare in modo strutturato, invece di basarsi su prompt lunghi, scraping o integrazioni ad hoc. Perché è importante per chi fa frontend Automazioni più robuste : meno script fragili che si rompono al primo refactor del DOM. Integrazioni più standard : se più strumenti parlano lo stesso “dialetto”, il costo di collegare agenti e applicazioni scende. Esperienze utente nuove : assistenti che completano task complessi dentro l’app (es. compilazioni, ricerca guidata, operazioni amministrative) con maggiore affidabilità. Implicazione pratica Inizia a ragionare sull’app come su un insieme di azioni esplicite (es. “crea ordine”, “esporta report”, “filtra dataset”), non solo come UI. Questa mentalità ti rende pronto a esporre capacità in modo sicuro e controllato, quando lo stack lo renderà semplice. 2) DevTools per agenti: debugging e performance nell’era dell’automazione Se Web MCP è il “ponte”, DevTools per agenti è la cassetta degli attrezzi per controllare quel ponte: osservabilità, diagnosi e iterazione rapida su flussi in cui non è

If you run deny-by-default tool guards on AI agents, your refusal is a security decision — not a logging afterthought. I watched one source mutate a malformed tool call ~1,400 times against a production agent in a weekend. Every identical BLOCKED response was feedback for the attacker's automated search: same input shape → same refusal → "colder," changed shape → changed response → "warmer." A Keysight paper (arXiv:2606.20470) quantifies it: deterministic detect-and-block lets attack success rate approach 1 as the query budget grows, because predictable refusals feed model-guided search. Their detect-and-misdirect approach cuts the ASR upper bound by up to ~2 orders of magnitude. The cheap version of the fix, in pseudocode: ​ # BEFORE: a stable refusal = a label for the attacker's search def on_blocked ( call ): return { " error " : " TOOL_CALL_BLOCKED " , " code " : 4031 } # identical every time # AFTER: vary a non-operational response so the deny path isn't a compass def on_blocked ( call ): # return a controlled, plausible-but-non-operational response; # randomize shape/latency so block != stable signal return misdirect ( call , vary = [ " shape " , " delay " , " message " ]) Caveats from doing this in prod: It makes YOUR debugging harder (your own false positives now look noisy too) — log the real reason internally, only vary the external response. Varying text isn't enough if latency still leaks. Treat timing + error-shape as part of the response surface. Open question I don't have a clean answer to: does misdirection just move the oracle one layer up into side channels? I maintain an open-source deny-by-default firewall for agent tool calls (agent-airlock), which is how I had the logs to catch this. The lesson generalizes to any guardrail: a denied call's response is attack surface.

Sometime in 2024 I had a Coinbase wallet on my laptop. I had created the wallet some months back, backed up and all, and just sent very little amount of $ETH to the wallet. Then in 2024 I was paid $100 for a gig which I sent to this wallet, I also sent another $650 worth of cryto as "savings". The next morning I decided to check my "savings", wallet was empty. At first I didn't believe that I was hacked, because I had some $1.50 or so worth of $ETH in the wallet for months and it was safe, so what happened? I traced the transaction history and there was the full detail of how someone sent some $ETH to the wallet, then moved out my "savings" and afterwards also took back the remaining $ETH from the one they had sent in for the attack. I checked on Twitter and saw many other posts of people who had experienced the same exploit, exactly the same pattern... and some of the people who lost their funds were experienced blockchain developers and crypto guys. I made a post about it, told my friends to avoid the wallet and tried to forget about the experience. Blockchain hit instant PMF for many, especially people in parts of the world where there are crazy high fees and bank charges. The moment people tried sending crypto and for a few cents in gas fees, there was no going back for them. The only issue has always been how to secure users' funds, desperate people will always find a way no matter how complex the UX was. After losing my savings I stopped using self custodial wallets and only used Centralized Exchanges for a while. I thought, even though that was a non-custodial wallet, the builders still should have ensured strong security and secure backups, so users don't lose funds unnecessarily. This happened to me when AI and LLMs were still at their early development stages. You can only imagine how sophiscated the attacks have gotten, now that AI and LLMs are very advanced and more capable. To put things in perspective, more than $640 million was lost to deFi hacks and

Last year I was asked to investigate a startup's AWS bill. It had jumped from roughly $200/month to over $3,000 in a few weeks. Nobody knew why. After digging through logs, metrics, and database traffic, I found the culprit: a polling loop with no backoff strategy. The code looked harmless: async function processQueue () { const jobs = await getJobs () for ( const job of jobs ) { await processFile ( job ) } processQueue () } processQueue () At first glance, this seems reasonable. Process all available jobs, then check again. The problem appears when the queue is empty. When getJobs() returned no work, the loop immediately queried the database again. And again. And again. There was no delay, no backoff, and no event-driven trigger. As a result, the service continuously hammered the database looking for work that didn't exist. Each iteration generated: A database query Network traffic CPU usage Logging overhead Additional infrastructure load Individually, each operation was cheap. Executed hundreds of thousands of times per day, they became expensive. The fix was simple: async function processQueue () { while ( true ) { const jobs = await getJobs () for ( const job of jobs ) { await processFile ( job ) } await new Promise ( resolve => setTimeout ( resolve , 5000 )) } } Even better would have been replacing polling entirely with an event-driven design using a message queue. What this incident taught me: 1. Empty queues are production workloads. Many engineers optimize for peak traffic and forget about idle traffic. Systems often spend more time idle than busy. 2. Polling needs backoff. If you're polling, always define what happens when no work is found. 3. Cost bugs rarely look like bugs. Nothing crashed. No exceptions were thrown. The system was technically working exactly as written. It was just doing useless work 24/7. 4. Always monitor cost alongside performance. CPU, latency, and error rates looked normal. The AWS bill was the first real alert. One question I ask

Many Python beginners can write basic programs but get stuck when they try to run a real project on their own laptop. The issue is not always coding. Sometimes the real problem is setup . You may know loops, functions, and lists, but still face problems like: ModuleNotFoundError Python is not recognized Package installed but not working in VS Code Wrong interpreter selected These are common beginner setup issues. Why online compilers are not enough Online compilers are good for quick practice. But real Python projects need: project folders multiple files external packages virtual environments dependency files terminal commands debugging tools Git basics So, when the goal is to build real projects, it is better to move to a local Python setup early. Basic Python project setup flow A simple Python project setup flow looks like this: Install Python Install VS Code Create project folder Create virtual environment Activate virtual environment Install packages Save requirements.txt This setup may look basic, but it prevents many beginner-level errors later. Example folder structure A beginner-friendly Python project folder can look like this: python-project/ │ ├── main.py ├── requirements.txt ├── README.md └── venv/ Here is what each file or folder means: main.py is the main Python file. requirements.txt stores project dependencies. README.md explains the project. venv/ contains the virtual environment. Create a virtual environment Create a virtual environment using: python -m venv venv Activate it on Windows: venv \S cripts \a ctivate Activate it on Mac/Linux: source venv/bin/activate A virtual environment keeps each project’s packages separate. This helps avoid package conflicts when working on multiple Python projects. Install a package After activating the virtual environment, install packages using pip . Example: pip install requests Now create a Python file: import requests response = requests . get ( " https://api.github.com " ) print ( response . status_code ) If

I've compiled a list of the best fitness tech deals this Amazon Prime Day, including smartwatches, walking pads, and recovery gear. You can thank me later.

Summer is for relaxing, not cleaning. Upgrade your backyard setup with a robot that cleans your pool for you.

These camera-equipped feeders will introduce you to birds you never knew were visiting, and many WIRED favorites are on sale for Prime Day.

AWS released Blocks in public preview, an open-source TypeScript framework where each Block bundles application code, local mocks, and AWS infrastructure. Designed for AI agents to write correct backends from the start, it runs locally without an AWS account and deploys the same code to Lambda, DynamoDB, Aurora, and Bedrock with zero changes. By Steef-Jan Wiggers

OpenAI has launched Patch the Planet, a new initiative part of its Daybreak cybersecurity program.

Move over, spring cleaning. These vacuum deals will give your home the cleanest summer yet thanks to Amazon Prime Day.

Your menu bar turns red when PRs pile up Discussion | Link

India is home to about 60% of the world’s wild Asian elephants, and around 80% of the animals’ habitat lies outside protected areas, according to the Ministry of Environment, Forest, and Climate Change. That brings people and wildlife into close contact, and clashes can turn lethal: There have been some 3,000 human casualties in the…

Jos Benschop is climbing a ladder to get to the top of his newest machine. It’s a bit of a schlep. The contraption is the size of a double-decker bus—more than 150 tons of gleaming precision-milled aluminum covered in thousands of snaking tubes, colored cables, and pressurized tanks. From the ground, it looks like a…

Coherence Neuro has started testing a brain-computer interface that could one day use electrical stimulation to prevent tumors from growing.

I've been using DEV.to for a while, and one area that often feels inconsistent is the notification...

Not all mushroom coffees taste like dirt. These are the on-sale blends WIRED tried and actually recommends.

These are the hottest Prime Day deals on our favorite TVs and streaming devices.

Everything is expensive. Treat yourself to one of these WIRED-tested and -approved Prime Day picks under $30.