标签：#optimization

LLM costs scale linearly with usage. A system processing 10,000 requests a day at $0.01 per request costs $100 daily — $365 a year. At enterprise scale, that's over $10,000. Cost optimization isn't about cutting corners. It's about spending tokens where they matter. Every token you waste is a token you could have spent on a better answer. Token budgeting The simplest way to control costs is to set limits. Per session, per task, or per day. Strategy 1: Per-Session Budgets Per-session budgets are straightforward: class SessionBudget : def __init__ ( self , budget_tokens : int = 10000 ): self . budget = budget_tokens self . used = 0 def allocate ( self , tokens : int ) -> bool : if self . used + tokens <= self . budget : self . used += tokens return True return False def remaining ( self ) -> int : return self . budget - self . used Strategy 2: Per-Task Budgets Per-task budgets are more useful. Different tasks need different amounts of context: task_budgets : classify : max_tokens : 100 model : qwen2.5-1.5b summarize : max_tokens : 500 model : qwen2.5-7b code_review : max_tokens : 2000 model : qwen2.5-coder-7b reason : max_tokens : 4000 model : qwen2.5-32b Strategy 3: Adaptive Budgets Adaptive budgets adjust based on what actually happens. If classification tasks consistently use 80 tokens, stop allocating 100: class AdaptiveBudget : def __init__ ( self ): self . task_history = {} def allocate ( self , task_type : str ) -> int : if task_type in self . task_history : return int ( self . task_history [ task_type ] * 1.5 ) return 1000 def record ( self , task_type : str , tokens_used : int ): if task_type not in self . task_history : self . task_history [ task_type ] = tokens_used else : self . task_history [ task_type ] = ( 0.9 * self . task_history [ task_type ] + 0.1 * tokens_used ) The exponential moving average (0.9 weight) means recent usage matters more than history. Adjust the weight based on how volatile your workloads are. API vs local inference Local inference

I Cut My Next.js + Supabase App Load Time by 73% - Here Are the 5 Techniques That Actually Worked Last month, our SaaS dashboard was embarrassingly slow . 4.2 seconds to load the main page. Users were complaining. Conversion rates were tanking. Today? 1.1 seconds . 73% faster. Here's exactly what worked (and what didn't). The Problem: Death by a Thousand Database Calls Our dashboard showed user projects, team members, recent activity, and notifications. Sounds simple, right? Wrong. Each component was making its own database calls. The projects list fetched projects, then made separate calls for each project's stats. The activity feed loaded events, then fetched user details for each event. Classic N+1 query problem, but worse. Technique #1: Strategic Data Fetching Consolidation Before: 47 database calls to load the dashboard After: 3 database calls The fix wasn't fancy. We consolidated related data into single queries using Supabase's nested select syntax: // ❌ Before: Multiple separate calls const projects = await supabase . from ( ' projects ' ). select ( ' * ' ) const stats = await Promise . all ( projects . map ( p => supabase . from ( ' project_stats ' ). select ( ' * ' ). eq ( ' project_id ' , p . id )) ) // ✅ After: Single consolidated call const projects = await supabase . from ( ' projects ' ) . select ( ` *, project_stats(*), team_members(count), recent_activity:activities(*, user:users(name, avatar_url)) ` ) . limit ( 10 ) Result: Dashboard load time dropped from 4.2s to 2.8s (33% improvement) Technique #2: Aggressive Caching with Smart Invalidation Most dashboard data doesn't change every second. We implemented a three-tier caching strategy: // Static data: Cache indefinitely const categories = await supabase . from ( ' categories ' ) . select ( ' * ' ) . cache ({ revalidate : false }) // Semi-static data: Cache with revalidation const userProjects = await supabase . from ( ' projects ' ) . select ( ' * ' ) . eq ( ' user_id ' , userId ) . cache ({ revali

Next.js + Supabase Performance Optimization: From Slow to Lightning Fast Last month, I optimized a Next.js + Supabase application that was frustratingly slow. Initial page load took 4.2 seconds, Lighthouse performance score was 62, and users were complaining. After applying these optimization techniques, we achieved: 70% faster load times (4.2s → 1.3s) Lighthouse score of 96 (up from 62) LCP improved by 65% (3.8s → 1.3s) 50% reduction in database queries Here's exactly how we did it. The Starting Point: Measuring Performance Before optimizing anything, we measured current performance using: Lighthouse (Chrome DevTools): Performance: 62 First Contentful Paint (FCP): 2.1s Largest Contentful Paint (LCP): 3.8s Total Blocking Time (TBT): 420ms Cumulative Layout Shift (CLS): 0.18 Real User Monitoring: Average page load: 4.2s Time to Interactive: 5.1s Database query time: 850ms average The Problems: Unoptimized database queries No caching strategy Large JavaScript bundles Unoptimized images Blocking render paths Too many client-side fetches Let's fix each one. 1. Database Query Optimization Problem: N+1 Queries The biggest performance killer was N+1 queries. We were fetching posts, then fetching the author for each post individually. // ❌ Bad: N+1 queries (1 + N database calls) async function getPosts () { const { data : posts } = await supabase . from ( ' posts ' ) . select ( ' id, title, author_id ' ) // Fetching author for each post = N queries const postsWithAuthors = await Promise . all ( posts . map ( async ( post ) => { const { data : author } = await supabase . from ( ' users ' ) . select ( ' name, avatar ' ) . eq ( ' id ' , post . author_id ) . single () return { ... post , author } }) ) return postsWithAuthors } Impact: 50 posts = 51 database queries (850ms total) Solution: Use Joins // ✅ Good: Single query with join (1 database call) async function getPosts () { const { data : posts } = await supabase . from ( ' posts ' ) . select ( ` id, title, author:users(nam

I run a production multi-agent AI system on a single M1 Mac in Jamaica. 6 autonomous agents. 26 cron workflows. 5-layer persistent memory. All containerized, all running 24/7. I checked my OpenRouter dashboard last week and realized something: I'd processed 2.4 billion tokens across 52 different AI models and spent a total of $0.52 . That's not a typo. Here's exactly where that money went and what it means. The Numbers Metric Value Total Requests 26,600+ Tokens Processed 2.4 Billion Models Used 52 Total Cost $0.52 Cost per Token $0.00000021 Tokens per Dollar 4.6 Million For context: GPT-4 Turbo costs about $0.00001 per token at scale. I'm running at roughly 50x below that rate. Where the $0.52 Actually Went Here's the breakdown by model: Model Requests Tokens Cost openrouter/owl-alpha 1,334 251.2M $0.00 nvidia/nemotron-3-super-120b 32 1.8M $0.00 google/gemma-4-31b-it 47 1.8M $0.00 openai/gpt-5 1 2.8K $0.03 google/gemini-3.1-pro-preview 1 3.2K $0.04 anthropic/claude-opus-4 1 2.0K $0.13 qwen/qwen3.5-plus 1 6.3K $0.01 z-ai/glm-5-turbo 1 3.0K $0.01 moonshotai/kimi-k2.5 2 4.1K $0.01 google/gemini-2.5-flash 2 5.5K $0.01 +42 other models ~125 ~8.5M ~$0.28 99.6% of my requests cost exactly $0.00. They ran on free-tier models or local inference. The $0.52 comes from a handful of premium model calls: Claude Opus, GPT-5, Gemini Pro. These are reserved for specific high-quality tasks — not everyday inference. What This Would Cost on Cloud Approach Hardware Monthly Cost Annual Cost My setup (M1 Mac) M1 Mac 16GB, local + free tier ~$0.09 ~$1.04 OpenRouter Paid Tier API-only, no local $15-30 $180-360 AWS (g4dn.xlarge + API) 1x T4 GPU, on-demand $350-500 $4,200-6,000 AWS (g5.xlarge + API) 1x A10G GPU, on-demand $700-1,000 $8,400-12,000 A $1,200 laptop replaces $500-1,000/month in cloud bills. The break-even point is about 2 weeks. How the Architecture Works The key insight: not every task needs a $20/month model . My system routes tasks intelligently: Local inference (free): Ollama

Published on : 2026-06-06 Reading time : 6 min Tags : #python #async #performance #optimization The Problem: Fake Async The supabase-async library claimed to be async but actually wrapped synchronous calls with ThreadPoolExecutor: # ❌ Fake async (old code) class SupabaseAsync : def __init__ ( self ): self . _executor = ThreadPoolExecutor ( max_workers = 3 ) async def select ( self , table : str ): loop = asyncio . get_event_loop () r = await loop . run_in_executor ( self . _executor , lambda : requests . get ( url ) # Sync call wrapped as async ) return r . json () Problems : Max 3 concurrent requests (not scalable) Thread overhead per request High memory usage No connection pooling Solution: httpx AsyncClient Use true async HTTP with httpx: # ✅ Real async (new code) import httpx class SupabaseAsync : def __init__ ( self ): self . _client : Optional [ httpx . AsyncClient ] = None async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 ) ) return self . _client async def select ( self , table : str ): client = await self . _get_client () r = await client . get ( f " { self . _base } / { table } " ) r . raise_for_status () return r . json () Performance Gains Metric ThreadPoolExecutor(3) httpx(10) Max concurrent 3 requests 10 requests Avg response 450ms 150ms Memory usage 250MB 180MB Throughput 6.7 req/s 20 req/s Real benchmark : 100 concurrent requests ThreadPoolExecutor: 15 seconds httpx AsyncClient: 5 seconds 3x faster ⚡ Migration Steps 1. Client Initialization with Lazy Loading async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 , max_keepalive_connections = 5 ) ) return self . _client 2. HTTP Methods (GET, POST, etc.) async def _request ( self , metho

Published on : 2026-06-06 Reading time : 6 min Tags : #python #async #performance #optimization 문제: 거짓 비동기 supabase-async 라이브러리는 이름은 async이지만, 실제로는 ThreadPoolExecutor로 동기 호출을 래핑하고 있었습니다. # ❌ 거짓 비동기 (기존 코드) class SupabaseAsync : def __init__ ( self ): self . _executor = ThreadPoolExecutor ( max_workers = 3 ) async def select ( self , table : str ): loop = asyncio . get_event_loop () r = await loop . run_in_executor ( self . _executor , lambda : requests . get ( url ) # 동기 호출을 async로 포장 ) return r . json () 문제점 : 최대 3개 동시 요청만 가능 (동시성 부족) 스레드 오버헤드 (각 요청마다 스레드 생성) 높은 메모리 사용량 해결책: httpx AsyncClient 진정한 비동기 HTTP 클라이언트인 httpx를 사용합니다. # ✅ 진정한 비동기 (수정된 코드) import httpx class SupabaseAsync : def __init__ ( self ): self . _client : Optional [ httpx . AsyncClient ] = None async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 ) ) return self . _client async def select ( self , table : str ): client = await self . _get_client () r = await client . get ( f " { self . _base } / { table } " ) r . raise_for_status () return r . json () 성능 개선 동시성 비교 지표 ThreadPoolExecutor(3) httpx(10) 최대 동시 요청 3개 10개 평균 응답 시간 450ms 150ms 메모리 사용량 250MB 180MB 초당 처리량 6.7 req/s 20 req/s 벤치마크 # 100개 동시 요청 처리 시간 ThreadPoolExecutor : 15 초 httpx AsyncClient : 5 초 → 3 배 빠름 마이그레이션 단계 1. 클라이언트 초기화 async def _get_client ( self ) -> httpx . AsyncClient : if self . _client is None : self . _client = httpx . AsyncClient ( headers = self . _headers , timeout = 30 , limits = httpx . Limits ( max_connections = 10 , max_keepalive_connections = 5 ) ) return self . _client 2. 요청 메서드 async def _request ( self , method : str , url : str , ** kwargs ): client = await self . _get_client () if method == " GET " : return await client . get ( url , ** kwargs ) elif method == " POST " : return await client . post ( url , ** kwargs ) # ... 3. Context Manager 지원 async def clos

Originally published on rikuq.com . Republished here for Dev.to's readers. The FinOps Foundation Framework is the reference architecture for cloud financial management. It's been maintained by the FinOps Foundation (a Linux Foundation project) since 2018 and has matured into the de facto standard most serious cloud cost work is built on. In 2026 it received a substantial refresh that extended its scope from pure cloud spend to include AI/ML, SaaS, licensing, and broader technology categories. For practitioners thinking about formalising FinOps practice — or evaluating providers who claim to do FinOps — knowing what the Framework actually covers is what separates a real implementation from a marketing label. This post walks through the Framework structure, the 2026 updates, and how it applies specifically to AI/ML spend. I'm Ravi. I run three production AI SaaS solo ( Prism , Citare , BatchWise ) and do advisory work on FinOps via rikuq services . The walkthrough below is what I use when teams ask "what does the FinOps Foundation Framework actually look like in practice?" TL;DR Element What it is Phases Three concurrent operational modes: Inform, Optimize, Operate Principles Six foundational principles guiding all FinOps practice Capabilities The functional areas of activity a FinOps practice covers Personas Engineering, Finance, Procurement, Leadership, Operations, ITAM, Sustainability 2026 additions Executive Strategy Alignment, Technology Categories taxonomy, Converging Disciplines recognition AI/ML extension New Technology Category with specifics on GPU/CPU differential, token pricing, make-vs-buy economics The six foundational principles Before the structural mechanics, the Framework's six principles establish the cultural and operational mindset. They're worth knowing because they're how the Framework's authors test whether something is "really" FinOps or just cloud cost cutting. Teams need to collaborate. Engineering, Finance, Procurement, and Business teams w

Token Budgeting: Optimizing Generative AI Costs and Performance Modern generative AI applications offer unprecedented capabilities, yet their operational costs can quickly escalate. The primary driver of these costs, alongside computational resources, is token consumption . Understanding and implementing effective token budgeting strategies is not merely an optimization; it is fundamental to building scalable, efficient, and economically viable AI systems. The Economics of Tokens Tokens are the atomic units of text that large language models (LLMs) process. Whether you're sending a prompt (input tokens) or receiving a response (output tokens), each token incurs a cost. This cost varies by model, but the principle remains: more tokens mean higher expenses and often, increased latency due to longer processing times. Efficient token management directly impacts your application's bottom line and user experience. Strategic Pillars of Token Efficiency Optimizing token usage requires a multi-faceted approach, focusing on both input and output, as well as the underlying model choices. 1. Input Optimization: Crafting Smarter Prompts The most direct way to save tokens is to be judicious with the information sent to the model. Every word in your prompt counts. Concise Prompt Engineering : Avoid verbose instructions or unnecessary conversational filler. Get straight to the point. Instead of: "Hey AI, I was wondering if you could please help me summarize this really long article I have here. It's about quantum computing. Could you make it brief, maybe just a few sentences?" Opt for: "Summarize the following article about quantum computing in three sentences: [Article Text]" This significantly reduces input tokens without sacrificing clarity. Context Window Management : LLMs have a finite context window , the maximum number of tokens they can process at once. Sending an entire document when only a specific section is relevant is wasteful. Employ techniques like: Summarization : P

How the GitHub Issues team used client-side caching, smart prefetching, and service workers to make navigation feel instant. The post From latency to instant: Modernizing GitHub Issues navigation performance appeared first on The GitHub Blog .