标签：#gpu

An LLM-written kernel benchmarked 38% faster on a microbench. Here is what kernel-level validation showed it actually did at runtime. TL;DR Multi-agent LLMs are now writing CUDA kernels (RightNow AI’s AutoKernel, Meta’s KernelEvolve, a multi-agent system claiming 38% speedup on Blackwell). Source-level benchmarks measure clean throughput on a single isolated kernel. They do not measure SM occupancy under co-scheduling, DRAM bandwidth saturation, dispatcher off-CPU during a real serving workload, or NCCL wait correlation with sibling kernels. Kernel-level validation closes that gap: an eBPF trace of the same kernel running under the same workload as production answers all four questions in one capture. The kernel-writing wave Three pieces of work in April surfaced the same pattern: agents generate CUDA kernels, then quote a single throughput number against a baseline. RightNow AI’s AutoKernel (announced Apr 6) – LLM agents iteratively rewrite CUDA kernels for a target metric, claiming substantial speedups on selected microbenchmarks. Meta’s KernelEvolve – similar shape: agents propose kernel variants, rank by throughput, keep the best. Multi-agent system on Blackwell (Apr 29 reports) – claims a 38% speedup on a public kernel benchmark using a coordinated agent setup. All three are real research, all three produce real kernels, and all three report numbers that come from microbenchmarks. The microbench setup is exactly what you want for the optimization loop. It is not what you get in production. What microbenchmarks do not see Run an LLM-generated kernel under nvprof or nsight-compute on an otherwise-idle GPU and the throughput number is real. Put the same kernel in front of a vLLM serving workload and four properties change immediately: SM occupancy under co-scheduling. The kernel that achieves 95% SM occupancy in isolation will achieve 40-50% with three other kernels sharing the same SMs. The optimizer never sees this regime. DRAM bandwidth saturation. A kernel tha

Last week a company nobody can name spent $500 million in a single month on Anthropic's Claude API. Not $500K. Not $5M. Half a billion dollars. In one month. Because nobody set a spending limit. Uber burned through its entire 2026 AI coding budget by April . Four months into the year, done. Microsoft quietly cancelled its internal Claude Code licenses and told engineers to go back to GitHub Copilot. All three stories broke within days of each other, and they all point to the same thing. Token-based billing, when given to an ungoverned team, is a financial weapon pointed at your own company. Every prompt, every context window, every agentic loop gets billed. An engineer running Claude Code seriously can rack up $500 to $2,000 a month just by doing their job well. The answer is not stricter policies. The answer is owning the infrastructure and making tokens free. This article breaks down exactly how to do that for a 100-person engineering team for under $1 million, with real 2026 hardware prices and honest tradeoffs. The Root Problem: You Are Renting the Meter When your team uses Claude Code or any external AI API, you do not own anything. You rent compute by the token. The model is not yours. The data leaves your building on every single request. The bill scales with how well your engineers actually use the tool. That last part is the trap. The better your engineers get at using AI, the more it costs you. Uber's Claude Code adoption jumped from 32% to 84% of their 5,000-person engineering org. That is a success story that turned into a budget crisis. Owning the infrastructure flips this completely. The better your engineers get at using AI, the more value you extract from hardware you already paid for. The Solution: Private On-Premise AI The setup is straightforward: Buy GPU server hardware once Download a state-of-the-art open-source model (free) Run an inference server that speaks the OpenAI API format Point Claude Code, Cursor, or any agent at your local endpoint

Nvidia says the Control Panel's features have been migrated to the Nvidia app.

Joseph Stein discusses engineering an enterprise AI-as-a-Service platform within a private cloud data center. He explains how to maximize underutilized GPU pools via multi-namespace scheduling, leverage Valkey and Lua for atomic priority queuing and backpressure management, mitigate OWASP Top 10 LLM risks via central proxy gateways, and scale batch pipelines using a custom S3-to-Kafka proxy. By Joseph Stein