标签：#nvidia

The AI hardware landscape has shifted significantly in 2026, with NVIDIA, AMD, and Intel all competing for developers who need GPUs capable of running local large language models and AI inference workloads. Choosing the right GPU for AI workloads requires looking beyond marketing numbers and focusing on the specifications that actually affect real-world performance. Memory capacity, memory bandwidth, and software ecosystem maturity consistently matter more than theoretical compute peaks when running transformer models locally. This comparison covers the most relevant workstation and prosumer GPUs available in mid-2026, including NVIDIA's Blackwell architecture (RTX 50-series), AMD's Radeon AI Pro R9700, and Intel's Arc Pro B70. The goal is to provide a practical reference for developers deciding which hardware best fits their model sizes, software stack, and budget constraints. Which GPU specifications matter for AI workloads Marketing materials from GPU vendors emphasise AI TOPS and tensor performance, but these metrics rarely tell the complete story for local inference. The specifications below are ranked by their actual impact on running large language models. VRAM capacity VRAM is typically the first limiting factor when running LLMs locally. A model cannot execute entirely on the GPU if it does not fit into available memory. Once model weights spill into system RAM, inference performance drops dramatically. Approximate VRAM requirements for common model sizes: Model Size Recommended VRAM 7B 8-12 GB 14B 16 GB 32B 24-32 GB 70B 48-64 GB 120B+ Multiple GPUs For most homelab users, moving from 16 GB to 32 GB of VRAM provides a substantially larger practical benefit than increasing raw compute performance. A 32 GB GPU capable of running an entire model will often outperform a theoretically faster 16 GB GPU forced to offload tensors into system memory. Memory bandwidth Memory bandwidth determines how quickly model weights can be streamed into compute units. Large tran

Today, I’m talking with Xinzhou Wu, who is the head of automotive at Nvidia. Nvidia is obviously in the news constantly because of the AI boom — it’s one of the most valuable companies in the world, because the AI industry can’t get enough of the company’s GPUs. But Nvidia is also a key supplier […]

Linux 7.2 Improves Multi-GPU Displays, M3 Support, Mesa Rusticl Defaults Arm Mali Today's Highlights This week's hardware and driver news highlights include critical Linux 7.2 kernel updates for multi-GPU display detection and initial support for Apple M3 Pro/Max/Ultra SoCs. Additionally, Mesa's Rusticl OpenCL implementation now defaults to enabling Arm Mali Panfrost driver support, simplifying GPGPU access on embedded devices. Linux 7.2-rc3 Improves Multi-GPU Display Detection (Phoronix) Source: https://www.phoronix.com/news/Linux-7.3-rc3-Multi-GPU-Fix This update for the Linux 7.2-rc3 kernel targets a persistent issue within multi-GPU setups on x86_64 systems: inconsistent display detection. The patch specifically addresses scenarios where certain graphics cards, particularly in configurations mixing integrated and discrete GPUs or multiple discrete cards, would fail to initialize displays correctly or report their presence erratically to the operating system. This is a crucial fix for users and developers deploying workstations with diverse GPU hardware, ensuring more reliable and stable display outputs without manual configuration workarounds. The improvement lies in refining the kernel's ability to probe and correctly identify active display outputs across various GPU architectures. It directly impacts system boot times and user experience by reducing potential black screens or incorrect display layouts. For enterprise and professional users relying on multiple monitors or specific GPU setups for tasks like rendering or scientific computing, this kernel patch is a significant quality-of-life enhancement, removing a long-standing friction point in Linux graphics stack stability. This contributes to the broader goal of making Linux a more robust platform for high-end graphics and compute workstations. Comment: This is a welcome fix for anyone who's wrestled with inconsistent display outputs on multi-GPU Linux machines; it often means less time debugging Xorg conf

Having proven how valuable compute can be, the company finds itself at the center of a market everyone wants to be in — while simpler technologies and less interesting companies get rich on the sidelines.

I'm probably not the only one who checks every few months whether a GPU alternative has finally shipped, mostly so I can cancel a few subscriptions. Nobody doubts it's physically possible or that people have tried. The real question is why it hasn't actually happened, and the answer is economic and structural, not technical. GPUs are not uniquely ideal. They're uniquely general LLM workloads are dense matmul, high parallelism, memory-bandwidth-bound compute. GPUs handle this well but weren't built for it specifically. An ASIC purpose-built for transformer inference should beat a GPU on perf-per-watt and perf-per-dollar, and in narrow slices, it already does: Groq's LPU beats GPUs on single-stream inference throughput for models that fit its architecture Cerebras' WSE cuts interconnect overhead by putting the whole model on one wafer Google TPUs have run production workloads for years and are now sold externally via GCP So specialized hardware can win, sometimes even in production. The real question isn't whether something can beat a GPU, it's why none of these have dented Nvidia's share. 1. The capital barrier Custom silicon needs hundreds of millions in NRE cost, access to TSMC's leading-edge nodes with multi-year allocation queues, and several iterations before a design is commercially viable. That caps the field to hyperscaler balance sheets or venture funding measured in billions. The barrier isn't just the chip either. CUDA, the surrounding tooling, and production pipelines took a decade of capital and engineering to mature, and matching that means rebuilding all of it, not swapping a part. That's a second capital sink on top of the silicon itself. There's also a timing risk specific to fixed-function silicon: if the underlying model architecture shifts significantly, an ASIC taped out for today's transformer variant can become dead weight, while a GPU just needs a software update to run whatever comes next reasonably well. That risk hasn't actually played out,

Over the past week, the AI hardware news I've been tracking adds up to more than $610 billion in capital deployed globally — in just seven days. Not valuations. Not market cap. Actual capital expenditure commitments. Korea $550B, Japan $6B, Qualcomm's new accelerator, Kawasaki Heavy Industries' $1B AI infrastructure bond — this round of moves has already surpassed the wildest half-year of the 2000 dot-com bubble in scale. But this time the money isn't flowing into web pages. It's flowing into chips, memory, and power. Watching all of this over the past few days, I've been thinking: for investors and for builders like us making products on top of AI, what does this gamble actually mean? The Real Story Behind AI Training Bottlenecks: From GPU Scarcity → Memory Scarcity → Power Scarcity Honestly, everyone watches AI through the lens of models, but the real bottleneck was never the models — it's been the hardware. From 2023 to 2025, the bottleneck shifted from GPU scarcity to memory scarcity, and is now pushing toward power scarcity. When GPUs were tight, everyone scrambled for H100s and NVIDIA raked it in — but the part that actually throttled the H100 wasn't the GPU core, it was the HBM high-bandwidth memory. On the B200, the HBM3E stacked on top has its capacity locked up entirely by NVIDIA at SK Hynix, while Samsung is chasing hard but its yields can't keep up. That's why South Korea just committed $518B to build 4 memory fabs plus $52B for the central regions, totaling $550B ( TechCrunch ). This isn't just about filling upstream capacity — the key is that Samsung + SK Hynix are trying to flip themselves from being NVIDIA's downstream suppliers into becoming the dominant players in AI hardware. Why did downstream hardware investment kick off so late? Because for the past two years people were still watching and waiting to see if "this AI hype cycle would cool down again." By 2026, GPT-6, Claude 4, and Gemini 3 are all live, inference costs have come down, user numbe

The news comes about a week after OpenAI announced its own custom AI chip in a partnership with Broadcom.

By Hiral Shah, Senior Director, Product Management, Docusign A major recurring theme among the engineering teams at this week’s AI Engineer World’s Fair in San Francisco is the push to move specialized AI models out of research and directly into high-volume production. At Docusign, that optimization challenge happens at massive scale: we handle millions of transactions daily and have nearly 1.9 million customers in over 180 countries. Organizations have historically lost significant value every year to the friction, delays, and missed obligations that come from treating these agreements as static documents rather than live sources of business data. Much of that trapped value sits inside tables: the pricing schedules, SLA obligations, and contractor rate cards that define enterprise relationships but are often the hardest part of a contract to extract accurately. To solve this, we integrated NVIDIA Nemotron Parse , a vision-language model purpose-built for document understanding, directly into our document processing pipeline. Docusign and NVIDIA took the AI Engineer World’s Fair stage this week to give attendees a look at how the architecture works under the hood. Here’s what that looks like: Why Contract Tables Break General-Purpose AI Contracts routinely contain merged cells, multi-page structures, mixed formatting, and nested layouts that general-purpose vision language models (VLMs) and broad AI models weren't designed to handle. The result is inaccurate extractions that require manual correction, slowing down the workflows they are intended to accelerate. Our teams watch this operational friction play out across real enterprise scenarios every day: System Downtime: When a critical system goes down, operations teams need to know immediately which SLA notification requirements apply and to whom. Resource Tracking: When business stakeholders ask legal what hourly rate was agreed to in a contractor engagement, the answer is often buried deep inside a rate card tabl

June 2026 is shaping up to be the month open models stopped playing catch-up. Three major releases in as many weeks have shifted the landscape, and none of them involve the usual frontier-lab drama. NVIDIA Nemotron 3 Ultra: 550B Parameters, Zero Restrictions On June 4, NVIDIA quietly dropped Nemotron 3 Ultra — a 550-billion-parameter behemoth under a fully permissive open license. That's not "open-weight with strings attached" — it's the most capable model you can download, modify, and deploy commercially without asking permission. Early benchmarks show it competitive with GPT-4.5-class models on code generation and reasoning tasks, while significantly outperforming Llama 4 on mathematical reasoning. If you have the hardware (think 8×H100 nodes minimum), this is the new default for self-hosted enterprise AI. GLM-5.2: China's Answer, MIT License Z.AI launched GLM-5.2 on June 13, and it arrived with full MIT-licensed weights within the week. What makes this noteworthy isn't just the permissive license — it's that GLM-5.2 punches well above its weight class on long-context retrieval and multilingual benchmarks. Developers running locally can deploy it on consumer-grade hardware with quantization, making it a strong contender for privacy-sensitive applications. The API tier starts at ~$18/month, but the real value is in the self-hosted path. Gemini 3.5 Flash Gets Computer Use Google DeepMind also shipped computer use capabilities in Gemini 3.5 Flash this month. Think Claude's computer-use agent paradigm, but running on the fastest Flash-tier model Google offers. Early demos show agents completing multi-step browser tasks — form filling, data extraction, web scraping — at significantly lower latency than competing solutions. The throughline is clear: open models are no longer a compromise . Whether you need 550B monsters for reasoning, MIT-licensed alternatives for compliance, or fast agents for automation, June 2026 delivered on all fronts.

I recently built a multi-agent customer support system on Azure AI Foundry and NVIDIA NIM. First time doing anything like this. Made four predictions upfront about what would happen. Three of them were wrong. Here is what I actually learned. 1. "Tokens" is not a unit of cost It is a unit of work. The price per unit of work varies by 5-10x depending on which model did the work. I was tracking total token count across both the small 9B model and the large 49B model as if they cost the same. They do not. Total tokens went up in the optimized version. Cost in dollars probably went down. I was measuring the wrong thing the whole time. 2. A verbatim hash cache on natural language traffic deflects ~0% of queries I predicted 25-40% cache deflection. The actual number was 0%. Every query in my test set was a unique string, so the hash-based cache never had a single chance to fire. A verbatim cache is not a simpler version of a semantic cache. It is a different thing entirely. If your workload is natural language, build semantic similarity caching from day one, not as an upgrade later. 3. configure_azure_monitor() does not capture OpenAI SDK calls by default You need to install and initialize opentelemetry-instrumentation-httpx explicitly: pip install opentelemetry-instrumentation-httpx==0.61b0 from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor HTTPXClientInstrumentor().instrument() Without this, your App Insights Logs will show customMetric and performanceCounter entries (CPU, memory) but nothing about what your agent actually did. 4. Pin your OpenTelemetry versions or everything breaks Installing opentelemetry-instrumentation-httpx without version pinning pulled in opentelemetry-api 1.42.1. But azure-monitor-opentelemetry-exporter needs opentelemetry-api==1.40. The conflict is silent until things start misbehaving. Pin everything to the 0.61b0 / 1.40.0 line: pip install \ "opentelemetry-api==1.40.0" \ "opentelemetry-instrumentation==0.61b0" \ "opentelem

Eager to find more public AI-related companies that may do as well as Nvidia, Wall Street investors think they've found a winner with Micron.

Public pushback against data centers has emphasized their water and energy consumption, and now Nvidia is highlighting its claim that the Rubin generation reference design for a fully liquid-cooled data center has "eliminated massive amounts of power usage and pretty much all water usage." Still, it doesn't address all of the concerns around AI data […]

Nvidia announced a new cooling system that cuts water use inside the data center. But it does nothing to address AI's biggest water use — fossil fuel power plants.

Nvidia's Nader Khalil — Director of Developer Technologies and co-founder of Brev.dev, acquired by Nvidia two years ago — sat down with The New Stack to talk agents, OpenClaw, and where enterprise AI is heading. His opening line is worth keeping: "An agent is an LLM and a harness. And if you think about that, it involves two things. It involves the loop and the LLM… Each loop should take us closer to our goal." That's not a complicated definition. It's also exactly right — and the fact that Nvidia's internal framing lands here matters more than the quote itself. What actually happened Nvidia has full-time OpenClaw contributors. Khalil: "We have a couple of developers at the company that contribute to OpenClaw full time." That's a real commitment, not a press-release mention. NemoClaw is their enterprise blueprint — a reference architecture for running OpenClaw (and Hermes) in production, with GPU routing, security policies, and a runtime called OpenShell. Khalil traces the harness evolution directly: from ChatGPT's system prompts → memory → file context → Cursor → Claude Code. All of it is harness, not model. The model is constant; the harness is where the product lives. On OpenClaw's PR backlog: "It got more stars than Linux in months… so I think you're gonna see a mountain of PRs." Their response — roll up their sleeves and start merging. Why this framing matters Nvidia makes money when AI compute scales. For that to happen, agents need to work reliably in enterprise environments — and the harness is the reliability layer. Their NemoClaw blueprints aren't a product play; they're an enablement play. Enterprise teams get a reference architecture that works on Nvidia silicon. Nvidia gets demand for the GPUs underneath. It's the CUDA X model applied to agentic AI. The microwave analogy Khalil uses is useful: "when it's your microwave at home, you just go 'Boop, boop. Done.'" Every enterprise will build specialized agents tuned to their domain — CrowdStrike, Cadence, P

nvidia-peermem "Invalid argument" on Ubuntu — Fix GPUDirect RDMA with DMA-BUF TL;DR: If modprobe nvidia-peermem fails with Invalid argument ( -EINVAL ) on a system using the inbox Ubuntu InfiniBand stack ( rdma-core ), the module is not broken and you do not need it. nvidia-peermem requires an API that only exists in MLNX_OFED. On Hopper/Blackwell GPUs with the NVIDIA open driver, use DMA-BUF instead — it does GPUDirect RDMA natively. The one gotcha: you must enable nvidia-drm modeset=1 . Applies to: Ubuntu 22.04 / 24.04, inbox rdma-core stack, NVIDIA open kernel driver, H100 / H200 / B200, ConnectX-6/7 (or any HCA with ODP support). The symptom $ sudo modprobe nvidia-peermem modprobe: ERROR: could not insert 'nvidia_peermem' : Invalid argument dmesg shows nvidia-peermem loaded but registered nothing, or the load returns -EINVAL . GPUDirect RDMA appears to be unavailable. Why this happens (and why it is not a bug) nvidia-peermem is the legacy path for GPUDirect RDMA. It registers GPU memory with the InfiniBand subsystem through a Mellanox-proprietary kernel API: ib_register_peer_memory_client () That symbol only exists in MLNX_OFED's build of ib_core . It is not in the mainline kernel, and it is not in rdma-core , which is the inbox InfiniBand stack on Ubuntu. If you are on the inbox stack, nvidia-peermem was compiled without that API present, so it can never bind and always returns Invalid argument . No module parameter or config change will fix it, because the thing it needs was never there. Do not install MLNX_OFED just to make nvidia-peermem load. That works, but it is the wrong fix — you would be adding a heavy proprietary stack to revive an obsolete module. There is a native path already in your kernel. The fix: use DMA-BUF On Hopper and newer with the open driver, GPUDirect RDMA works through DMA-BUF , a mainline Linux framework. No external module, no MLNX_OFED. Requirements (check these first) NVIDIA open kernel driver (not the proprietary build) nvidia-drm

Nvidia's self-improvement program for robots enlists teams of AI coding agents.

Debt sale set to test investor appetite for further exposure to AI sector amid a deluge of borrowing.

CUDA for AMD Lemonade, Intel Arc Pro Linux Gains, XPU Manager 2.0 Today's Highlights Today's top GPU news highlights include AMD's Lemonade SDK gaining NVIDIA CUDA support, significant performance improvements for Intel Arc Pro GPUs on Linux 7.1, and the major 2.0 overhaul of Intel's XPU Manager for better GPU management on both Windows and Linux. AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support (Phoronix) Source: https://www.phoronix.com/news/AMD-Lemonade-10.7-Released AMD has released a new version of its Lemonade SDK, a powerful local AI server solution designed to leverage AMD's diverse hardware ecosystem, including their CPUs, GPUs, and NPUs. The most significant update in this release is the addition of NVIDIA CUDA support. This integration allows developers to utilize NVIDIA GPUs within their Lemonade-powered local AI deployments, bridging a critical gap in cross-platform AI development. The inclusion of CUDA support is a strategic move, enabling Lemonade to tap into NVIDIA's extensive CUDA ecosystem and a vast array of pre-optimized models and libraries. This means that applications built with Lemonade can now seamlessly target a wider range of hardware, offering unprecedented flexibility for developers working with local AI. For users, it provides the choice to deploy their AI models on either AMD or NVIDIA hardware using a single, unified SDK, expanding the potential reach and efficiency of their AI workloads. Comment: This is a massive step for cross-vendor AI development. Being able to use AMD's Lemonade SDK to deploy local AI models and then seamlessly target NVIDIA GPUs via CUDA truly unifies the AI backend landscape for diverse hardware setups, making it incredibly practical for hybrid environments. Intel Arc Pro B70 Showing Off Some Performance Wins With Linux 7.1 (Phoronix) Source: https://www.phoronix.com/review/linux-71-arc-pro-b70 Recent testing by Phoronix indicates that Intel's Arc Pro B70 discrete GPUs are demonstrating notable perform

In 2025 Google Cloud added G4 , powered by NVIDIA's RTX PRO 6000 Blackwell Server Edition GPUs to their offering, allowing them to offer hardware not only for AI applications, but also for other applications, such as rendering, simulations or gaming. A single G4 instance with one accelerator ( g4-standard-48 ) comes equipped with 48 CPU cores, 180 gigabytes of RAM and 96 gigabytes of GPU memory. This is a lot of resources for a single cloud workstation, that only the most demanding workstreams would utilize. Most professionals who require a graphics accelerator to do their job, don't really need this much compute power for day to day tasks. It wasn't financially reasonable to pay for a G4 instance, when you weren't utilizing all the resources you paid for. If only there were smaller machine types… If only you could share that one very powerful GPU between multiple virtual machines… Introducing fractional VMs! During Google Cloud Next 2026, Google announced GA for fractional G4 VMs and was the first provider to bring vGPU functionality to RTX PRO 6000 accelerators. vGPU stands for virtual graphical processing unit . Just like VMs (virtual machines) are a way to split one physical computer into smaller, independent systems, vGPU allows for a single physical accelerator to be split into 2, 4 or 8 virtual accelerators! The new fractional machine types ( g4-standard-24 , g4-standard-12 , g4-standard-6 ) now allow you to perfectly match the compute capabilities to your needs! Who is it for? The existence of those new machine types makes it much more cost-efficient to move many GPU-dependent tasks to the cloud. Replacing physical workstations in offices with cloud infrastructure is not a new thing , but till now, Google Cloud didn't offer a good platform for those who needed workstations to process images, post-process videos, simulate physics or render 3D graphics. Those users now can get exactly the hardware they need, allowing their companies to move away from maintaini

Vortex 3.0 RISC-V GPGPU, Pragtical SDL GPU Backend, NVIDIA RTX Spark Launch Today's Highlights Today's top stories highlight significant advancements in open-source GPU hardware with Vortex 3.0 adding a 3D pipeline and a lightweight code editor, Pragtical, leveraging an SDL GPU backend for UI rendering. NVIDIA also unveiled RTX Spark, a new 'superchip' aimed at bringing personal AI agents to Windows PCs with accelerated on-device processing. Vortex 3.0 Released As Full-Stack, Open-Source RISC-V GPU Now With 3D Pipeline (Phoronix) Source: https://www.phoronix.com/news/Vortex-3.0-RISC-V-GPGPU Vortex, an open-source, OpenCL-compatible RISC-V GPGPU implementation developed by Georgia Tech, has released its next major version, 3.0. This significant update introduces a full 3D rendering pipeline, marking a crucial evolution from its previous focus solely on general-purpose GPU (GPGPU) compute. The expansion into 3D graphics capabilities makes Vortex a more comprehensive open-source GPU solution, enabling it to handle a wider range of visual and computational tasks. As an open-source hardware design, Vortex 3.0 provides developers, researchers, and hardware enthusiasts with unparalleled access to study, modify, and implement its architecture. Its OpenCL compatibility ensures that it can leverage existing GPGPU codebases, fostering experimentation with RISC-V-based GPU development, custom hardware accelerators, and exploring alternative GPU instruction sets and architectures. This release allows for deeper exploration into the integration of compute and graphics within an open framework. This development is pivotal for the open-source hardware and RISC-V ecosystems. It underscores the growing maturity of RISC-V for demanding compute and graphics workloads, offering a royalty-free alternative to proprietary GPU designs. The inclusion of a 3D pipeline extends its utility beyond just general-purpose compute to full graphics rendering, potentially impacting future embedded syst