NVIDIA CUTLASS: High-Performance CUDA Templates for AI Linear Algebra

If you've trained a transformer in the last three years, your GPU spent most of its wall-clock time inside a matrix multiplication. The kernels doing that work were probably written by cuBLAS, generated by a compiler stack like Triton, or hand-assembled on top of NVIDIA's CUTLASS templates. CUTLASS is the one most people don't see directly, but it sits underneath a surprising amount of modern AI infrastructure — from FlashAttention to vLLM to several internal kernels inside PyTorch. What CUTLASS actually is CUTLASS — CUDA Templates for Linear Algebra Subroutines — is a header-only C++ template library NVIDIA publishes on GitHub under Apache 2.0. It is not a drop-in replacement for cuBLAS. cuBLAS gives you a closed-source binary with a stable API: you call cublasGemmEx and you get a tuned kernel. CUTLASS gives you the building blocks to write your own kernel, with control over tile sizes, data layouts, epilogues, and how the kernel decomposes work across the GPU's memory hierarchy. That control is the point. If you're building a custom inference engine and your projection layer needs to fuse a GEMM with a SiLU activation and a residual add, cuBLAS can't fuse the epilogue for you — you'd launch the GEMM, then a separate elementwise kernel, paying twice for global memory traffic. With CUTLASS, the epilogue is a template parameter. You write the fusion once, instantiate the template, and the compiler emits a single kernel. This is why CUTLASS shows up wherever standard cuBLAS shapes don't fit — unusual data types like FP8, custom epilogues, sparse or grouped GEMMs, attention-shaped matrix products. Anywhere the stock library doesn't have what someone needs and the performance ceiling matters, you tend to find a CUTLASS kernel. CUTLASS is not a productivity library. It is a kernel-author's library. If you're writing PyTorch model code, you'll never import cutlass directly — you'll consume kernels that were built on top of it. The audience here is people who write the ker