Steering Vectors: The Hidden Control Knobs Inside Large Language Models

Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every commit. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product. What if you could change how an AI thinks without retraining it? Not by rewriting prompts. Not by fine-tuning billions of parameters. Not by collecting another mountain of training data. Instead, imagine finding a direction inside the model's internal representation space and nudging the model a little in that direction. A small push. A different behavior. This idea sits at the heart of one of the most fascinating areas of modern AI interpretability: steering vectors . Steering vectors suggest that many behaviors we care about—careful reasoning, honesty, coding style, security awareness, verbosity, and more—may already exist inside a model. The challenge is learning how to activate them. Let's explore what steering vectors are, how they're created, and why they might become one of the most practical tools for controlling AI systems. 1. What Exactly Is a Steering Vector? Large language models process information through layers of high-dimensional activations. At any point during generation, the model's internal state can be represented as a vector containing thousands of numbers. Researchers discovered something surprising: Different behaviors often correspond to different regions of this activation space. For example: Writing Python code Solving math problems Speaking French Explaining concepts carefully Producing insecure code Each tends to produce distinctive activation patterns. A steering vector is essentially the difference between two activation patterns. Suppose we gather examples where the model is: Careful Methodical Thorough and compare them to examples where it is: Rushed Superficial Incomplete The average difference between these internal states becomes a steering vector. At inference time, we can add that vector back into the model's activatio