Understanding Underfitting and Overfitting: An Introduction

Have you ever trained a model that performed beautifully on your training data but fell apart the moment it saw new data? Or perhaps you built something so simple it couldn't even learn the training data properly? These are the classic traps of overfitting and underfitting — and every machine learning practitioner runs into them. In this article, we'll cover what they are, how to detect them, how to fix them, and where the bias-variance tradeoff ties it all together — with real-world examples and code throughout. What is Model Fitting? Model fitting is the process of training a predictive model on a dataset to find the optimal parameters that best capture the underlying patterns in the data. The goal is simple: the model should generalize well to unseen data — not just memorize the training examples. There are three possible outcomes when fitting a model: Outcome Description Good fit Captures underlying patterns, generalizes well Underfitting Too simple, misses patterns even in training data Overfitting Too complex, memorizes noise, fails on new data What is Underfitting? Underfitting occurs when a model is too simple to capture the underlying patterns in the data. It performs poorly on both the training set and on new, unseen data. Think of it like this: imagine asking a child to predict house prices and they only use the rule "all houses cost $100,000." That model ignores all relevant features (size, location, age) and will be wrong almost every time. Why Does Underfitting Occur? Model is too simple : A linear model trying to fit a curved, nonlinear relationship Too few features : Important variables are left out Too much regularization : Penalizing complexity so heavily that the model can't learn anything meaningful Insufficient training : The model hasn't been trained long enough Real-World Example Suppose you're predicting whether an email is spam. If you only use the feature "email length" and ignore word content, sender, and links, your model will underfit —