Detecting PII in Real-World Text

In Part 1 we installed Presidio and ran a basic detection on clean sample text. Real data is messier. Emails have signatures with phone numbers buried in HTML. Support tickets mix PII with technical jargon. Chat logs have informal name references that NER models struggle with. And sometimes the PII isn't in text at all. It's in screenshots and scanned documents. This part covers how Presidio's detection engine actually works under the hood, how to process different text types you'll encounter in production, and how to handle structured data and images. How the Analyzer Engine Works Presidio doesn't rely on a single detection method. It layers three approaches and combines their results. Named Entity Recognition (NER) The NER model (spaCy by default) processes the text and identifies entities based on the language model's training. It's good at catching names, locations, and organizations even when they don't follow a fixed pattern. "John Smith" is easy. "Dr. J. Martinez-Garcia" is harder but the NER model handles it because it understands context and word patterns. The tradeoff is that NER is probabilistic. It can miss unusual names or flag common words as entities. That's why Presidio doesn't stop here. Pattern Matching (Regex) For entities with predictable formats, Presidio uses regex recognizers. Credit card numbers, SSNs, email addresses, IP addresses, phone numbers all have known patterns. A Luhn-validated 16-digit number is almost certainly a credit card. A string matching \d{3}-\d{2}-\d{4} in the right context is probably an SSN. Pattern-based detections typically get higher confidence scores than NER detections because the pattern itself is strong evidence. Context Scoring Here's where it gets interesting. Presidio looks at the words surrounding a potential match to boost or lower confidence. If the text says "my SSN is 123-45-6789," the phrase "my SSN is" provides strong context that the number is actually a social security number and not some random ID. Th