Building a Deterministic Security Scanner for AI-Generated Code

Building a Deterministic Security Scanner for AI-Generated Code TL;DR: I built TruffleKit , a CLI security scanner that catches 22 vulnerability classes in under 2 seconds with zero false positives. Here's how the scanning engine works under the hood. AI code generation is producing more production code than ever. But AI models are trained on public code — which means they reproduce the same security mistakes the open-source ecosystem has been making for decades. In my tests, 73% of AI-generated code snippets contain at least one security vulnerability that a standard linter would completely miss. I couldn't find a tool that was fast, deterministic, and had zero false positives. So I built one. The Architecture The scanner is a rule-based deterministic engine written in Python. Each rule is a self-contained module that pattern-matches against a file's AST or raw content. scanner/ ├── __init__.py ├── engine.py # Orchestrator ├── reporter.py # Output formatting ├── rules/ │ ├── __init__.py │ ├── secret_detection.py │ ├── sql_injection.py │ ├── path_traversal.py │ ├── weak_encryption.py │ ├── cors_misconfig.py │ └── ... (22 rules total) └── models.py Key Design Decisions 1. AST-Based Pattern Matching For languages like Python and JavaScript, we parse the file into an AST and match against structural patterns — not regex. This eliminates false positives from strings that happen to look like code. import ast class SQLInjectionRule ( BaseRule ): def check ( self , tree : ast . AST , filename : str ) -> list [ Finding ]: findings = [] for node in ast . walk ( tree ): # Match: cursor.execute(f"...{variable}...") if isinstance ( node , ast . Call ): func_name = self . _get_call_name ( node ) if func_name in ( ' cursor.execute ' , ' db.execute ' , ' connection.execute ' ): for arg in node . args : if self . _is_f_string_or_concat ( arg ): findings . append ( self . _make_finding ( severity = ' high ' , message = ' SQL injection: parameterized query required ' , line = node .