CrowdStrike's AI Triage Research: How Well Can AI Automatically Judge SOC Alerts?

CrowdStrike's AI Triage Research: How Well Can AI Automatically Judge SOC Alerts? 1. Basic Information Article Title : Teaching AI to Reason Through Detection Triage Publisher : CrowdStrike Publication Date : 2026-08-17 Severity/Importance : High (Practical value as defense research) Original Source : CrowdStrike Related Source : Research Paper (arXiv PDF) Target : Security alerts generated on Windows endpoints Main Models & Methods : Nemotron-3-Nano-30B, Nemotron-3-Super-120B, GEPA, AdaSTaR, LoRA, GRPO 2. Summary in One Sentence This is research on having AI judge whether Windows endpoint alerts are "real attacks" or "harmless false positives." While it showed certain results, misjudgments increased as time passed, revealing that continuous accuracy checks are necessary to automatically close alerts using AI alone. Terms Used in This Report True Positive : An alert that was genuinely an attack False Positive : An alert that was actually not a problem Precision : The rate of items judged by the AI as a specific classification that were actually correct Recall : The rate of alerts that the AI successfully caught out of all alerts that should belong to that classification Confidence : How certain the AI is about its own judgment Calibration : Adjusting and checking whether the AI's confidence matches its actual correctness Distribution Shift : Changes in operational alerts compared to training data due to new attacks or product updates 3. Defense Challenges to Solve SOCs receive a huge volume of alerts, and human review alone takes time The team wants to quickly separate real attacks from false positives It is necessary not only to output classification results, but also to judge which decisions can be processed automatically Accuracy may drop over time or due to environmental differences, even if accuracy is high during training 4. Proposed Methods and Architecture The research combined an AI that classifies alerts with another AI that checks the certainty of the ans