L3: I built continuous runtime monitoring because certification is point-in-time, attacks are runtime
Four independent reviewers said the same thing: "Certification is point-in-time. Attacks are runtime." — @correctover (CrewAI), @wrencalloway (dev.to), @mads_hansen (dev.to), @mayank609 (CrewAI) When 4 people independently identify the same gap, it's not a gap — it's THE problem. So I built L3. The gap My 8-layer Sentinel pipeline audits skills at import time: L1.5-L1.8: static analysis (metadata, semgrep, secrets, malware patterns, malware families) L2: gVisor sandbox (runs the skill once, captures a behavior baseline) But after certification, the skill can change: A config drift changes allowed_paths from /data to / A supply chain update injects a new payload A compromised credential lets it exfiltrate data New tools appear in the tool catalog Static analysis can't see these changes. L2 captured a snapshot. Neither catches drift. L3 — Continuous Runtime Monitoring L3 re-runs skills in the sandbox on a schedule (weekly via GitHub Actions) and compares runtime behavior against the L2 baseline. If behavior drifts, the skill is flagged. 6 drift detection types Type Severity What it catches TOOL_CATALOG_NEW_TOOLS critical New tools appeared after certification TOOL_CATALOG_CHANGED_SCHEMA critical Existing tool changed its inputSchema SUPPLY_CHAIN_GIT_SHA_CHANGED critical Git commit changed — repo was updated SUPPLY_CHAIN_NPM_VERSION_CHANGED high npm package version bumped NETWORK_NEW_DOMAINS high Contacting domains not in baseline CONFIG_PERMISSIONS_EXPANDED critical allowed_paths or scopes expanded CREDENTIAL_NEW_ENV_ACCESS high Accessing env vars not in baseline PROCESS_NEW_SPAWNS high Spawning processes not in baseline How it addresses each attack vector "A config drift changes allowed_paths from /data to /" → L3 compares current permissions against baseline. If paths expanded → CRITICAL alert → skill re-quarantined. "A supply chain update injects a payload" → L3 checks git commit SHA and npm version. If changed since certification → CRITICAL alert → skill must be r