今日已更新 246 条资讯 | 累计 21112 条内容
关于我们

variant-confidence v0.1.0: a calibrated confidence layer for variant-effect pathogenicity scores

Fenix 2026年07月18日 11:30 0 次阅读 来源:Dev.to

variant-confidence v0.1.0: a calibrated confidence layer for variant-effect pathogenicity scores State-of-the-art variant-effect models are accurate in cross-validation but their scores are poorly calibrated on temporal data. variant-confidence adds an auditable calibration layer on top of existing predictors — it does not train a new model. The problem: accuracy is not trust Protein variant-effect predictors (AlphaMissense, ESM-1v, EVE) report pathogenicity scores, but a clinician or researcher needs to know how much to trust the number , not just its rank. The gap is calibration, not accuracy: AnnotateMissense (2026) reports MCC 0.94 in cross-validation, dropping to 0.76 on temporal ClinVar, accuracy 0.8798. A raw score near 0.9 may not mean 90% probability. Acting on an uncalibrated score is a risk. What it does variant-confidence wraps an existing predictor's score and produces a calibrated, uncertainty-aware output: Probability calibration (AC1): Platt scaling or isotonic regression over a separate holdout. Selectable, not hardcoded. Conformal prediction (AC1b): coverage 1−α intervals, split or Mondrian by gene. ECE (AC2, AC9): Expected Calibration Error reported before/after calibration, with bootstrap CI and per-bin counts. Bins with too few samples are flagged as low-reliability. Leakage-free split (AC3): temporal split by ClinVar release date with gene isolation — the same gene never appears in both train and test. This is unit-tested. Missing-score handling (AC4): works with AlphaMissense or ESM-1v alone; emits an explicit warning instead of failing silently. Non-deceptive reporting (AC7): every result includes interval/ECE + method + threshold, never a bare calibrated score. Verification (clean clone, no network) Built under a three-party governance loop: implement → independent audit in a clean clone → merge approval. ruff check . → All checks passed. pytest tests/ → 28 passed in 8.90s (offline fixture). An honest bug we caught in audit The first ECE tes

本文内容来源于互联网,版权归原作者所有
查看原文