Half a day chasing AI-model traceability — how a CAPA from data provenance broke the loop and how we fixed it

Half a day lost is the honest cost of treating an AI model like a document. I discovered that the hard way: a CAPA opened for a data-provenance gap rolled forward into missing documentation, which then exposed weaknesses in change control and supplier traceability. This is what happened, what we changed, and the small automation that stopped the loop from repeating. The trigger: a CAPA that looked simple and wasn't An engineer flagged a discrepancy between on-device inference behaviour and the validation test bench. The CAPA looked routine: reproduce, find root cause, correct datasets or model weights. Quickly it turned into: We couldn't identify which training dataset produced the deployed model (no manifest, only folder names). Preprocessing steps changed between runs (different label encodings, a silent resampling step). Model binaries were overwritten in a shared location without an immutable model registry entry. Change control only referenced a release ticket number — not the dataset or container image digest. What began as a data-provenance finding became a documentation finding, then a change-control finding. Auditors would call this a traceability gap. The EU AI Act (and notified bodies increasingly expect traceability for high‑risk AI components) means you must show how a model version ties to the data, the training pipeline, the verification evidence, and the approval record. We didn't have that linkage. By midday my filter coffee was cold and I had a long list of evidence to assemble. Why CMOs see this differently As a CMO handling components and supplier networks, our "models" are often supplier-provided (analytics, inspection classifiers, OCR of COAs), or built from datasets stitched from multiple vendors. The usual eQMS workflows assume a device maker controls the full pipeline. They rarely fit a supplier-heavy reality where: Sub-tier suppliers supply datasets or models. Incoming inspection depends on vendor-provided models for automated checks. Suppl