AI systems introduce new validation challenges: model drift, data lineage, and explainability. Here's how we're framing the problem.
Every framework discussed on this blog so far — the CSA shift, the Annex 11 and Annex 22 revision, GAMP 5's new Appendix D11 — arrives at roughly the same three sticking points once the system being validated is an AI model rather than conventional software. Model drift. Data lineage. Explainability. They showed up again, named almost identically, in the FDA's first real attempt to write down what it expects: the January 2025 draft guidance on Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products. That convergence, across regulators and standards bodies that don't coordinate with each other closely, is a decent signal that these three are the real questions, not just the fashionable ones.
Why these three
The guidance frames the core exercise as establishing credibility — trust in a model's performance, backed by evidence, for a specific context of use — and walks through it in seven steps: define the question the model is actually answering, establish exactly where and how it's used, assess how much its output actually drives the decision, evaluate whether the data behind it is fit for purpose, audit how it was designed and built, validate its performance against real test data, and then keep managing it across its lifecycle rather than treating validation as something that finishes at deployment. Model drift, data lineage, and explainability aren't separate concerns bolted onto that framework — they're what steps four, six, and seven actually demand once you get specific about them.
Model drift is the lifecycle problem: a model validated against last year's data distribution doesn't stay validated just because nobody touched its code. Inputs shift, usage patterns shift, and a model's real-world accuracy can degrade quietly, the same way a signed protocol ages the moment the system around it changes. The guidance is direct about this — continuous monitoring has to be able to detect performance degradation and trigger revalidation, not wait for the next scheduled review to notice.
Data lineage is the traceability problem, and it's a step deeper than the record-keeping GxP teams already do. It's not enough to know a model was trained on approved data. Regulators are asking for an immutable trail connecting a specific output back to the specific inputs that produced it and the specific model version that was running at the time — because "the model" isn't one fixed thing once it gets retrained, fine-tuned, or reconfigured. ALCOA+ was built for records that don't change shape. This is data integrity for a system that does.
Explainability is the trust problem: a correct answer nobody can account for isn't good enough when the decision touches product quality or patient safety. The bar isn't full transparency into every weight in a model — that's not realistic for most systems in use today. It's a defensible account of why the model produced this output rather than another one: feature importance, decision-level audit trails, some sense of the model's own uncertainty, and a human positioned to actually catch it if the reasoning doesn't hold up.
Where this gets personal
Entelion is an AI agent that operates inside regulated systems and produces validation evidence. Which means the three problems above aren't a slide in someone else's deck — they're the standard our own system has to clear before anyone should trust what it produces.
Some of that standard is already public, because it shows up on our own security page: customer data is never used to train our models, so what a validation team feeds into Entelion doesn't quietly become part of a shared model shaped by every other customer's data — a meaningfully different shape of drift risk than a model that keeps learning from everyone's inputs at once. Every piece of evidence the agent produces is meant to trace back to a specific requirement and a specific system state, not a generic summary — data lineage isn't an afterthought bolted on for inspectors, it's what the audit trail is for in the first place. And the agent is built to operate the software and produce evidence a person reviews, not to hand down conclusions nobody checks — which is the explainability question answered the only way that actually holds up: a human stays positioned to catch it.
None of that is a claim that validating an AI system is a solved problem, ours included. It's closer to a discipline: the same three questions — is this drifting, can we trace this, can someone account for this — asked continuously rather than once, of every AI component in the loop, including the one doing the validating. That's the framing worth sitting with before the deeper technical conversation about how to actually build monitoring, lineage, and explainability into a validation program that has to hold up under an inspector's questions rather than a demo.
.png)
