AI Governance
Model Drift
Definition
Model drift is the gradual decline in an AI model's accuracy over time, as the real-world data it meets in production drifts away from the patterns it originally learned.
What is model drift?
Model drift is the tendency of an AI model's performance to decline over time as the world it operates in moves away from the data it learned from. A model is trained on a snapshot: a fixed set of examples from a particular moment. Reality keeps moving, and as the gap between the two widens, the model's answers grow less accurate, even though the model itself has not changed at all.
It is best understood as a mismatch, not a malfunction. The model is still doing exactly what it was built to do; it is the ground beneath it that has shifted.
Types of model drift
Drift is usually split into two kinds:
- Data drift. The inputs change. The questions, phrasing, or topics coming in no longer look like the training data, even when the underlying answers are the same.
- Concept drift. The answers change. The relationship the model learned no longer holds because the real-world truth has moved: a plan is renamed, a policy is updated, a feature is added.
The two often arrive together, and both produce the same visible symptom: a model that used to be right starts being wrong.
A note for retrieval-based systems: where an AI answers from a live knowledge base rather than only from what a base model memorised, "drift" is usually less about the model and more about the content going stale. Keeping the source current is then the main defence.
How to manage model drift
You cannot prevent the world from changing, so the goal is to notice drift early and correct it. Noticing is a job for AI observability: watch accuracy and resolution over time, and investigate when they slip. Confirming it is a job for AI evaluation: re-run your test set and compare against the baseline. Correcting it means refreshing the data or retraining on newer examples, and for a knowledge-base system, simply keeping that content up to date. Deciding how often to check, and who is responsible, sits within AI governance.
Why it matters
Example
An AI assistant answers a product's questions well at launch. Six months on, the product has new features and renamed plans, but the assistant still describes the old ones, so its answers grow subtly wrong. The model has not changed; the world it describes has, and that gap is model drift.
How Resolve247 helps
Answers that keep up with change
Resolve247's AIChatbot answers only from your knowledge base, so its knowledge is exactly as current as your content. Auto-retraining means that when you update an article, the assistant updates with it, closing the gap that lets answers drift out of date.
30 day free trial, no cc required!
Related terms
Frequently asked questions
What is model drift?
Model drift is the gradual decline in an AI model's accuracy as the real-world data it meets moves away from the data it was trained on. The model itself does not change; the world it describes does, so its answers slowly become less reliable.
What are the main types of model drift?
The two main types are data drift, where the inputs change so incoming questions no longer resemble the training data, and concept drift, where the correct answers change because the underlying reality has moved. They often occur together.
How do you keep a model from drifting?
You cannot stop the world from changing, so the aim is to catch drift early and correct it: monitor performance over time, re-run evaluations to confirm a decline, and refresh the data or retrain on newer examples. For a knowledge-base system, keeping that content current is the main lever.
What is the difference between concept drift and data drift?
Data drift is a change in the inputs, the questions or their phrasing shift, while the right answers stay the same. Concept drift is a change in the answers themselves, because the real-world relationship the model learned no longer holds. Both reduce accuracy, but for different reasons.