I have shipped ML models that passed every CI check I could write, and then quietly returned bad predictions in production for weeks before anyone noticed. The bug was never in the code. The code was syntactically perfect and the unit tests were green. The bug was that the model in production was answering a different question than the model in development, and nothing in the pipeline was set up to catch that.
This is the failure mode I want to write down, because it is the one that costs ML teams the most time and they almost never see it coming. It is not a framework bug, not a data corruption bug, not a deployment bug. It is a context bug. The training environment and the production environment agree on the code and the syntax, but they disagree on the meaning of the inputs.
The containerisation illusion
Most ML teams I have worked with eventually reach the same conclusion: containerisation (packaging an application and its dependencies into a portable, reproducible unit called a container) solves the deployment problem. Docker (a popular containerisation tool) ensures the same Python version, the same library versions, the same CUDA driver (the NVIDIA software that lets Python talk to the GPU), the same everything on the laptop and the production server. If the model runs in the container locally, it runs in the container in production. This is correct, and it is also completely insufficient for ML systems.
A traditional web service’s behavior is fully encoded in its source code. Given the same request, it returns the same response. Containerise the code, and the behavior is identical everywhere. This is the assumption that traditional CI is built on.
An ML model’s behavior is encoded in three things: the source code, the learned weights (the numerical parameters the model adjusted during training), and the input data. The first two are static. The third one moves. The container ships the same Python and the same weights to every environment, but the input data in production is not the input data the model was trained on. It is close, often very close, but it is not the same. And in ML, “close but not the same” produces “logically flawless, completely wrong” predictions.
This is the gap that containers do not close.
What silent failure looks like
In a traditional web stack, the type system catches most data mismatches. If the API expects an integer and receives a string, the framework throws an exception, the error gets logged, the alert fires, somebody wakes up. The failure is loud.
ML systems fail differently. The model receives a malformed input, runs it through its matrix math, and returns a perfectly well-formed JSON response (a common data-interchange format used by web APIs) with a number in the predicted field. The HTTP status (the standard web response code indicating success or failure) is 200. The pipeline continues. Nothing throws an exception. The wrong answer propagates downstream like any other correct answer.
A concrete pattern I have hit: the training pipeline imputes missing user ages using the dataset’s historical median. The model learns that “missing age” is a signal that means “user like everyone else.” In production, the backend team defaults a missing age to zero so the API does not crash on a malformed payload. The model now sees “age 0” and produces a wildly skewed prediction, because age 0 is a wildly different data point from the median imputation the training data encoded. Nothing throws. The model still runs. The prediction is just wrong.
This is the silent failure mode. The pipeline reports success. The model reports confidence. The user gets a bad recommendation, a wrong classification, a mispriced loan decision. The system does not know it is wrong.
Why traditional CI does not catch it
Standard CI is built on the assumption that correctness is encoded in the codebase. Linters check syntax. Type checkers check types. Unit tests check isolated functions. These tools are exactly the right thing for traditional software, where reading the code is enough to understand the behavior. Reading the code is not enough for ML.
Consider reviewing a pull request that changes how the API handles missing values. A human reviewer reads the diff, sees if age is None: age = 0, and approves. They do not know that the training pipeline imputed missing ages using the median. They do not know that the model’s understanding of the data distribution assumes “missing age” was handled a specific way. The change is correct code that violates an unstated assumption, and the CI pipeline does not have visibility into the assumption.
This is the review gap. Code review checks the code. It does not check the assumptions the code depends on. And in ML, most of the correctness lives in the assumptions, not the code.
Context-aware review, and how it actually works
The fix I keep reaching for is what people in the open source ML community call context-aware review. The idea is straightforward: a code review should not just look at the diff. It should look at the diff alongside the project architecture, the linked issue ticket, and the data assumptions behind the change. If the code violates an assumption, the review should fail it before it merges.
Three inputs feed into the review:
- The repository’s architecture documents (usually markdown or ADRs, short for Architecture Decision Records, files that capture why a system is built a particular way).
- The issue ticket describing the data science task (the “why” and the data assumptions).
- The pull request diff (the “how”).
- A short human-written summary of the model’s assumptions, kept in a file the reviewer can read in under a minute.
A reviewer that has access to all three can spot the “default missing age to zero” bug, because it knows from the ticket that the model was trained on median-imputed ages. A reviewer that only sees the diff cannot.
The implementation pattern that worked for me is to ingest all three sources at review time and ask the reviewer (human or AI) to evaluate the diff against the assumptions. The reviewer returns pass/fail plus a short note explaining the mismatch. The CI fails the merge if the note is non-empty.
The tooling here is still maturing. Most teams I have seen roll their own pipelines rather than adopt a ready-made solution. The community is converging on treating the linked issue as a first-class artifact in the review, which is a healthy direction. The hard part is keeping the architecture documents honest. They drift. People forget to update them. A context-aware review system that trusts stale documents is worse than no system at all, because it produces false confidence.
Data contracts as infrastructure
The pattern that has saved me the most time is treating data contracts the same way I treat infrastructure code. The schema (the formal definition of what fields a piece of data must contain, and in what format) for model inputs, the imputation rules, the expected value distributions, all of it goes into version-controlled files alongside the application code. YAML (a human-readable data format often used for configuration) or JSON (JavaScript Object Notation, another structured data format) files describing “what the model expects” become part of the same review process as the code that calls the model.
When the contract changes, the review has to justify the change against the model’s assumptions. When the code violates the contract, the review catches it before merge. The contract is the lingua franca (a shared language both sides agree to use) between data science and engineering, and writing it down in version control turns implicit assumptions into explicit ones.
This sounds bureaucratic. It is, slightly. It is also the cheapest way I have found to prevent the silent failure mode from recurring.
Trade-offs
Context-aware review adds real overhead to the merge process. Every pull request now needs the linked issue to be precise, the architecture documents to be current, and the review tooling to ingest all three. For a small team working on a single ML system, this is manageable. For a large org with dozens of ML systems and rotating on-call reviewers, the maintenance cost of keeping the context sources honest is significant.
Data contracts are necessary and annoying. Writing them down is real work. Updating them when the model’s assumptions change is more real work. Skipping them produces silent failures that take weeks to debug. The cost-benefit is clear, but the costs come upfront and the benefits arrive months later.
The CI extension that ingests issue trackers and architecture docs adds a dependency on those systems being well-maintained. A stale architecture document produces false confidence. A vague issue ticket produces missed bugs. The pattern only works if the team treats the upstream sources as critical infrastructure.
Treating data assumptions as code is the most underrated shift. It is also the one that requires the most cultural buy-in. Engineering teams want to read code. Data science teams want to focus on models. Neither team naturally wants to maintain a YAML schema describing the contract between them, but that file is where the actual safety lives.
What I would tell past me
Three things, in order of how much pain they would have saved.
Treat data assumptions as code. The schema for what your model expects is part of your codebase, version-controlled and reviewed. Skipping this is the most expensive mistake I have seen teams make.
A green CI does not mean your model works. It means your code passes the checks. The model is doing something different, and you cannot see it from the code alone.
The bug is rarely in the code. When production predictions drift from training expectations, the cause is almost always a context mismatch the CI never looked at. Build the tooling that looks at it, before the production data drifts in a way that costs you weeks.
Bottom line: ML systems fail silently because their correctness lives in assumptions, not code. Containers close the infrastructure gap, not the context gap. The fix is to make assumptions reviewable: data contracts in version control, context-aware review tools that read the linked issue, and architecture documents the team actually maintains. None of it is glamorous. All of it is the cheapest way to avoid the 3 AM debugging sessions.