Your Model Vendor Gave You Three Days

Earlier this month a model provider announced that a set of models would be decommissioned. The notice period was three days.

Two of our systems depended on those models. Neither was doing anything exotic — one generates educational content on a daily schedule, the other runs a retrieval pipeline over a large video corpus. Both had a model identifier written into their configuration, the way essentially every AI system built in the last two years does.

We migrated both inside the window. It was not dramatic. But it is the clearest illustration I have of a risk that most AI strategy conversations skip entirely, so it is worth setting out plainly.

The thing that is actually different about this dependency

Enterprises are practised at managing software vendor risk. Contracts, notice periods, exit clauses, support windows. The muscle exists.

Model dependencies do not behave like software dependencies, in three specific ways.

The deprecation clock is set by the provider, not by you. A library you depend on can be pinned to a version and left alone for years. A hosted model cannot. When the provider retires it, it is gone from your side too, regardless of what your roadmap said.

The replacement is not equivalent. When a library deprecates a function, the replacement usually does the same thing. When a model is retired, the successor has different behaviour — different verbosity, different instruction-following, different failure modes under the same prompt. Your prompts were tuned, implicitly or explicitly, against the old one. Migration is not a find-and-replace; it is a re-qualification.

The failure is often silent. This is the part that gets people. When the model call fails, a well-intentioned error handler catches it and the pipeline continues with a placeholder or an empty result. One of our two systems did exactly this: the model chain died, the job caught the error, wrote a placeholder, and exited successfully. Everything downstream treated the placeholder as content. We fixed it to fail loudly, which is the correct behaviour for a step whose output nothing else can validate.

Three days’ notice on a dependency you cannot pin, whose replacement is not equivalent, in a system that may not tell you when it breaks. That is the actual risk profile.

What this does to the rent-versus-own argument

The usual framing is cost. Hosted API calls are an operating expense that scales with usage; self-hosted open-weight models are a capital and operational commitment that scales with infrastructure. Somewhere there is a crossover volume, and the analysis is about finding it.

That framing is incomplete, because it prices only the inference. It does not price the control.

What you are actually buying when you run your own weights is not cheaper tokens. It is the right to decide when you migrate. A model on your own infrastructure does not get retired on someone else’s schedule. You can run it, unchanged, for as long as you are willing to maintain the hardware — which means model changes become a thing you plan, rather than a thing that arrives.

That right has a price, and for most organisations it is not worth paying for most workloads. Hosted frontier models are better, cheaper to start with, and maintained by someone else. The mistake is not choosing hosted. The mistake is choosing hosted without recognising that you have accepted a dependency whose lifecycle you do not control, and then building as if you had.

The practical answer for most enterprises is neither pure position but a deliberate split:

• Hosted frontier models for the work where capability is the differentiator and interruption is survivable.
• Owned or pinned weights for the narrow set of workloads where a three-day notice would constitute an incident — anything embedded in a regulated process, a contractual SLA, or a system whose behaviour has been formally validated.

The useful question in a board paper is not “should we self-host?” It is: which of our AI-dependent processes could not survive a three-day deprecation notice? That list is usually short, and it is the only part of the estate that needs the expensive answer.

Four controls that make a deprecation notice boring

None of these are difficult. They are unglamorous, which is why they are usually absent.

1. Know where your model identifiers are. You cannot respond to a deprecation notice until you know what it affects. Every model identifier should be in configuration, discoverable by a single search across your estate, never buried in application code. If answering “which of our systems use this model?” takes more than a few minutes, that is your first fix — and it is worth doing before you need it, because you will need it under time pressure.

2. Verify model identifiers against live documentation, not memory. We now check identifiers against the provider’s current documentation as part of the change, rather than trusting what we believe them to be. Model naming is inconsistent across providers and changes more often than anyone expects; a plausible-looking identifier that does not exist fails at runtime, in production, at the moment of highest cost. This applies with particular force when the code was written with AI assistance — a model will confidently produce a model identifier that was correct at some point during its training and is not correct now.

3. Fail loudly on provider errors. Any step whose output cannot be validated by the next step must fail hard when the provider fails. The alternative — a placeholder that propagates — converts a visible outage into an invisible data-quality problem, which is strictly worse. It is discovered later, by someone else, in worse circumstances.

4. Keep a behavioural test set. A small set of representative inputs with known-acceptable outputs turns model migration from a judgement call into a measurement. Twenty examples is enough to catch the large regressions. Without it, “the new model seems fine” is an opinion formed by looking at three outputs.

Add a fifth if you are procuring: ask for the deprecation policy in writing. Notice period, minimum support window, and whether pinned versions are honoured. Providers vary enormously, and the difference between three days and twelve months is a genuine differentiator that rarely appears on a comparison page.

The wider point

Every organisation adopting AI is accumulating dependencies on artifacts that behave less like software and more like infrastructure it does not own. The industry conversation is still overwhelmingly about capability — which model is best, what it can do now that it could not last quarter.

The operational conversation is different and less exciting: what happens when the thing you built on is withdrawn, and how long do you have.

Our answer this month was three days, and it was fine, because the systems affected were ones where a bad day is a bad day rather than an incident. That was luck as much as design. The work now is making sure the parts of the estate where it would not be fine are on a footing where the next notice is an email somebody reads calmly.

The measure of maturity here is not which models you use. It is how long it takes you to answer the question “what breaks if this is withdrawn on Friday?”

Frequently asked questions

How much notice do model providers typically give before deprecation? It varies widely, from a few days to a year or more. Notice period, minimum support window, and whether pinned versions are honoured should be asked about explicitly during procurement rather than assumed.

Does self-hosting open-weight models eliminate this risk? It removes the provider’s ability to withdraw the model from under you, which is the specific risk discussed here. It replaces it with the cost and responsibility of running and securing the infrastructure yourself.

How do we find every place a model is referenced in our systems? Keep model identifiers in configuration rather than application code, and maintain an inventory. If you cannot answer the question with a single search across your repositories, that is the first thing to fix.

Why does a failed model call often go unnoticed? Because error handling frequently catches the failure and substitutes a placeholder or empty result so the pipeline can continue. The job then reports success. Steps whose output cannot be validated downstream should fail loudly instead.

What is the minimum viable test for a model migration? A small set of representative inputs with known-acceptable outputs — twenty is usually enough to catch significant regressions — run against both the old and new model before you switch.