Deployment & Production · BraivIQ AI Engineering Playbook
LLM Model Deprecation: Pin, Test And Approve Every Model Change Before Providers Reroute You
LLM model deprecation is now a scheduled production event. Google's Gemini API changelog of 2 November 2026 says requests to gemini-2.5-pro will automatically route to gemini-3.8-flash from 1 December 2026, with breaking changes to function calling and thinking settings. Anthropic removes fast mode for Claude Opus 4.8 on 30 November 2026, after which those requests return an error. This guide shows how to pin model IDs per workflow, catch a silent swap, gate every change on your own evaluation and keep a person's sign-off in the audit log, with two tested Python samples.
Published · Updated · 9 min read · By BraivIQ Engineering
Key takeaways
- Google's Gemini API changelog (2 November 2026) says requests to gemini-2.5-pro will route automatically to gemini-3.8-flash from 1 December 2026, billed at Gemini 3.8 Flash rates.
- The same entry lists breaking changes: function calling and thought_signature validation, thinking_level replacing thinking_budget, and removed sampling parameters and prefilled model turns.
- Anthropic deprecated fast mode for Claude Opus 4.8 on 30 October 2026, with removal on 30 November 2026, after which those requests return an error.
- Providers retire models in two ways, a silent reroute or a hard error. Plan for both with a pinned registry, a served-model check and a dated warning.
- No model change should reach production without your own evaluation and a named approver, recorded in the audit log.
LLM model deprecation is when a provider retires a model ID and either reroutes requests to a newer model or starts returning errors. For an agent that prepares reconciliations or drafts client emails, both outcomes change behaviour that someone has signed off. The fix is engineering, not vigilance: a registry of pinned models, a check on every response, an evaluation gate and an approval that is logged.
How does LLM model deprecation work at each provider?
Google's Gemini API changelog of 2 November 2026 deprecated gemini-2.5-pro and removed it from the model picker in Google AI Studio. From 1 December 2026, all API requests to gemini-2.5-pro will automatically route to gemini-3.8-flash and be billed at Gemini 3.8 Flash rates. The same entry deprecates gemini-2.5-flash from the same date.
The entry is clear that this is not a drop-in swap. It lists breaking API changes in the move to Gemini 3.8 Flash: function calling and thought_signature validation, thinking_level replacing thinking_budget, and removed sampling parameters and prefilled model turns. An agent that sets a temperature or a thinking budget today may behave differently, or fail, after the reroute.
On 6 November 2026 Google rerouted gemini-embedding-2-preview to gemini-embedding-2 with immediate effect. The changelog says the two share the same API schema, pricing and embedding space, so no re-indexing is needed. That is the gentle case, and it still means the model behind your search index changed on a date you did not choose.
Anthropic took the other route. Its Claude API release notes of 30 October 2026 deprecate fast mode for Claude Opus 4.8, with removal on 30 November 2026. After that date, requests to claude-opus-4-8 with speed set to fast return an error. The notes point to Claude Opus 5.5 for teams that want to keep fast mode.
| Date | Provider | Change | How it lands |
|---|---|---|---|
| 30 Oct 2026 | Anthropic | Fast mode for claude-opus-4-8 deprecated, removed 30 Nov | Hard error after removal |
| 2 Nov 2026 | gemini-2.5-pro and gemini-2.5-flash deprecated | Automatic reroute from 1 Dec at a new billing rate | |
| 6 Nov 2026 | gemini-embedding-2-preview deprecated | Rerouted to gemini-embedding-2 the same day | |
| 9 Nov 2026 | Anthropic | claude-fable-5-2 launched at $8 and $40 per million tokens | New option, nothing changes until you switch |
Why is a silent reroute riskier than an error?
An error is loud. Your retry logic gives up, an alert fires and someone looks. A reroute is quiet. Responses keep arriving, often well formed, from a model your evaluation never saw. In a reconciliation agent, the first sign may be a different explanation style in the exceptions report, or a classification that drifts on a category you rarely see.
Billing changes with it. The Gemini entry says rerouted requests are billed at Gemini 3.8 Flash rates, so the cost line moves on the same day as behaviour.
For a regulated firm, the deeper issue is evidence. If an auditor asks which model prepared a figure that a person approved in December, "the provider changed it" is a poor answer. Our AI model testing article covers the business side.
How do you pin models and catch a silent swap?
Keep one registry entry per workflow, not per application. Each entry holds the pinned model ID, any announced reroute date and the replacement the provider named. A daily job reads the registry and warns 30 days ahead, which for gemini-2.5-pro means a warning has been live since 1 November.
Then check every response. Most provider responses report which model served the request. Log that value next to the pinned ID and hold the output for a person when they differ. The sample below does both, and it refuses a pin change unless a second person approves it and an evaluation ID is attached.
model_registry.py"""Pin model IDs per workflow, warn before provider reroutes and catch silent swaps.
Python 3.14, standard library only. Model IDs and dates are from the Gemini API
changelog (2 and 6 November 2026) and the Claude API release notes (9 November 2026)."""
import datetime as dt
import hashlib
import json
REGISTRY = {
"recs-summary": {"model": "gemini-2.5-pro", "reroute": "2026-12-01", "to": "gemini-3.8-flash"},
"doc-search": {"model": "gemini-embedding-2-preview", "reroute": "2026-11-06", "to": "gemini-embedding-2"},
"breaks-triage": {"model": "claude-fable-5-2", "reroute": None, "to": None},
}
AUDIT: list[dict] = []
def audit(event: str, **data) -> None:
prev = AUDIT[-1]["hash"] if AUDIT else "0" * 16
rec = {"at": dt.datetime.now(dt.UTC).isoformat(timespec="seconds"), "event": event, **data}
rec["hash"] = hashlib.sha256((prev + json.dumps(rec, sort_keys=True)).encode()).hexdigest()[:16]
AUDIT.append(rec)
def deprecation_report(today: dt.date, warn_days: int = 30) -> list[str]:
lines = []
for wf, r in REGISTRY.items():
if r["reroute"]:
days = (dt.date.fromisoformat(r["reroute"]) - today).days
if days <= warn_days:
state = "ALREADY REROUTED" if days <= 0 else f"{days} days left"
lines.append(f"{wf}: {r['model']} -> {r['to']} ({state})")
return lines
def check_served(workflow: str, served_model: str) -> bool:
"""Compare the model a response reports with the pinned model."""
pinned = REGISTRY[workflow]["model"]
if served_model != pinned:
audit("served_model_mismatch", workflow=workflow, pinned=pinned, served=served_model)
return False # hold the output for a person, do not apply it
return True
def change_pin(workflow: str, new_model: str, requested_by: str, approved_by: str, eval_id: str) -> None:
if approved_by == requested_by:
raise PermissionError("a second person must approve a model change")
old = REGISTRY[workflow]["model"]
REGISTRY[workflow].update(model=new_model, reroute=None, to=None)
audit("pin_changed", workflow=workflow, old=old, new=new_model,
requested_by=requested_by, approved_by=approved_by, eval_id=eval_id)
if __name__ == "__main__":
for line in deprecation_report(dt.date(2026, 11, 11)):
print("WARN", line)
print("served ok:", check_served("doc-search", "gemini-embedding-2"))
change_pin("doc-search", "gemini-embedding-2", "dev.ops", "head.of.data", "EVAL-2026-11-11-01")
print("served ok after pin change:", check_served("doc-search", "gemini-embedding-2"))
for rec in AUDIT:
print(json.dumps(rec))Python 3.14, standard library only. Model IDs and dates from the Gemini API changelog (2 and 6 November 2026) and the Claude API release notes (9 November 2026). Run once, it warns that gemini-2.5-pro has 20 days left, holds a mismatched response and logs the approved pin change.
The mismatch check returns False rather than raising, so other items keep moving while one output waits for review. Each audit record chains to the previous hash, so a quiet edit shows up.
How should an evaluation gate decide on a new model?
The gate compares the candidate model with the current one on a golden set your own team labelled. For a breaks agent that might be fifty real breaks with the agreed root cause. The gate blocks if the score falls below a floor, if it drops against the current model, or if p95 latency goes over the limit the operations team agreed.
Report misses by label, not only the headline score. A candidate can match the baseline overall and still get every settlement-date break wrong, which is the category that costs money. Our playbook on LLM evaluation release gates goes deeper on building the set.
eval_gate.py"""Promote a model change only when it passes your own golden set.
Python 3.14, standard library only. The answers below are made-up sample
outputs to show the gate, not results from any real model."""
import statistics
GOLDEN = { # break id -> expected root cause, labelled by the operations team
"BRK-001": "price", "BRK-002": "quantity", "BRK-003": "settlement-date",
"BRK-004": "price", "BRK-005": "counterparty", "BRK-006": "quantity",
"BRK-007": "price", "BRK-008": "settlement-date", "BRK-009": "counterparty",
"BRK-010": "quantity",
}
BASELINE = dict(GOLDEN) | {"BRK-009": "price"}
CANDIDATE = dict(GOLDEN) | {"BRK-003": "price", "BRK-008": "quantity"}
CANDIDATE_LATENCY_MS = [820, 790, 1010, 760, 2400, 880, 905, 840, 930, 870]
def score(answers: dict[str, str]) -> float:
return sum(answers.get(k) == v for k, v in GOLDEN.items()) / len(GOLDEN)
def per_label_misses(answers: dict[str, str]) -> dict[str, int]:
misses: dict[str, int] = {}
for k, v in GOLDEN.items():
if answers.get(k) != v:
misses[v] = misses.get(v, 0) + 1
return misses
def gate(base: float, cand: float, latency: list[int], *, min_score=0.9,
max_drop=0.0, max_p95_ms=2000) -> list[str]:
reasons = []
if cand < min_score:
reasons.append(f"score {cand:.2f} is below the floor of {min_score:.2f}")
if cand < base - max_drop:
reasons.append(f"score fell from {base:.2f} to {cand:.2f}")
p95 = statistics.quantiles(latency, n=20, method="inclusive")[18]
if p95 > max_p95_ms:
reasons.append(f"p95 latency {p95:.0f} ms is above {max_p95_ms} ms")
return reasons
if __name__ == "__main__":
base, cand = score(BASELINE), score(CANDIDATE)
print(f"baseline {base:.2f} candidate {cand:.2f}")
print("candidate misses by label:", per_label_misses(CANDIDATE))
reasons = gate(base, cand, CANDIDATE_LATENCY_MS)
if reasons:
print("BLOCKED:", "; ".join(reasons).replace(";", ","))
else:
print("PASSED: send to the model owner for approval")Python 3.14, standard library only. The answers are made-up samples to show the gate, not results from any real model. Run once, it blocks the candidate because its score of 0.80 is below the 0.90 floor and below the baseline, with both misses on settlement-date breaks.
A pass does not promote anything by itself. It sends the result to the model owner, who approves or rejects, and the approval references the evaluation ID. That is the human approval point. The agent prepares, a person approves.
What breaks in production?
- **Hidden parameters.** Code that sets temperature or a thinking budget meets a model that ignores or rejects them. The Gemini 3.8 Flash entry lists removed sampling parameters and thinking_level replacing thinking_budget.
- **Prefilled turns.** Agents that prefill the start of the model's answer to force a format lose that control when prefilled model turns are removed.
- **Tool calls.** Changes to function calling and thought_signature validation can turn a working tool loop into a stream of validation errors.
- **Aliases.** A floating alias moves without a changelog entry in your repository. Pin exact IDs in production and keep aliases for experiments.
- **Retries hiding the problem.** A retry loop that treats a removed mode as a temporary error burns time and money until someone notices. Classify removal errors as permanent.
What are the trade-offs and alternatives?
Pinning exact IDs gives control but means providers will tell you to move, sometimes with weeks of notice. Floating aliases keep you current at the cost of unannounced change. For agents that prepare work a person signs off, pin in production and run the alias in a shadow environment so you see change coming.
A multi-provider router reduces dependence on one vendor, but every extra model adds evaluation work. Self-hosting removes provider retirement and replaces it with your own upgrade cycle, which our self-hosted LLM security worked example covers. Most firms with 10 to 250 staff are better served by one primary provider, pinned, with a tested fallback.
How do approvals and audit fit a model change?
Treat a model change like any other production change. It needs a ticket, an evaluation result, a named approver who is not the requester and a rollback plan. The audit log should let you answer three questions for any output: which model produced it, which pin was in force, and who approved that pin.
Keep the registry in version control and the audit log somewhere the agent cannot edit. Our agent registry and decision records playbook shows a fuller schema. Our AI Agency London team uses the same pattern in every agent build.
Where should a team start?
This week, check every production model ID against both changelogs. Anything on gemini-2.5-pro or gemini-2.5-flash has until 1 December 2026, and anything using fast mode on claude-opus-4-8 has until 30 November 2026. Then add the served-model check, which is a few lines in your client wrapper.
If you want this built around a live workflow such as the daily mismatch report, a 14-day Proof Run is the fastest route. This guide is engineering guidance, not legal or regulatory advice.
Frequently asked questions
What happens to gemini-2.5-pro on 1 December 2026?
According to the Gemini API changelog of 2 November 2026, all API requests to gemini-2.5-pro will automatically route to gemini-3.8-flash and be billed at Gemini 3.8 Flash rates. Google lists breaking changes, so test before that date.
When is fast mode removed for Claude Opus 4.8?
The Claude API release notes of 30 October 2026 set removal for 30 November 2026. After that, requests to claude-opus-4-8 with speed set to fast return an error. Anthropic points to Claude Opus 5.5 for fast mode.
How can I tell if a provider has swapped my model?
Log the model name each response reports and compare it with the pinned ID for that workflow. Hold any output from an unexpected model for a person to review.
References
- Google, "Gemini API changelog", 2 to 11 November 2026. https://ai.google.dev/gemini-api/docs/changelog
- Anthropic, "Claude API release notes", 28 October to 9 November 2026. https://docs.claude.com/en/release-notes/api
- Anthropic on GitHub, "anthropic-sdk-python v2.7.0 release notes", 9 November 2026. https://github.com/anthropics/anthropic-sdk-python/releases/tag/v2.7.0