LLM API Error Handling: Retry, Fix The Request Or Call A Person, With Idempotent Jobs

LLM API error handling decides whether a busy morning costs a few seconds of retries or a duplicated client email. The rule of thumb: retrying only helps when the problem is on the other side and temporary. Everything else needs the request fixed or a person to look. Two releases in the last week make the categories easier to apply.

What changed in the SDK and the API?

The anthropic Python SDK 2.5.0, released on 4 November 2026, adds error details types and new error type values, and error classes for HTTP statuses and API error codes, with error_code, details and api_message on APIStatusError. It also raises a ValueError on Bedrock when the model name is empty, '.' or '..' instead of sending the request to a different path.

On 30 October 2026 Anthropic deprecated fast mode for Claude Opus 4.8, with removal on 30 November 2026. After removal, requests to claude-opus-4-8 with speed set to fast will return an error. The release notes point to Claude Opus 5.5 for continued fast mode. A request like that should be fixed, not retried.

What does good LLM API error handling look like?

How do you build the policy in Python?

"""Sort LLM API errors into retry, wait for a person, or fix the request, with idempotent retries.

anthropic Python SDK 2.5.0 (4 November 2026) adds error classes for HTTP statuses and API error codes,
with error_code, details and api_message on APIStatusError. Anthropic deprecated fast mode for Claude Opus
4.8 on 30 October 2026, with removal on 30 November, after which those requests return an error. This
policy uses HTTP status and your own code list, so it works whatever the SDK names. Standard library only.
"""
from __future__ import annotations

import random
import time
from dataclasses import dataclass

RETRY = {429, 500, 502, 503, 504, 529}
PERSON = {401, 403}  # keys, permissions, allowlists: a person must look
DONE: dict[str, str] = {}  # idempotency key -> result, so a retry never repeats a side effect


@dataclass
class ApiError(Exception):
    status: int
    error_code: str = ""


def classify(e: ApiError) -> str:
    if e.status in RETRY:
        return "retry"
    if e.status in PERSON:
        return "escalate"
    return "fix_request"  # 400, 404, 413 and removed features: retrying will not help


def run(job_id: str, attempt_fn, max_tries: int = 4) -> str:
    if job_id in DONE:
        return f"{job_id}: already done, not repeated"
    for n in range(1, max_tries + 1):
        try:
            DONE[job_id] = attempt_fn(n)
            return f"{job_id}: {DONE[job_id]} after {n} attempt(s)"
        except ApiError as e:
            action = classify(e)
            if action != "retry" or n == max_tries:
                return f"{job_id}: {action} (status {e.status} {e.error_code})"
            time.sleep(min(2 ** n, 30) * random.uniform(0.0, 0.01))  # jittered backoff, shortened for the demo
    return f"{job_id}: gave up"


if __name__ == "__main__":
    def flaky(n: int) -> str:
        if n < 3:
            raise ApiError(529, "overloaded")
        return "summary drafted"
    def removed(n: int) -> str:
        raise ApiError(400, "feature_removed")
    def key(n: int) -> str:
        raise ApiError(403, "ip_not_allowed")
    for job, fn in [("BRK-1043", flaky), ("BRK-1043", flaky), ("PACK-11", removed), ("RECS-05", key)]:
        print(run(job, fn))

An overloaded call succeeds on the third attempt. The same job ID is then refused as already done, so the summary is not drafted twice. A request using a removed feature goes back to be fixed, and a 403 from an IP allowlist goes to a person. The policy keys on HTTP status, so it keeps working if error code strings change.

What breaks in production?

• **Retries without idempotency.** A timeout after the side effect happened looks like a failure. Record success under the job key before anything is retried.
• **Retrying 400s.** A bad request stays bad. Retries only add cost and noise.
• **Silent fallbacks to another model.** Switching models on error changes behaviour. Treat it as a change that needs evaluation, not an automatic fallback.
• **Deprecations found in production.** Track provider deprecation dates, such as 30 November for Opus 4.8 fast mode, in your change calendar.

What are the alternatives to your own retry policy?

Our playbook on the Claude API 2026-10-21 migration covers SDK upgrades, and LLM evaluation as a release gate covers testing model changes. Our blog explains AI model testing for business readers.

Where should a team start?

List every error your agents have hit in the last month and sort each into retry, fix or escalate. Then add idempotency keys to every job with a side effect. Our AI developers in London build this into the agents behind our services, and a 14-day Proof Run includes it.