Agent Web Fetch Security: Limits Before Parsing, Redaction Without Backtracking, Clean Redirects

Agent web fetch security matters because the agent, not your team, picks many of the URLs it reads, and anyone who can influence those URLs can influence what the tool processes. Three advisories in a week show where the work goes wrong: reshaping content before limiting it, redacting with patterns that can be made slow, and forwarding credentials when a server redirects elsewhere.

What did the advisories disclose?

The Pydantic AI 2.66.0 release notes, published on 4 November 2026, describe GHSA-m5vj-8mcw-82qx (high): web_fetch_tool and the WebFetch capability's local fallback re-indented a JSON response on the event loop before applying the returned-content limit, so a deeply nested document within the download size limit could expand to hundreds of times its size. The notes say you are affected if an attacker can choose or influence the URLs the tool fetches, and that provider-native web fetching is not affected. Fixed in 2.66.0 and 1.107.10.

The same release lists GHSA-mfx5-pc32-32m8 (high): a private key pattern in the project's secret redaction backtracked exponentially on crafted text, so a few hundred characters that begin like a private key could occupy the event loop. Separately, CVE-2026-116689 in follow-redirects 1.16.0 and 1.16.1, published on 30 October 2026, meant an array of sensitive headers created in another JavaScript realm was replaced with an empty list, so a custom credential header could be forwarded to an attacker-controlled cross-origin redirect. Version 1.16.2 fixes it.

What does agent web fetch security look like as a design?

How do you build the three guard rails?

"""Guard rails for an agent's web fetch tool: limits before transforms, linear-time redaction, and no
credential headers across origins.

Pydantic AI 2.66.0 and 1.107.10 (4 November 2026) fixed a web fetch that re-indented JSON before applying
the content limit (GHSA-m5vj-8mcw-82qx) and a redaction pattern that backtracked exponentially
(GHSA-mfx5-pc32-32m8). follow-redirects 1.16.2 (30 October 2026) fixed credential headers forwarded to a
cross-origin redirect (CVE-2026-116689). Python 3.14 standard library only.
"""
from __future__ import annotations

import json
from urllib.parse import urlsplit

MAX_BYTES, MAX_DEPTH = 200_000, 20
SENSITIVE = {"authorization", "cookie", "x-api-key", "proxy-authorization"}


def parse_limited(raw: bytes) -> object:
    if len(raw) > MAX_BYTES:
        raise ValueError("response over the byte limit, refused before parsing")
    depth = 0
    for ch in raw[:MAX_BYTES].decode("utf-8", "replace"):
        depth += ch in "[{"
        depth -= ch in "]}"
        if depth > MAX_DEPTH:
            raise ValueError(f"nesting deeper than {MAX_DEPTH}, refused before parsing")
    return json.loads(raw)  # never re-indent or pretty-print before these checks


def redact_keys(text: str) -> str:
    """Linear scan for PEM blocks, no regex backtracking."""
    start, out = 0, []
    while (i := text.find("-----BEGIN ", start)) != -1:
        j = text.find("-----END ", i)
        k = text.find("-----", j + 9) if j != -1 else -1
        out.append(text[start:i] + "[redacted key]")
        start = k + 5 if k != -1 else len(text)
    return "".join(out) + text[start:]


def redirect_headers(headers: dict[str, str], from_url: str, to_url: str) -> dict[str, str]:
    a, b = urlsplit(from_url), urlsplit(to_url)
    same = (a.scheme, a.hostname, a.port) == (b.scheme, b.hostname, b.port)
    return headers if same else {k: v for k, v in headers.items() if k.lower() not in SENSITIVE}


if __name__ == "__main__":
    try:
        parse_limited(b"[" * 50 + b"]" * 50)
    except ValueError as e:
        print("deep json:", e)
    print("redacted:", redact_keys("key: -----BEGIN PRIVATE KEY-----abc-----END PRIVATE KEY----- done"))
    print("pathological input:", len(redact_keys("-----BEGIN " * 5000)), "chars, returned at once")
    h = {"Authorization": "Bearer t", "X-Api-Key": "k", "Accept": "application/json"}
    print("cross-origin:", redirect_headers(h, "https://api.example.com/a", "https://evil.example.net/b"))

A fifty-level nested array is refused before parsing because nesting passes the limit of twenty. A PEM block is replaced by a placeholder with a linear scan, and five thousand repeated private key openings return at once, because nothing backtracks. On a redirect from one host to another, the Authorization and X-Api-Key headers are dropped and only Accept is kept.

What breaks in production?

• **Pretty-printing for the model.** Formatting a response to make it readable can multiply its size. Limit first, format second, and cap again after formatting.
• **Redaction patterns copied from the internet.** Patterns with nested repetition can be made slow. Test every pattern against long crafted input.
• **Redirect handling in the HTTP library.** Know which headers your client forwards, and upgrade follow-redirects to 1.16.2 or later if you use it.
• **Fetched text as instructions.** Mark everything fetched as data. Our prompt injection defence playbook covers why.

What are the alternatives to a local fetch tool?

Our playbook on keeping tokens out of CI logs covers redaction elsewhere, and our blog gives the AI agent security questions to ask vendors.

Where should a team start?

Upgrade Pydantic AI and follow-redirects if you use them, then test your own fetch tool with a deeply nested response, a long crafted key prefix and a cross-origin redirect. As a Workflow Automation Agency we build these tests into every agent, as described on our services page, and a 14-day Proof Run includes them.