LLM Conversation State: What Server-Side Message Threads Change For Long Agent Loops

LLM conversation state is the history a model needs to answer the next turn. In a client-held design you send all of it every time, so request size grows with every tool call. In a server-held design the provider keeps it and you send only what is new. Anthropic made the second option available for the Messages API on 2 November 2026, and it changes cost, latency and record-keeping in different ways.

What did Anthropic release on 2 November?

The release notes for 2 November 2026 say long conversations and agent loops on the Messages API can now send smaller requests and get a lower time to first token with message threads, in beta with the message-threads-2026-08-12 beta header. The API keeps the conversation on the server, so each later request sends only the new messages.

A thread expires five minutes after its last request and is deleted no more than one hour after its last use. Message threads are available on the Claude API, Claude Platform on AWS, Claude Models on Google Cloud (Anthropic managed) and Claude in Microsoft Foundry. The Python SDK added support for message threads in version 2.2.0 on 23 October 2026.

How does LLM conversation state work in an agent loop?

Each turn of an agent loop adds a tool result and a model reply. With client-held state, turn forty carries the system prompt, the tool definitions and all forty turns. With a server thread, turn forty carries only turn forty. The model still sees the whole conversation. What changes is how much travels with each request and how much the provider must read before it starts answering.

How much smaller do requests get?

"""What server-side conversation state changes for a long agent loop: request size per turn.

Anthropic's message threads (beta, 2 November 2026): the API keeps the conversation on the server, so
each later request sends only the new messages. A thread expires 5 minutes after its last request and is
deleted no more than 1 hour after its last use. This simulation makes no API calls. It compares bytes sent
per turn and shows the recovery path when a thread has expired. Python 3.14 standard library only.
"""
from __future__ import annotations

TURN_BYTES = 1800   # one tool result plus the model's reply, in bytes, sample
SYSTEM_BYTES = 6000  # system prompt and tool definitions, sample
EXPIRY_S = 5 * 60


def client_side(turns: int) -> list[int]:
    return [SYSTEM_BYTES + TURN_BYTES * t for t in range(1, turns + 1)]


def server_thread(turns: int, gaps_s: list[int]) -> tuple[list[int], list[int]]:
    sent, rebuilt = [], []
    for t in range(1, turns + 1):
        if t == 1 or gaps_s[t - 1] > EXPIRY_S:
            sent.append(SYSTEM_BYTES + TURN_BYTES * t)  # new thread: resend the full history from your own log
            if t > 1:
                rebuilt.append(t)
        else:
            sent.append(TURN_BYTES)
    return sent, rebuilt


if __name__ == "__main__":
    turns = 40
    gaps = [20] * turns
    gaps[25] = 9 * 60  # a person took nine minutes to approve a step
    a = client_side(turns)
    b, rebuilt = server_thread(turns, gaps)
    print(f"turn 40 request: client-side {a[-1]:,} bytes, thread {b[-1]:,} bytes")
    print(f"total sent: client-side {sum(a):,} bytes, thread {sum(b):,} bytes ({sum(b) / sum(a):.0%})")
    print("thread rebuilt from local log at turn:", rebuilt)

With sample sizes of 6,000 bytes of system prompt and tools and 1,800 bytes per turn, turn forty sends 78,000 bytes from the client and 1,800 bytes through a thread. Across forty turns the thread sends 8% of the bytes. A nine-minute approval pause at turn 26 expires the thread, so that turn resends the history from the firm's own log and the thread starts again.

What does server-side state mean for audit and data retention?

• **The provider holds the conversation for a while.** Up to an hour after last use, according to the release notes. Check that against your data protection assessment and contracts.
• **Your record is still yours to keep.** The thread is not your audit trail. Keep every turn, tool call and approval in your own store.
• **Expiry is normal.** Approval pauses, overnight jobs and retries will outlast five minutes. Design the rebuild path from day one.

What breaks in production?

• **Assuming the thread exists.** Treat every request as possibly the first after expiry, and handle the error by rebuilding.
• **Two writers on one thread.** Parallel tool calls that both append can interleave. Serialise appends per thread.
• **Compaction that rewrites history.** If you summarise old turns to save space, keep the originals in your log. Inspect 0.3.287, released on 31 October 2026, added a way for a runtime to hold and rewrite a react() conversation at turn boundaries and gave compaction handlers reset(), which is the same idea in an evaluation framework.
• **Tracing personal data twice.** If the thread holds client data and your logs do too, both need retention rules.

What are the alternatives to server-side threads?

Our playbook on LLM evaluation as a release gate covers testing after a change like this, and the Claude API 2026-10-21 migration covers the SDK side. Our blog on AI cost reporting explains why per-workflow numbers matter to the business.

Where should a team start?

Measure request size and time to first token on one long agent loop today, then try a thread in a test environment with the rebuild path in place. Our AI Automation London team does this measurement as part of a 14-day Proof Run, on workflows such as the daily mismatch report.