Event-Driven Agents: Webhooks, Queues And Backpressure - Triggering Automations Reliably In Code

Events, not prompts - Most production automations are triggered by webhooks, queues, files and record changes - not by a person  ·  At-least-once - Every webhook and queue provider delivers at least once - duplicates are a certainty, so idempotency is mandatory  ·  Backpressure - A burst of events must not fan out into a burst of expensive agent runs. The queue absorbs, the worker paces  ·  Event → action - Every agent run carries the identifier of the event that triggered it, so surprising actions are traceable

The agent demos that circulate online all start the same way: a person types a request and the agent gets to work. The automations that actually run businesses almost never start that way. They start with an event. A payment provider sends a webhook that an invoice was paid, a message arrives on a queue that an order was placed, a file lands in a storage bucket, a record changes in the CRM, a customer submits a form, a scheduled job fires. The agent that then reconciles the payment, enriches the order, processes the document or follows up the lead is the interesting part, and the layer between the event and the agent is the part nobody talks about and everybody gets wrong. That layer decides whether the automation is reliable. Does a webhook that the provider sends twice - and every provider will - run the agent twice and double-process the payment? Does a burst of ten thousand events during a sale fan out into ten thousand simultaneous, expensive agent runs that exhaust your model budget and rate limits in a minute? When an event fails to process, is it lost forever, or parked and replayed? And when an agent does something surprising, can anyone trace it back to the exact event that caused it? As an AI Agency Developer London that builds automation platforms rather than one-off automations, we spend as much engineering on this layer as on the agents themselves, and this playbook is how to build it so that an agent becomes an automation you can run unattended.

Ingestion: Verify, Record, Acknowledge - And Nothing Else

A webhook endpoint has one job: get the event safely onto the queue and tell the sender it arrived. Three disciplines make it safe. Verification first: every serious provider signs its webhooks with a shared secret or a key, and the endpoint must verify the signature over the raw body before trusting a byte, because an unverified webhook endpoint is a public API that lets anyone trigger your agents with fabricated events. Replay protection next: signatures usually include a timestamp, and the endpoint should reject events older than a short window and remember recently seen event identifiers, so a captured request cannot be replayed to re-trigger an action. Then the fast path: the verified event is written to durable storage - the queue, or a database table from which a queue is fed - and the endpoint returns success immediately, typically within a few hundred milliseconds, because providers retry on slow or failed responses and a handler that runs the agent inline will time out, be retried, and run the agent again. Everything the agent needs to do happens later, from the queue. The same shape applies to other sources: a file-arrival notification, a change-data-capture record or a queue message from another system is verified for origin, recorded with its identifier, and acknowledged, and the work follows asynchronously. The outbox pattern closes the last gap: when your own system both changes state and must emit an event about it - an agent that updates a record and must trigger a follow-on automation - the event is written to an outbox table in the same transaction as the state change and published from there, so that a crash between the two can never produce a change without its event or an event without its change.

• Verify the signature over the raw body before trusting anything - an unverified endpoint lets anyone trigger your agents.
• Reject replays - enforce a timestamp window and remember recent event identifiers.
• Record durably, then acknowledge fast - never run the agent inside the webhook handler.
• Use the outbox pattern - when your system changes state and emits an event, commit both in one transaction and publish from the outbox.
• Carry the event identifier everywhere - it is the key for deduplication, ordering and tracing.

Processing: Idempotency, Ordering, And Backpressure

Once events are on the queue, the worker that runs the agent faces three problems that every event system has and that agents make more expensive. The first is duplicates. Every webhook and queue provider delivers at least once - network retries, provider retries, redeliveries after a worker crash - so the same event will arrive more than once, and an agent that charges a card, sends an email or creates a record will do it twice unless processing is idempotent. The discipline is an idempotency key - the event identifier, or a hash of its meaningful content - checked against a store of processed events before the agent runs and recorded atomically with the outcome when it completes, so a duplicate is recognised and skipped, and the agent's own side-effecting actions carry their own idempotency keys for the same reason. The second is ordering. Events about the same entity often must be processed in order - an order placed, then amended, then cancelled - while events about different entities can be processed in parallel. The pattern is to partition the queue by entity identifier so each entity's events are handled sequentially by one worker while the fleet processes many entities at once. Global ordering across all events is a trap: it serialises everything, throttles throughput to one worker, and is almost never what the business actually needs. The third problem is the one agents introduce: cost and capacity. A classic worker processing a burst of events just runs hot for a while. An agent worker running a burst of events runs a thousand model calls, hits provider rate limits, and burns a day's budget in minutes. Backpressure is the answer: the queue absorbs the burst, workers consume at a controlled concurrency, rate limits on model calls are enforced at the worker or the gateway, and a budget guard pauses consumption when spend exceeds a threshold rather than letting the fleet race to the limit. The queue is not just decoupling. It is the shock absorber that turns a spike in events into a steady, affordable stream of agent runs.

Failure, Replay, And Tracing

Events will fail to process - a downstream API is down, the agent hits an unexpected input, a tool errors - and the difference between a toy and a platform is what happens next. The worker retries transient failures with backoff, a bounded number of times, and if the event still fails it moves to a dead-letter queue rather than being dropped or retried forever. The dead-letter queue is a first-class operational surface: someone is alerted, the failed events can be inspected with their full context and error, and once the cause is fixed they can be replayed - re-queued for processing, in order, with their original identifiers so idempotency still holds. Replay is also the recovery mechanism for larger failures: if a bug caused a day's events to be processed wrongly, the durable record of every event lets you fix the bug and replay the day, which is only possible because ingestion recorded everything before processing touched it. Tracing ties the whole layer together: the event identifier is attached to the queue message, to the worker's log, to the agent's run and every tool call it makes, and to the records the agent creates or changes, so that a surprising outcome - an email a customer should not have received, a record changed unexpectedly - can be traced in seconds to the event that triggered it, the agent run that handled it, and the decisions inside that run. Testing the layer uses the same durable record: a corpus of recorded real events, including duplicates, out-of-order deliveries and malformed ones, replayed through the full pipeline in continuous integration, with assertions on exactly which agent runs occurred, that duplicates were skipped, that per-entity order was preserved, and that failures landed in the dead-letter queue. An event layer tested that way is one you can trust to trigger agents while nobody is watching.

The Bottom Line

The automations that run a business are triggered by events, not prompts, and the layer between the event and the agent is what makes them reliable. Its shape is consistent: ingestion that verifies signatures over the raw body, rejects replays, records the event durably and acknowledges within milliseconds - never running the agent inline, a durable queue that decouples receiving from processing, the outbox pattern when your own state changes must emit events, workers that deduplicate with idempotency keys because every provider delivers at least once, process each entity's events in order by partitioning on its identifier while parallelising across entities, and consume at a controlled rate with model rate limits and a budget guard so a burst of events becomes a steady stream of agent runs rather than a cost incident, bounded retries and a dead-letter queue with replay for the events that fail, and the event identifier carried through queue, worker, agent run, tool calls and resulting records so any action traces back to its cause. Tested by replaying recorded real events - duplicates, disorder, malformation included - through the whole pipeline, it is the plumbing that turns an agent into an automation you can run unattended, and building that plumbing as carefully as the agents it triggers is exactly the workflow-automation engineering we do.

References & Further Reading

• Stripe - webhooks best practices: signature verification, replay protection and fast acknowledgement: https://docs.stripe.com/webhooks
• Microservices.io - the transactional outbox pattern: https://microservices.io/patterns/data/transactional-outbox.html
• AWS - Amazon SQS dead-letter queues and message deduplication: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
• LangGraph - durable execution for long-running agent workflows (the processing layer events hand to): https://langchain-ai.github.io/langgraph/concepts/durable_execution/
• AI Agents Directory - AI agents news brief, October 1 2026 (agents moving into always-on, event-triggered operation): https://aiagentsdirectory.com/news/ai-agents-news-brief-october-1-2026