How to run a long job without hitting request timeouts
An operation takes minutes. Doing it inside the request times out at the proxy, and the client has no way to find out whether it eventually finished.
Accept the work, return 202 with a status URL, and do the job in a worker. The client polls the status URL or receives a webhook when it completes. Holding a request open for minutes fails at the first proxy that disagrees about timeouts, and leaves the client unable to distinguish slow from dead.
Return a handle, not a result
Respond immediately with an identifier for the job and a URL where its state can be read. The client now has something durable: it can poll, close the tab and come back, or hand the identifier to another process. A held-open request offers none of that.
Make submission idempotent
A client that times out while submitting will retry, and without an idempotency key that retry starts a second copy of an expensive job. This is the most common way these systems duplicate work — not in the worker, but in the accepting endpoint.
Visible progress and honest failure
A status endpoint that only reports queued, running and done is hard to trust for a job measured in minutes. Report a step or a percentage, and record failures with a reason. A job that vanished into "running" forever is indistinguishable from a lost one.
The redelivery trap
If a worker exceeds its visibility timeout the broker redelivers the message while the original is still running, and two workers process the same job in parallel. Extend the timeout for genuinely long work, or heartbeat, and make the work itself safe to run twice.
Where the work should live
Where the work should live
| Duration | Approach |
|---|
| Under 1 second | Do it in the request |
| 1-10 seconds | In the request, but review the timeout budget |
| 10-60 seconds | Background it; a proxy will eventually cut this |
| Minutes | 202 plus a status URL, always |
| Hours | Job record, checkpoints, resumable from the last step |
Key facts
- Returning 202 with a status URL gives the client something durable to poll, which a held-open request cannot provide.
- A submission endpoint without an idempotency key duplicates expensive jobs whenever a client retries after a timeout.
- Exceeding a visibility timeout causes the broker to redeliver while the original worker is still running, so two workers process the same job at once.
- A status that reports only queued, running and done makes a stalled job indistinguishable from a lost one.
- Proxies, load balancers and clients each enforce their own timeout, so the shortest one in the chain decides how long a request may take.
Frequently asked questions
Can I just raise the proxy timeout?
Sometimes, and it postpones the problem rather than solving it. Every hop has its own limit and the shortest wins, and a long-held connection still gives the client no way to recover from losing it.
Polling or webhook for completion?
Offer both. Polling always works and needs nothing from the client; webhooks are cheaper at scale but require a reachable endpoint and signature verification. Systems that offer only webhooks are hard to develop against.
How do I stop two workers processing the same job?
You cannot fully, so make it harmless. Extend visibility timeouts for long work, heartbeat while running, and make the work idempotent — parallel delivery is a property of the broker, not a bug in your consumer.
Machine-readable copy of this page:
/guide/run-long-jobs-without-timeouts.md