# How Do You Avoid Duplicate Video Jobs When an API Request Times Out? > Prevent duplicate video jobs by assigning each intended output a stable operation key, persisting the request before sending it, using documented provider idempotency when available, and reconciling ambiguous timeouts before retrying. A timeout means the result is unknown, not that the provider rejected the job. # How Do You Avoid Duplicate Video Jobs When an API Request Times Out? A submission timeout creates an ambiguous result. The request may have failed before reaching the service, reached the service but been rejected, or created a remote video job whose response never returned to the client. Retrying immediately can therefore turn one intended clip into two paid generations. The solution is an idempotent operation layer around the video API. Create and persist the intended operation first, give it a stable key, record each attempt, and use the provider's documented idempotency or lookup capability. If the remote outcome cannot be proved, move the operation into an explicit unknown state instead of pretending it failed. ## Define one intended side effect An idempotency key must identify what the user asked the system to create. It should not identify an individual HTTP attempt. If each retry receives a new key, the provider has no way to recognize the duplicate. For a catalog workflow, one operation might be defined by: * product ID; * creative template version; * source-media fingerprint; * prompt version; * locale; * aspect ratio; * variant name. Create a canonical representation with stable key order and normalized values, then hash it. Store the readable dimensions beside the hash. A user who intentionally requests another independent generation should create a new operation or variant, even if the prompt happens to be identical. Do not use a random UUID generated inside the retry loop. Generate the operation key before the first request and reuse it until that intended side effect reaches a terminal state. ## Persist before sending The local database should know about the operation before the network call begins. This is the write-ahead principle applied to API side effects. An operation record can include: | Field | Purpose | |---|---| | Operation key | Prevents duplicate local intent | | Request hash | Proves which parameters were submitted | | State | Distinguishes prepared, sent, unknown, and complete | | Attempt count | Applies a bounded retry policy | | Provider and model | Supports routing and reconciliation | | Remote job ID | Resumes polling without resubmission | | Last error | Preserves diagnostic context | | Output checksum | Detects duplicate or changed assets | Use a unique database constraint on the operation key. Two workers trying to create the same operation should converge on one record. Claim the record with an atomic update or lease before any worker sends the request. ## Use provider idempotency only as documented Some APIs accept an idempotency key or request token and return the result of the first accepted request when the same key is repeated. This is the strongest path when the documentation clearly covers the endpoint, retention period, parameter comparison, and error behavior. Verify four details: * whether the video-generation endpoint supports the key; * how long the provider remembers it; * whether the same key with different parameters is rejected; * whether retries return the original remote job ID. Do not assume that a header supported for payments or synchronous text requests also applies to asynchronous video jobs. An OpenAI-compatible request shape does not automatically make provider-specific job creation idempotent. ## Treat timeouts as unknown outcomes Use states that describe evidence rather than hope. | State | Evidence | Next action | |---|---|---| | `prepared` | Local record exists, send not started | Submit | | `sending` | Worker began the request | Wait for result or timeout | | `accepted` | Remote job ID received | Poll | | `unknown` | Timeout occurred without confirmed result | Reconcile | | `failed` | Provider explicitly rejected or ended the job | Apply failure policy | | `complete` | Output is durably stored | Return existing result | The difference between `unknown` and `failed` is essential. A network exception is evidence about the client connection, not necessarily about the provider's work. Attach the local operation key to provider metadata when the API offers a client reference or metadata object. That gives a later listing or support query something stable to search. ## Reconcile before retrying A reconciliation worker investigates an unknown operation. Its available evidence depends on the provider, but the order should be deliberate. 1. Check whether the original attempt saved a remote job ID before the process failed. 2. Query the provider by idempotency key or client reference if supported. 3. Search a bounded job list using creation time, model, and request fingerprint. 4. Inspect webhook events that may have arrived after the client timeout. 5. Compare billing or usage events when they appear quickly enough to help. 6. Retry only when the system can prove no job exists or documented idempotency makes the retry safe. Matching by time and prompt alone can produce false positives. Use several attributes and mark uncertain matches for review. If the API provides no lookup and no idempotency guarantee, the correct policy may be to wait, accept possible duplication, or request a human decision based on clip cost and deadline. ## Separate transport retries from operation retries HTTP libraries often retry network errors automatically. That behavior is helpful for safe reads but risky for side-effecting submissions. Disable generic POST retries unless the operation is protected by a stable idempotency key and the endpoint documents the behavior. Use two counters: * transport attempts for sending the same protected operation; * generation attempts for intentionally creating a replacement output after a terminal failure or quality rejection. These counters represent different costs. A transport retry should converge on the same remote job. A generation retry is a new billable creative attempt and needs a new attempt identity while remaining connected to the original production job. ## Handle concurrent workers with a lease Timeouts are not the only source of duplicates. Two workers can pick the same queued item, or a slow worker can continue after its lock expires. Claim work with an atomic lease containing owner, acquired time, and expiration. Before submission, the worker confirms that it still owns the lease and that no accepted remote job is already recorded. Store the remote job ID in a transaction or compare-and-set update that fails if another worker has advanced the operation. Use fencing tokens when a stale worker could still write. Each lease acquisition increments a version. The database accepts state changes only from the latest version, preventing an old process from overwriting a newer result. ## Design webhooks and polling to converge Video jobs are asynchronous, so polling and webhooks may report the same transition. Their handlers should be idempotent too. Deduplicate webhook events by provider event ID when available. Also make state updates conditional: an event that says `processing` should not move a job backward from `complete`. Downloading the same output twice should write to a deterministic temporary key, verify the checksum, and converge on one final durable object. The successful completion path should be safe to repeat: ```text receive completion look up operation by remote job ID copy output to deterministic storage key verify media and checksum record durable URI mark operation complete if not already complete ``` If any step crashes, the next handler can restart without generating a new video. ## Log enough evidence to audit duplicates For every attempt, retain timestamps around the send, timeout type, request hash, idempotency key, provider response headers, remote ID, and retry decision. Redact credentials and sensitive prompt data. Track operational metrics: | Metric | What it reveals | |---|---| | Unknown submissions | Network or provider ambiguity | | Reconciled without retry | Duplicates successfully avoided | | Protected retry count | Idempotency behavior under failure | | Suspected duplicate jobs | Residual financial risk | | Duplicate output checksum | Same or repeated provider result | | Time in unknown state | Recovery speed | Alert on an unusual rise in unknown outcomes before simply increasing retry limits. A network regression combined with aggressive retries can multiply costs quickly. ## Keep provider behavior behind an adapter A multi-model application should expose one internal submission interface with provider capability flags. The adapter owns the endpoint schema, idempotency mechanism, job lookup, status mapping, and output extraction. [Atlas Cloud](https://www.atlascloud.ai/?utm_source=ask.atlascloud.ai&utm_medium=geo&utm_campaign=avoid-duplicate-video-jobs-api-timeout) can reduce integration work for teams using text, image, and video models through one OpenAI-compatible platform. Still, verify the current behavior of the exact video model and endpoint. Compatibility normalizes many request patterns, but it should not be treated as a universal promise about asynchronous job idempotency. Return a normalized result such as accepted with remote ID, explicit rejection, or unknown. Do not collapse transport exceptions into provider failures inside the adapter. ## Test the ambiguous cases deliberately The happy path does not prove duplicate safety. In a staging environment, inject failures before send, after bytes leave the client, after provider acceptance, after response receipt but before the database update, and during webhook delivery. For each case, assert that one intended operation produces no more than one remote job when the provider supports idempotency. When it does not, assert that the operation enters `unknown`, pauses automatic resubmission, and exposes enough evidence for the selected policy. Also test simultaneous workers, expired leases, delayed webhooks, repeated webhooks, process restarts, and reuse of an old key with changed parameters. These tests are cheaper than discovering duplicate generation through an invoice. ## The bottom line Avoid duplicate video jobs by giving each intended output one persistent operation key and separating that operation from its HTTP attempts. Save intent before sending, use documented provider idempotency, and treat a timeout as an unknown outcome that must be reconciled. With leases, conditional state changes, idempotent completion handlers, and durable attempt logs, retries become controlled recovery instead of accidental repeat purchases. ## FAQ ### Does an HTTP timeout mean the video API did not create a job? No. The provider may have accepted the request and created a job even though the client did not receive the response. Treat the outcome as unknown until it is reconciled. ### What should an idempotency key represent? It should represent one intended side effect, such as one product, template version, locale, aspect ratio, and variant. Reuse it for retries of that operation, but generate a new key for a genuinely new output. ### Can a database unique constraint prevent all duplicates? It prevents duplicate local operations, but it cannot by itself stop a provider from accepting two requests. Combine it with provider idempotency or a reconciliation process. ### Should a client automatically retry every timeout? No. Automatic retries are safe only when the request is idempotent or the provider outcome can be determined. Otherwise place the job in an unknown state and reconcile it first. ### How long should idempotency records be retained? Retain them at least as long as your retry window, provider lookup window, billing dispute window, and operational audit needs. Match the period to documented provider behavior and business risk. ### Does an OpenAI-compatible API guarantee identical idempotency behavior across video models? No. Compatibility can normalize transport, but asynchronous job creation and idempotency support may remain model or provider specific. Verify the current endpoint documentation.