Genace
Sign inGet started

Redis generation job lease: coordinate workers without storing the ledger in a lock

A short Redis lease can reduce duplicate observer work across application processes. It is not a durable record of a generation, and it is not proof that an external side effect occurred once. Keep job identity and accounting in your database, then use Redis only for the coordination role your design has explicitly assigned it.

Acquire with a unique owner token

Use a random token for each lease acquisition and an expiration appropriate for the short action. With a Redis client, the acquisition corresponds to SET with NX and PX options. A simple command-level example is:

SET app:observe:operation-42 random-owner-token NX PX 45000

An unsuccessful acquisition means another worker currently holds the lease. Do not spin continuously or treat it as a failed remote generation. Reschedule the local observation and retain the same job ID.

For release, compare the stored token before deleting the key. The following Lua logic illustrates that condition; execute it through the scripting API supported by your Redis client.

if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
end
return 0

Deleting unconditionally can remove a newer worker’s lease after yours expired. Renewal needs the same ownership check and should fail clearly when the token no longer matches. These examples describe a single coordination pattern, not a complete high-availability lock guarantee for every Redis deployment.

Fence durable state changes separately

Store a revision or fencing value in the application database and require it when saving observations. A lease can expire while a paused worker is still executing. That stale worker must not overwrite the current operation merely because it once acquired Redis ownership.

For a known remote ID, duplicate GET observations do not create another generation, but they can waste rate budget and race local state. Use conditional transitions so a late running snapshot cannot replace a saved succeeded outcome. Keep output delivery deduplicated by operation and output identity as well.

Do not use lease loss as a submission retry signal

A dispatcher may send the POST and lose its Redis connection before persisting the response. Another worker acquiring the lease cannot infer that nothing happened. Mark an operation without a reliable outcome uncertain and reconcile it. The public generation route has no documented request idempotency key tied to your Redis token.

Once a job ID is saved, recovery should observe that same job. An HTTP 502 submission failure can provide error.job_id for local accounting inspection. Refund idempotency is independent of the Redis lease and must not be described as creation deduplication.

Acceptance checks

Test lease expiry during a paused worker, renewal after ownership changes, and a delayed release using the old token. Verify the newer token survives. Simulate Redis unavailability and confirm your application stops new claims or follows a reviewed fallback rather than dispatching without coordination accidentally.

Finally, restart Redis in a test environment and confirm durable job records remain available from the database. A vanished lock should not erase accepted generations or authorize their recreation.

Continue with database worker leases, PostgreSQL job storage, and queue visibility.

Continue with production operations

Where these facts come from