Reliable automation: a trigger does not finish the job
Separate execution, durable results and delivery; recover individual steps and test retries without duplicating business effects.
Content produced with AI assistance for Thiago Silva’s website. Independent editorial analysis; it does not represent clients or employers.
The news behind this analysis
November curation revisits official AWS guidance on idempotency, retries and event consistency. This is an architectural synthesis, not a new product announcement or incident.
Original source ↗Specify the evidence that completes each step
Consider a hypothetical scheduled workflow that prepares a report for each organization and notifies its owner. A configured time demonstrates an intention to run. A started process demonstrates an attempt. Neither signal alone proves that the correct report was persisted or that its notification reached the destination. Architecture must preserve those distinctions so a partial failure remains diagnosable.
Editorial proposal: record the period, organization, input version, step, outcome and observation time. Keep prepared, persisted, provider-accepted and confirmed-delivered states separate when evidence for the last state exists. Exclude credentials and personal content from logs. This analysis was produced with AI assistance; its fictional scenario is neither a customer test nor a human review by Thiago.
Retries must preserve intent
The Amazon Builders’ Library describes request identifiers that let callers retry an operation without additional effects. It also addresses a repeated key with different parameters. This is an API contract, not a promise that an entire workflow executes only once.
For the hypothetical workflow, one option is a key composed of organization, period and result type. Recalculation with changed inputs requires an explicit revision rather than silent key reuse. Enforce uniqueness in storage: a preliminary lookup alone leaves a race between concurrent attempts. Continue checking authorization for the organization on every access.
Checkpoints and notifications serve different purposes
AWS’s transactional outbox pattern coordinates a data change and its event record in one local transaction. Publication follows separately; consumers must still handle duplicate messages. The pattern does not establish that a person read a notification.
In the proposed design, resume from the last identifiable durable result, not a generic success statement. Read the record back and compare its expected key and version. If the report is saved but notification failed, recover notification without regenerating the report. Keep delivery unknown until specific evidence supports a stronger state.
Bound retries and expect repeated events
Amazon’s timeout and backoff guidance warns that retries can amplify overload and that a timeout does not prove absence of effects. Jitter spreads attempts. Lambda’s asynchronous invocation documentation says that an event may recur even when the function returns no error.
Give each step a deadline, attempt budget, maximum delay and stop condition. Invalid credentials or a removed route require diagnosis rather than an unbounded loop. When an external effect may already have occurred, query its operation identifier before retrying. If neither reconciliation queries nor idempotency are available, assign the unresolved state to an accountable operator.
Test recovery as well as the happy path
Proposed exercise in an authorized environment with synthetic data: interrupt after report persistence but before notification; run two concurrent attempts; replay an old event; change the input version; and force destination failure. Check for one result per key, no access across organizations and recovery confined to the pending step.
Acceptance evidence should state the expected and observed outcome, limitations and owner of each outstanding issue. HTTP availability, a mocked-provider test and actual receipt are different checks. A passing execution supports that exercise; ongoing reliability depends on monitoring and reconciliation. The operational goal is fewer failures and recoverable work, without claiming that failures are impossible.