Define how an outage is recognised
Monitor both technical availability and the expected flow of work. A running job can still fail to read its source, while a quiet source may legitimately produce no new items. Compare progress with what the source actually contains. Decide which errors should pause writes and who receives the alert. The alert should name the affected process and known impact without claiming data was lost when that is not established. Include where the operator can find the current state and the agreed recovery instructions.
Protect incoming work
Determine whether new items remain in the source, enter a durable queue or require a temporary manual process. Do not assume an event will be delivered again after a failure. Verify the actual source behaviour and retention limits during preparation. If intake must pause, tell the responsible team what to do with new requests. Keep any manual register linked to the later recovery plan so those items are not forgotten or processed twice. The important question is where each request waits while the normal route is unavailable.
Identify partially completed items
Separate work that never started, work known to have completed and work with an uncertain destination result. A timeout after a write belongs in the uncertain group until checked. Avoid resetting all items to new, because that can repeat orders, messages or record updates. Use the original source and destination references to establish what happened. If automatic checks cannot resolve an item, assign it to a person with the necessary access. Preserve the evidence rather than clearing the error history simply to make the dashboard look healthy.
Resume with a small verified batch
After the underlying service is restored, process a controlled subset and inspect the destination outcomes. Check authentication, field mappings and any changes introduced during the outage. A successful connection test does not prove the full workflow works. Watch for duplicate writes and notifications. Choose a batch size that staff can reconcile before widening recovery, rather than inventing a universal number. Account for provider limits and the capacity of downstream teams. A large backlog can overwhelm people even when the systems can technically process it quickly.
Reconcile the backlog
Compare the original affected work with completed, pending and explicitly excluded items. Check that the totals account for the source and inspect critical records directly. Include work handled manually during the interruption. A hypothetical request entered by staff must not become a second task when automation catches up. Record unresolved items with owners instead of declaring recovery complete because the queue is smaller. Keep customer-facing statements accurate: service restored and all delayed work completed are different milestones.
Improve the recovery plan from evidence
After recovery, record the cause, affected period, actual business impact and the steps that worked. Update the operating guide with any missing checks. Test the recovery path in a controlled environment where practical, including uncertain writes and delayed events. Avoid adding repeated alerts without improving who acts on them. Give the business owner a clear account of what remains unresolved. The plan is useful when another authorised person can use it during the next interruption without relying on the memory of whoever handled this one.