Validation and preparation are complete and approval waits on a decision, while a restart replays recorded results and can rerun unfinished work.
A multi-step operation, such as an asset deployment or a corporate action, recovers from a crash by reusing the result of every step it recorded. The engine stores each activity's result and reuses it on replay. An activity interrupted before its result was saved runs again, so an external effect needs its own duplicate protection.
The engine tracks a run as running, waiting, suspended, completed, or failed. A run enters waiting for an approval, a threshold, or a timer with a deadline the operation defines, and suspended when an infrastructure retry sequence is exhausted and awaits automatic recovery.
On-chain writes still use the Transaction Queue for deduplication and sequencing. Durable execution alone does not make external effects happen exactly once.
Why a multi-step operation needs durable execution
An operation such as deploying an asset, processing a corporate action, or approving a batch of identity claims is never one call. It chains several steps, each one a separate effect: a custody signature, an on-chain write, a compliance check, sometimes a wait for a threshold or an approval. If the process running that chain restarts before it finishes, nothing outside the platform can tell which of those effects already happened.
A naive retry fails in one of two ways. Restarting the whole operation can resubmit a step that already reached the chain, so an asset deploys twice or a claim is minted twice. Resuming from wherever the process happens to pick back up can skip a step that never ran, so a required step is silently missing from an operation the caller believes is complete. Both mistakes are invisible from outside. The caller sees a result and cannot tell whether each internal step ran once.
The Workflow Engine keeps the steps inside one multi-step operation from running twice or from silently not running at all after a restart.
Why one recovery layer serves every operation, not one per domain
The alternative is for each kind of operation to carry its own recovery logic, with one retry scheme for an asset deployment, another for a corporate action and a third for a batch of identity claims. Each scheme would answer the same question, whether a given step already ran, in its own way, so recovery would depend on which team wrote the retry code.
The Workflow Engine is shared machinery instead. One execution record and one replay rule serve every domain that runs a multi-step operation, whether it deploys an asset, provisions a custody wallet, or processes a corporate action. An integrator reads the same kind of status and the same kind of recorded evidence whatever operation they started, because the recovery layer underneath does not change.
What the engine records, and what replay does with it
The engine records each activity's result as it completes. A restarted process replays the workflow's code from the top, but a completed activity returns its stored result instead of running its work again. An activity interrupted before its result was saved executes again.
Replay only guarantees the workflow's own bookkeeping. It says nothing about the business outcome of an external effect, so external effects still need their own duplicate protection, including writes submitted through the Transaction Queue. A stored result does not prove every business outcome was successful. Read the operation's result and any failure details before reporting success.
Running, waiting, and suspended are different states
A running workflow executes its next activity. Waiting means the operation needs a condition such as an approval, funds, or a timer. It can continue when that condition is satisfied. A deadline can end a wait, but the workflow decides what happens when the deadline expires.
Suspended means an infrastructure failure exhausted the activity's immediate retries. The platform schedules further recovery attempts with backoff, and persistent failure can end the run as failed. When an incident prevents progress, restore the unavailable dependency.
A Canton submission behaves the same way. When the participant is unreachable or answers with a retryable error, the submission suspends and the platform runs it again later instead of failing it. If every recovery attempt fails, a submission that never reached the ledger ends as failed with the code canton.. A submission the ledger may already have run is not failed. The platform holds it and, on each later attempt, first reads the ledger's completion for the command: a successful completion settles it, a rejection fails it with the ledger's reason, and no completion once the command's validity window has closed expires it. It never signs the command again or releases funds while the outcome is open. Operators see a held submission through the CantonSubmissionInDoubt alert and the canton. log line, and resolve it by restoring the participant so its completion can be read.
Completed and failed are terminal workflow statuses. They describe the workflow's outcome. Waiting and suspended are non-terminal. Status projections can lag during a database outage, so a stale running status does not prove that an activity is still executing.
Running leads to completed or failed, and a step can pause it into waiting, or the platform can park it as suspended, with both pauses returning to running.
When an operation pauses for funds
An operation configured to wait for gas funding pauses the affected step when the paying wallet has insufficient funds. The work recorded before that step remains available for replay. The funding request identifies the wallet or deposit to fund, the required amount, and the deadline.
The amount required to retry the refused operation controls when it can resume. The operator may also see an estimate for the remaining work. That estimate helps plan funding but does not raise the resume threshold.
Fund the destination shown in the request. An ordinary signing account or a self-funded smart account uses an address balance. A paymaster uses its deposit in the EntryPoint contract. Funding the paymaster address alone does not increase that deposit.
A balance observation and a transaction broadcast can use different network nodes. A deposit visible to one node may take time to reach another. The platform makes bounded broadcast retries after funding. A repeated rejection does not always mean another deposit is needed.
When the funding deadline expires, read the recorded error, fund the requested destination, and start a new attempt as the operation's API specifies. Operations that do not enable the funding pause fail with GAS_REQUIRED, which identifies the funding shortfall.
The engine and the API run separately
The API hands durable work to the Workflow Engine, which runs in a separate process. The engine uses database-backed execution records to recover after a restart, so recovery waits for that database when it is unavailable. Its availability and capacity can therefore differ from the API's: an accepted request can remain in progress while the engine recovers.
Workflow Engine
Coordinates activities and stores their results for replay. Its result describes the multi-step operation.
Transaction Queue
Deduplicates, sequences, and executes the on-chain writes submitted by activities. Each write has its own outcome.
Follow the operation's status contract
An asynchronous start returns an operation identifier or status link. Acceptance means the platform accepted the request for processing. It does not mean the operation finished. Use the status endpoint returned by that operation and interpret its documented states.
After a lost response, inspect the original operation before sending another request. Repeat the API request with the same Idempotency-Key only when its route documents replay safety. In the Workflow Engine, reusing the key does not create a fresh attempt after a terminal failure. Correct the failure's cause before starting a new business attempt with a new key.
A client polls until the operation reaches a documented terminal state. Read the resulting resource after success. A pending operation does not promise that a separate resource read contains its changes.
For operational monitoring, the Workflows panel on the Console's Platform status page shows workflow health, as Platform Health describes. That view complements the status endpoint used by the integration. It does not replace the operation's result.
Recovery records and audit evidence
The execution journal records activity results needed for recovery. It has a retention policy and does not replace the platform's Audit Trail, the append-only record of who did what and when, or your compliance archive. Retain the operation identifier, transaction references, approvals, and other evidence your policy requires.
A workflow's recorded decision also does not prove that a token transfer passed compliance. The token's transfer checks decide whether its balances may change. Read the write's Settlement Verdict, the typed terminal outcome recorded for that on-chain write, to establish the on-chain outcome.
A failed workflow exposes error information through its operation's status contract. Read the error code and retry guidance before deciding whether to change the request, restore a dependency, or escalate.
Frequently asked questions
Can a workflow wait indefinitely?
Yes, if its wait has no deadline. Durable execution preserves the wait and imposes no universal time limit. Check the deadline the operation you are integrating documents for its wait, and design an approval or funding process around it.
Does completed mean everything the operation requested succeeded?
Read the operation's result. A workflow can complete after recording a business rejection or a partial outcome that its own contract permits. The workflow status alone does not describe each external effect.
Should I restart a suspended workflow manually?
The platform schedules recovery attempts for infrastructure suspension. Check dependency health and the operation's recorded details before starting another request. A second request with a new key can create separate work while the original run is still recoverable.