Key takeaways
- Model planning does not make external side effects atomic; each tool call can succeed, fail, time out, or be repeated independently.
- Assign idempotency keys, checkpoints, verification, timeout policy, and a compensation or repair owner to every material step.
- Do not promise rollback when the real action is a new business operation that can also fail.
- Test partial completion and delayed responses before broadening workflow authority.
01
Write the failure story before the happy path
Imagine an agent creates a CRM record, adds a person to an audience, books a meeting, and sends confirmation. The booking succeeds but its response times out. A blind retry creates a second meeting; a later failure leaves the audience and CRM out of sync. The model did not cause distributed systems to stop being distributed.
List every externally visible effect, whether it is reversible, how success is verified, how duplicate execution is prevented, and who repairs it when automated compensation fails.
02
Make progress durable and repeatable
Give the logical workflow and each side effect stable identifiers. Persist checkpoints outside the model context. Before executing, check whether the operation already completed; after executing, verify authoritative state. Retries should repeat a decision safely, not repeat its consequence blindly.
Use bounded retries with backoff and a dead-letter or exception path. An agent should not keep spending money or calling an unavailable service because the error message looked temporary.
Swipe to compare every column
| Failure | Unsafe response | Designed response |
|---|---|---|
| Timeout after success | Repeat the write | Check idempotency key and authoritative state |
| Step three fails | Start the whole run again | Resume from checkpoint or compensate completed effects |
| Compensation fails | Mark the run rolled back | Escalate with exact remaining state |
| Policy changes mid-run | Use old approval | Re-evaluate before the next privileged action |
03
Call compensation what it is
Deleting a newly created draft may approximate rollback. Canceling a sent message or undoing a public post does not erase who saw it. A refund is a new financial transaction, not time travel. Describe the residual effect and customer communication honestly.
Research on transactional agent tool use highlights side effects, idempotency, isolation, and compensation as runtime concerns. Even without a specialized runtime, these concepts belong in the application layer around the agent.
04
Rehearse partial failure
Inject timeouts before and after tool success, stale reads, duplicate events, unavailable approval, changed permissions, malformed responses, and failed compensation. Confirm that the run stops within limits and leaves a precise repair record.
Report incomplete workflows, duplicate prevention, compensation success, manual repair time, and customer impact. Completion rate alone can hide a trail of damaged state.
Primary sources and further reading
Use the source material to validate details against your own context and current platform configuration.
- Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows
- OWASP: Excessive Agency
- NIST: Tool Use in Agent Systems
This field note follows the XenGrowth editorial policy: primary sources where available, visible limitations, material review dates, and no invented first-hand experience.
Stay with the problem
