What is a failure signature?
A failure signature identifies one kind of agent failure by tool, error and input shape, with its frequency and the fix that resolved it.
Updated
A failure signature is a recognisable description of one kind of agent failure, usually the tool, the error class and the shape of the input, together with how often it happened and, when known, the fix that resolved it.
Why fingerprint failures
Agent failures repeat. The same tool rejects the same malformed input, the same API refuses the same request without approval. Fingerprinting each failure lets a system count it, recognise it next time and attach what fixed it, so the agent can be warned in advance instead of failing again.
What a failure signature holds
- a fingerprint, a hash of tool, error class and input shape,
- occurrences and when it was last seen,
- the tool's error message, cleaned so it applies beyond one task,
- the recovery: what a successful retry changed,
- whether it is unrecoverable, so retrying is pointless.
Uses
- Warnings in the agent's prompt before a related task.
- A circuit breaker that stops retries that cannot succeed.
- Regression tests asserting the failure does not come back. See regression testing for agents.
In Retrack
See Failure signatures and the docs.