Documentation menu

How Retrack works

The pipeline behind experience memory: capture every agent run, verify it, distil playbooks and failure signatures, recall them, and protect memory.

Updated

Retrack turns your agent's own history into memory it can use. Every run goes through the same pipeline, in the background, without slowing your agent down.

The pipeline

1. Capture

The connector records each agent run when it finishes: the task, every tool call with its arguments and result, the final answer, tokens, cost, latency, the model and your tool versions. Runs are sent in the background with a bounded buffer, so a slow network never blocks your agent.

Secrets are stripped twice. The connector removes obvious credentials (API keys, tokens, passwords) on your machine before sending, and the server redacts secrets and personal data (emails, phone numbers, card numbers and national ID numbers) before anything is stored or embedded.

2. Verify

A run only teaches if Retrack knows whether it worked. Retrack uses the strongest signal available, in this order:

SourceExampleConfidence
testA test or assertion in your code passed1.0
eventA business event: the ticket was resolved, the refund settled0.9
feedbackA user's thumbs-up or rating0.8
AI judgeNo signal arrived, so an AI judge reads the runat most 0.7

The judge checks the final answer against the steps, so an agent that claims success after a failed key step is scored as a failure. A crash in a tool or in your agent's code counts as a failure; an outage of the model provider does not.

See Verifying runs.

3. Distil

Verified successes of the same kind of task are merged into one playbook: preconditions, steps (some only under a condition) and pitfalls. Failures become failure signatures: the tool, the error and, when a later retry worked, the fix.

Playbooks keep learning. When a verified run teaches something new (a tool the task needed, an error it recovered from), the playbook gets a new version, and every version is kept so you can roll back.

4. Recall

Before a task, your agent calls retrack.recall(task). Retrack searches by meaning and by keywords, scores each candidate playbook, and returns one only when it clearly applies, together with warnings about failures relevant to the task. See Recall and the confidence gate.

5. Protect

Throughout, the memory firewall decides what may become memory. Runs that touched untrusted input (user text, emails, web pages, third-party tools) cannot create or change active memory without passing checks or your review.

The life of a playbook

StatusMeaningRecalled?
candidateLearned, waiting for evidence or your approvalNo
activeApproved by you, or promoted after 3 trusted, verified successesYes
quarantinedStopped by the firewall or by you; waits for reviewNo
retiredTaken out of service by you, or retired after its success rate fell below 50% over 10 or more usesNo

You see and manage all of them in the dashboard under Memory and Review.

The same memory, four jobs

  • Learning: playbooks and warnings before each task.
  • Testing: every active playbook and recurring failure becomes a regression case for retrack test.
  • Migrating: retrack migrate replays memory on a new model and reports what still works.
  • Proving: a holdout group gets no memory, so the effect is measured.