CLI reference: retrack test and migrate
Reference for the retrack command: run memory-derived regression tests and model-migration replays, with options, entry points and exit codes.
Updated
Installing retrack-ai adds a retrack command with two subcommands. Both run your agent on your machine through an entry point you provide, and read RETRACK_API_KEY (or --api-key).
retrack test
Runs the regression cases generated from your agent's memory. See Regression tests in CI.
retrack test --entry my_agent:run_task --agent support-agent [--threshold 0.9] [--model-id MODEL]| Option | Default | Meaning |
|---|---|---|
--entry | required | module:function that runs one task: run_task(task, callbacks) -> str |
--agent | required | Agent name used with Retrack |
--threshold | 0.9 | Minimum pass rate |
--model-id | none | Model under test; recorded, and passed to a three-argument entry point |
--api-key, --base-url | environment | Override RETRACK_API_KEY and RETRACK_BASE_URL |
Exit codes: 0 pass; 1 pass rate below the threshold, or a critical case that passed last time now fails; 2 usage or connection error.
retrack migrate
Replays memory on a new model and reports what still works. See Model migration.
retrack migrate --to NEW_MODEL --entry my_agent:run_task --agent support-agent [--from OLD_MODEL] [--sample N --seed S]| Option | Default | Meaning |
|---|---|---|
--to | required | Target model, passed to run_task(task, callbacks, model_id) |
--from | none | Current model, for the report |
--sample | all cases | Replay only N cases |
--seed | 0 | Seed for --sample |
--entry, --agent, --api-key, --base-url | As for retrack test |
Exit 1 when anything breaks or a known failure returns, so it can gate a model switch.
The entry point
def run_task(task: str, callbacks: list, model_id: str | None = None) -> str:
...Pass callbacks to your agent (config={"callbacks": callbacks}) so Retrack can see which tools were called. Runs made by these commands are never stored as runs and never become memory.