Documentation menu

CLI reference: retrack test and migrate

Reference for the retrack command: run memory-derived regression tests and model-migration replays, with options, entry points and exit codes.

Updated

Installing retrack-ai adds a retrack command with two subcommands. Both run your agent on your machine through an entry point you provide, and read RETRACK_API_KEY (or --api-key).

retrack test

Runs the regression cases generated from your agent's memory. See Regression tests in CI.

retrack test --entry my_agent:run_task --agent support-agent [--threshold 0.9] [--model-id MODEL]
OptionDefaultMeaning
--entryrequiredmodule:function that runs one task: run_task(task, callbacks) -> str
--agentrequiredAgent name used with Retrack
--threshold0.9Minimum pass rate
--model-idnoneModel under test; recorded, and passed to a three-argument entry point
--api-key, --base-urlenvironmentOverride RETRACK_API_KEY and RETRACK_BASE_URL

Exit codes: 0 pass; 1 pass rate below the threshold, or a critical case that passed last time now fails; 2 usage or connection error.

retrack migrate

Replays memory on a new model and reports what still works. See Model migration.

retrack migrate --to NEW_MODEL --entry my_agent:run_task --agent support-agent [--from OLD_MODEL] [--sample N --seed S]
OptionDefaultMeaning
--torequiredTarget model, passed to run_task(task, callbacks, model_id)
--fromnoneCurrent model, for the report
--sampleall casesReplay only N cases
--seed0Seed for --sample
--entry, --agent, --api-key, --base-urlAs for retrack test

Exit 1 when anything breaks or a known failure returns, so it can gate a model switch.

The entry point

def run_task(task: str, callbacks: list, model_id: str | None = None) -> str:
    ...

Pass callbacks to your agent (config={"callbacks": callbacks}) so Retrack can see which tools were called. Runs made by these commands are never stored as runs and never become memory.