Command Line Interface

The RetroCast CLI provides two ways to run schema-2 workflows.

Project mode uses the standard data directory and writes manifests for reproducible benchmarking. Ad-hoc mode works directly on explicit files and is useful for small experiments, debugging adapters, or notebooks.

!!! info "Two modes of operation"

**Project mode** is the structured workflow for reproducible benchmarking of one or more models.

**Ad-hoc mode** is for direct file-to-file commands when you do not want to use the project directory layout.

Installation

Download the standalone archive for your platform from GitHub Releases. Each archive contains the retrocast executable and its required native libraries. The PyPI wheel is the Python binding and does not install this command.

Verify installation:

retrocast --version

Global Options

These options apply to all commands:

retrocast [--config CONFIG] [--data-dir DATA_DIR] <command>

Data Directory Resolution

The data directory is resolved in this order:

1. CLI flag: --data-dir /custom/path
2. environment variable: RETROCAST_DATA_DIR=/custom/path
3. config file: data_dir: /custom/path in retrocast-config.yaml
4. default: data/retrocast/

Check the resolved configuration with:

retrocast config

Examples:

# Use CLI flag. This has highest priority.
retrocast --data-dir ./my-data ingest --model my-model --dataset mkt-cnv-160

# Use environment variable.
RETROCAST_DATA_DIR=./my-data retrocast ingest --model my-model --dataset mkt-cnv-160

# Inspect where RetroCast is looking.
retrocast --data-dir ./my-data config

!!! warning "Migration from older versions"

The default data directory changed from `data/` to `data/retrocast/` in version 0.6. If you have existing data at `data/`, either move the RetroCast subdirectories under `data/retrocast/` or set `RETROCAST_DATA_DIR=data`.

Ad-Hoc Workflow

Use ad-hoc commands when you want to process explicit files without placing them in 2-raw/.

!!! tip "When to use ad-hoc mode"

Use this mode for quick experiments, one-off evaluations, adapter debugging, and notebooks. If you are running a full benchmark across model directories, use project mode instead.

raw artifact -> adapt -> candidates -> collect -> collected candidates -> score-file -> evaluation

adapt - Convert Raw Predictions

adapt casts raw planner output into ranked schema-2 Candidates. Each candidate contains either a valid Route or a FailureRecord.

retrocast adapt \
  --adapter paroutes \
  --input raw.json.gz \
  --output candidates.json.gz

Options:

List adapters with:

retrocast list-adapters

collect - Align Candidates To A Benchmark

collect maps candidates onto benchmark targets.

retrocast collect \
  --input candidates.json.gz \
  --benchmark benchmark.json.gz \
  --output collected.json.gz

Successful candidates are collected by route target identity. Failed candidates are collected by the target hints preserved in their FailureRecord.

score-file - Evaluate Collected Candidates

score-file scores collected candidates against a benchmark task and stock file.

retrocast score-file \
  --benchmark benchmark.json.gz \
  --candidates collected.json.gz \
  --stock buyables-stock.txt \
  --output evaluation.json.gz

Options:

score-file writes an Evaluation artifact. Tier-0 validity comes from adaptation success or failure; task satisfaction comes from the configured constraints. --acceptable-route-match is applied when this artifact is scored; rerunning analyze on an existing Evaluation does not retroactively change its stored acceptable-route matches.

compare pareto-frontier - Compare Reports

compare pareto-frontier compares schema-2 analysis reports against cost or runtime metadata.

retrocast compare pareto-frontier compare.yaml --no-open

benchmark: small
stock: buyables
metric: solv_0[buyables]_rate
output_dir: comparisons
sources:
  - root: data/retrocast
    models:
      - name: model-a
        hourly_cost: 1.0
      - name: model-b
        hourly_cost: 0.5

Project Workflow

!!! success "Recommended for research"

Project mode is the right default for reproducible benchmarking, multi-model comparison, manifest tracking, and long-running experiments.

Project mode uses the standard data directory:

data/retrocast/
  1-benchmarks/
    definitions/
    stocks/
  2-raw/
  3-processed/
  4-scored/
  5-results/

Project Setup

Put raw output under:

<data-dir>/2-raw/<model>/<benchmark>/<raw-results-file>

Project-mode ingest resolves the adapter in this order:

1. --adapter on the command line
2. manifest.json next to the raw file with directives.adapter
3. error if neither is provided

If the raw filename is not declared in manifest.json, ingest reads results.json.gz.

{
  "directives": {
    "adapter": "aizynthfinder",
    "raw_results_filename": "results.json.gz"
  }
}

Evaluation flow

graph LR
    A[2-raw/] --> B[ingest]
    B --> C[3-processed/]
    C --> D[score]
    D --> E[4-scored/]
    E --> F[analyze]
    F --> G[5-results/]

All paths are relative to your data directory.

ingest - Adapt and Collect Candidates

ingest runs adapt + collect for one or more model/benchmark pairs.

retrocast ingest \
  --model my-model \
  --dataset mkt-cnv-160 \
  --adapter aizynthfinder

Useful options:

Input:

<data-dir>/2-raw/<model>/<benchmark>/<raw_results_filename>

Output:

<data-dir>/3-processed/<benchmark>/<model>/candidates.json.gz

The output is collected Candidates keyed by target id. Failed adaptation slots are preserved so Solv-0 and MRR denominators remain honest.

score - Evaluate Candidates

score evaluates processed candidates for one or more model/benchmark pairs.

retrocast score \
  --model my-model \
  --dataset mkt-cnv-160

Useful options:

--acceptable-route-match is applied when the Evaluation is scored; rerun score before analyze to apply different matching semantics to existing processed candidates.

Input:

<data-dir>/3-processed/<benchmark>/<model>/candidates.json.gz

Output:

<data-dir>/4-scored/<benchmark>/<model>/<stock>/evaluation.json.gz

The output is an Evaluation: scored candidates plus the task context needed to interpret Solv-N metrics.

analyze - Generate Reports

analyze summarizes evaluation artifacts into an AnalysisReport and a markdown report.

retrocast analyze \
  --model my-model \
  --dataset mkt-cnv-160

Useful options:

Outputs:

<data-dir>/5-results/<benchmark>/<model>/<stock>/analysis.json.gz
<data-dir>/5-results/<benchmark>/<model>/<stock>/report.md

Verification & Auditing

RetroCast writes a manifest.json next to generated project-mode artifacts. The manifest records inputs, outputs, parameters, hashes, and workflow statistics.

verify - Check Data Integrity

Verify all manifests under a data directory:

retrocast --data-dir data/retrocast verify --all

Verify one manifest or directory:

retrocast verify --target data/retrocast/5-results/mkt-cnv-160/my-model/buyables

Use --deep to follow generated-artifact manifest links. Use --strict to treat missing files as failures.

Configuration & Debugging

config - Show Resolved Configuration

Shows the resolved data directory and project paths.

retrocast config

If commands cannot find your files, start here. It shows exactly which data directory RetroCast resolved and which stage directories exist.

Helper Commands

list - Show Raw Manifests

Lists raw model/benchmark manifests discovered under 2-raw/.

retrocast --data-dir data/retrocast list

list-adapters - Show Available Adapters

Lists built-in adapters and deprecated adapter slug aliases.

retrocast list-adapters

Command Reference