Skip to content

Quick Start

This guide gets you from raw model output to a rigorous statistical report in under 5 minutes.

What you'll learn

  • Install RetroCast and initialize a project
  • Configure an adapter for your model's output format
  • Run the ingest → score → analyze pipeline
  • Generate statistical reports with confidence intervals

1. Install

We recommend installing RetroCast as a standalone tool using uv:

uv tool install retrocast

If you don't have uv, you can install it in one minute:

curl -LsSf https://astral.sh/uv/install.sh | sh

pip install retrocast

Verify installation:

retrocast --version

2. Initialize Project

Go to your working directory and create the default configuration and directory structure:

retrocast init

This creates:

  • retrocast-config.yaml - Configuration file
  • data/ - Structured data directories (1-benchmarks, 2-raw, 3-processed, 4-scored, 5-results)

Configure Your Model

Open retrocast-config.yaml and register your model. You need to tell RetroCast which adapter to use to parse your files.

retrocast-config.yaml
models:
  # The name you will use in CLI commands
  my-new-model: # (1)!
    # The parser logic (see docs/developers/adapters.md)
    adapter: aizynth # (2)!
    # The filename RetroCast looks for in data/2-raw/
    raw_results_filename: predictions.json # (3)!
    sampling: # (4)!
      strategy: top-k
      k: 10
  1. Choose a descriptive name (lowercase with hyphens)
  2. See supported adapters - includes AiZynthFinder, Retro*, DMS, SynPlanner, Syntheseus, ASKCOS, and more
  3. Must match the filename you'll place in data/2-raw/
  4. Optional: Limit routes per target (omit to keep all routes)

3. The Workflow (Ingest → Score → Analyze)

RetroCast enforces a structured workflow to ensure reproducibility:

graph LR
    A[Place Raw Data<br/>data/2-raw/] --> B[Ingest<br/>Standardize]
    B --> C[Score<br/>Evaluate]
    C --> D[Analyze<br/>Statistics]

    B -.-> E[data/3-processed/]
    C -.-> F[data/4-scored/]
    D -.-> G[data/5-results/]

Step A: Place Raw Data

Put your model's raw output file in the data/2-raw/ directory following this structure:

data/2-raw/<model-name>/<benchmark-name>/<filename>

Example:

mkdir -p data/2-raw/my-new-model/mkt-cnv-160
cp predictions.json data/2-raw/my-new-model/mkt-cnv-160/

Available benchmarks

See Benchmarks Guide for details on evaluation sets:

  • Market Series (mkt-*): Practical utility with commercial stock
  • Reference Series (ref-*): Algorithm comparison with ground-truth stock

Step B: Ingest

Convert raw output into the canonical RetroCast Route format. This standardizes data and removes duplicates.

retrocast ingest --model my-new-model --dataset mkt-cnv-160

Output: data/3-processed/my-new-model/mkt-cnv-160/routes.json.gz

Step C: Score

Evaluate routes against the benchmark's defined stock.

retrocast score --model my-new-model --dataset mkt-cnv-160

Output: data/4-scored/my-new-model/mkt-cnv-160/scores.json.gz

Step D: Analyze

Generate final report with bootstrapped confidence intervals and visualization plots.

retrocast analyze --model my-new-model --dataset mkt-cnv-160

Output: data/5-results/mkt-cnv-160/my-new-model/

  • report.md - Statistical summary
  • *.html - Interactive plots (add --make-plots arg and make sure to install viz dep group, i.e. uv tool install "retrocast[viz]")

You're done!

Check data/5-results/mkt-cnv-160/my-new-model/report.md for your results!

Alternative: Quick Evaluation

Just want to score one file?

If you don't want to set up a full project structure, use the score-file command:

retrocast score-file \
  --benchmark data/1-benchmarks/definitions/mkt-cnv-160.json.gz \
  --routes my_predictions.json.gz \
  --stock data/1-benchmarks/stocks/buyables-stock.txt \
  --output scores.json.gz \
  --model-name "Quick-Check"

This skips the config setup and directly evaluates a single predictions file.

Next Steps

Learn the Concepts
Read Concepts to understand why we use adapters and manifests.

Use the Python API
Want to use RetroCast inside your own scripts? See the Library Guide.

Write Custom Adapters
Need to support a new output format? Learn how to write an Adapter.

Full CLI Reference
See all available commands in the CLI Reference.

Explore Benchmarks
Learn about stratified evaluation sets in the Benchmarks Guide.