Quick Start

This guide gets you from raw planner output to a RetroCast analysis report.

!!! tip "What you'll learn"

- Install RetroCast and inspect the data directory layout
- Place raw planner output where project-mode commands expect it
- Evaluate planner output from ingestion through analysis

1. Install

pip install retrocast

Verify installation:

retrocast --version

2. Check Project Paths

Project-mode commands use a structured data directory. Inspect the resolved layout before placing files:

retrocast config

By default, RetroCast uses data/retrocast/ with subdirectories for benchmarks, raw planner outputs, processed candidates, scored evaluations, and analysis reports. The directories are created as commands write artifacts.

!!! tip "Custom data directory"

You can customize the data directory location via:

- CLI flag: `retrocast --data-dir ./my-data <command>`
- Environment variable: `export RETROCAST_DATA_DIR=./my-data`
- Config file: Add `data_dir: ./my-data` to `retrocast-config.yaml`

Run `retrocast config` to see the resolved paths.

3. Choose An Adapter

Adapters cast planner-specific raw output into schema-2 Routes. List supported adapters:

retrocast list-adapters

For one-off runs, pass the adapter directly to ingest:

retrocast ingest --model my-new-model --dataset mkt-cnv-160 --adapter aizynthfinder

For repeatable raw-data folders, put a manifest.json next to the raw results file:

{
  "directives": {
    "adapter": "aizynthfinder",
    "raw_results_filename": "predictions.json.gz"
  }
}

If no filename is declared, project-mode ingest reads results.json.gz.

To see examples of runner scripts for different planners that we use for benchmarking, take a look at the ischemist/project-pandora repo.

4. The Workflow

The project-mode workflow is:

graph LR
    A[Place Raw Data<br/>2-raw/] --> B[Ingest<br/>adapt + collect]
    B --> C[Score<br/>validity + constraints]
    C --> D[Analyze<br/>metrics + report]

    B -.-> E[3-processed/]
    C -.-> F[4-scored/]
    D -.-> G[5-results/]

All paths are relative to your data directory.

Step A: Place Raw Data

Put your model's raw output file in 2-raw/:

<data-dir>/2-raw/<model-name>/<benchmark-name>/<filename>

Example:

mkdir -p data/retrocast/2-raw/my-new-model/mkt-cnv-160
cp results.json.gz data/retrocast/2-raw/my-new-model/mkt-cnv-160/

!!! info "Available benchmarks"

See [Benchmarks Guide](guides/benchmarks.md) for details on evaluation sets.

Step B: Ingest

ingest adapts raw planner output and collects the resulting rank-preserving Candidates onto benchmark targets.

By default, project-mode ingest reads results.json.gz. If your raw file uses a different name, add a manifest.json in the same directory with a raw_results_filename directive.

retrocast ingest --model my-new-model --dataset mkt-cnv-160 --adapter aizynthfinder

Output:

data/retrocast/3-processed/mkt-cnv-160/my-new-model/candidates.json.gz

Step C: Score

score applies Tier-N validity checks and task constraints, producing an Evaluation.

retrocast score --model my-new-model --dataset mkt-cnv-160

Output:

data/retrocast/4-scored/mkt-cnv-160/my-new-model/<stock>/evaluation.json.gz

Step D: Analyze

analyze summarizes the evaluation into Solv-N rates, MRR@Solv-N, confidence intervals, and acceptable-route reconstruction metrics when available.

retrocast analyze --model my-new-model --dataset mkt-cnv-160

Outputs:

data/retrocast/5-results/mkt-cnv-160/my-new-model/<stock>/analysis.json.gz
data/retrocast/5-results/mkt-cnv-160/my-new-model/<stock>/report.md

!!! success "You're done"

Open the generated `report.md` for the benchmark summary.

Alternative: Explicit Files

If you do not want to use the project directory layout, run the explicit-file commands:

retrocast adapt \
  --adapter paroutes \
  --input raw.json.gz \
  --output candidates.json.gz

retrocast collect \
  --input candidates.json.gz \
  --benchmark benchmark.json.gz \
  --output collected.json.gz

retrocast score-file \
  --benchmark benchmark.json.gz \
  --candidates collected.json.gz \
  --stock buyables-stock.txt \
  --output evaluation.json.gz

This is useful for small experiments, notebooks, or custom pipelines where you do not want RetroCast to manage 2-raw through 5-results.

Choose A Path

RetroCast has three common entry points depending on what you are trying to do:

Use adapt for inspection. Use ingest for evaluation because it preserves every ranked prediction slot, including typed failures.

Library API

import json

import retrocast

task = json.loads(benchmark_path.read_text())
predictions = retrocast.ingest(raw_payload, "paroutes", task, workers=12)

let adapter = retrocast_core::adapters::built_in("paroutes")
    .expect("built-in adapter");
let predictions = retrocast_core::adapt::ingest(
    raw_payload,
    adapter.as_ref(),
    &task,
    retrocast_core::route::AdaptMode::Strict,
    None,
    12,
)?;

from retrocast import get_adapter
from retrocast.io import load_benchmark
from retrocast.workflow import ingest_candidates

task = load_benchmark(benchmark_path)
adapter = get_adapter("paroutes")
predictions = ingest_candidates(raw_payload, adapter, task)

Python receives an opaque Rust-owned value. Write or inspect it before passing it to score, which consumes the prediction graph. Rust owns the typed Predictions value directly.

Next Steps

Learn the Concepts

Read Concepts to understand the schema-2 model and workflow.

Understand Schema Design

Read Schema Design for the deeper data-model rationale.

Use the Library API

Use the top-level Python functions or retrocast-core from Rust applications.

Write Custom Adapters

Need to support a new output format? See Writing a Custom Adapter.

Full CLI Reference

See all available commands in the CLI Reference.

Explore Benchmarks

Learn about evaluation sets in the Benchmarks Guide.

From isChemist: Structure precedes quantity.

Essays and software that make better scientific questions possible. Subscribe at ischemist.com/newsletter, or check service status if hosted RetroCast resources or SynthArena look off.