Benchmarks

Quick Start

# Quick smoke test (TRG, small size, 1 trial)
python -m benchmarks.run --backend cpu --algorithm trg --size small --trials 1

# Full CPU baseline
python -m benchmarks.run --backend cpu -o benchmarks/results/cpu_baseline.json

# GPU comparison
python -m benchmarks.run --backend cuda -o benchmarks/results/cuda.json

# Show available backends
python -m benchmarks.run --list-backends

CLI Reference

python -m benchmarks.run [OPTIONS]

Algorithm Sizes

Each algorithm defines three problem sizes. Larger sizes stress-test bond dimension scaling and hardware throughput.

Supported Backends

Check what’s available on your machine:

python -m benchmarks.run --list-backends

Output Formats

JSON

Full structured output with timings, parameters, and device info:

python -m benchmarks.run -b cpu -a dmrg -s medium -n 3 -o results.json

CSV

Flat table for analysis in pandas, Excel, or plotting tools:

python -m benchmarks.run -b cpu -a all -s all --csv results.csv

Example Experiments

Is my GPU helping?

python -m benchmarks.run -b cpu -a dmrg -s medium -n 3 -o cpu.json
python -m benchmarks.run -b cuda -a dmrg -s medium -n 3 -o gpu.json

GPU typically helps when chi >= 64. For smaller bond dimensions, CPU may be faster due to transfer overhead.

Algorithm scaling

Fix the backend and vary size to study computational scaling:

python -m benchmarks.run -b cpu -a dmrg -s small medium large -n 5 --csv dmrg_scaling.csv

Tips

• Report median time, not mean — avoids JIT warmup skew from the first trial.
• GPU memory — large sizes may OOM on GPUs with limited VRAM.
• First trial is slow — JAX recompiles on first call. Use --trials 5 or more for stable numbers.

Chi ramping

CTMConfig.chi_ramp runs CTM convergence in stages at increasing chi. It is deprecated (#512) in favor of in-CTM chi growth (ctmrg_heuristic_increase_chi=True with chi_max). Benchmarks of chi_ramp showed 1.2–2.1x speedup on GPU with identical energies: