Simulation output contract (machine-readable)¶

Status: Active (2026-05-24)
Parent: sim-packages-algorithm-plan.md §6 · AL-15
Harness today: benchmarks/harness/verify.py, bench.py, md_external_oracle.py

All sim.* runs and tier-2 physics benches should emit structured, parser-friendly artifacts. Humans read plots and Studio UI; agents, CI, and dashboards read JSON (and CSV where ingest already exists).

Format choice: JSON vs JSONC vs others¶

Recommendation: Prefer plain JSON for anything CI or agents must consume. Use JSONC only inside publish.zip-style repro bundles where a scientist may annotate // basis: def2-svp. Provide scripts/validate-sim-summary.sh that parses JSON (and optionally JSONC via strip-json-comments or jq with a preprocess step).

Use YAML summaries only when explicitly requested (--summary-format yaml / LI_SIM_SUMMARY_FORMAT=yaml). CI gates should prefer JSON or minified JSON. Do not rely on stdout prose or multi‑MB .traj text for gates.

Schema: li_sim_summary_v1¶

One file per run (or per bench×lang verify):

Paths (same schema, different serializers):

Also: aggregate benchmarks/results/<benchmark_id>/summary.json when a single canonical run exists.

{
  "schema": "li_sim_summary_v1",
  "benchmark": "md_lennard_jones",
  "vertical_id": "md_lennard_jones",
  "workload_class": "stub",
  "lang": "cpp",
  "variant": "reference_native",
  "ok": true,
  "git_sha": "64e36ec",
  "cpu_model": "x86_64",
  "flags": "-O3 -march=native",
  "params_digest": "sha256:…",
  "metrics": {
    "energy_drift_rel": 1.2e-4,
    "checksum": "…",
    "wall_time_s": null
  },
  "invariants": {
    "energy_drift_ok": true
  },
  "artifacts": {
    "params": "benchmarks/tier2_physics/md_lennard_jones/params.toml",
    "tier_f": null,
    "tier_d": null
  },
  "updated": "2026-05-24T12:00:00Z"
}

Required keys (all domains)¶

metrics by domain (extend, don’t fork schema)¶

Perf timing may duplicate latest.csv but belongs in metrics.wall_time_s when written from the same harness invocation.

Tier R repro bundle layout¶

publish/
  manifest.json          # strict JSON index (machine)
  manifest.notes.jsonc   # optional human comments (not CI-gated)
  params.toml
  summaries/
    cpp.summary.json
    li.summary.json
  fields/                # optional Tier F
    heat_000.vtk

manifest.json lists paths + digests (SHA-256). Agents validate the bundle without parsing JSONC.

Adjustable detail (parameters + CLI + env)¶

CLI (harness / native benches)¶

# Tier-2 smokes → machine-readable summaries
python3 benchmarks/harness/verify.py --write-summary --output-detail summary
python3 benchmarks/harness/verify.py --write-summary --summary-format json_min
python3 benchmarks/harness/verify.py --write-summary --summary-format yaml --output-detail debug

./li-tests/tooling/sim_li_run_summary.sh
./scripts/validate-sim-summary.sh

Higher detail never removes Tier S JSON; it only adds optional paths.

Callable from other Li programs¶

Packages: import sim (types + ids) · import sim.scientific (MD/PDE/rigid smokes today).

import sim
import sim.scientific

def main() -> int
=
  var r: SimRunResult = run_simulation(vertical_md_lennard_jones(), output_detail_summary())
  if run_result_ok(r) != 1:
    return 1
  var a: SimRunResult = run_algo(algo_pde_heat_explicit_2d(), output_detail_fields())
  return 1 - run_result_ok(a)

(lic today: import both packages; use unqualified types and run_* from their defining package — no sim.scientific.run_simulation qualified calls yet.)

Future: sim.drug_design, sim.additive, chem export the same SimRunResult + write JSON via harness or sim.write_summary when path strings land in lic.

Composable smoke: li-tests/composable/import_sim_scientific_run.li.

Harness integration¶

1. Done: verify.py --write-summary → li_sim_summary_v1 per tier-2 smoke (json / json_min / yaml).
2. Done: sim_summary.py + sim-write-summary.py + sim_li_run_summary.sh for Li runs.
3. Done: benchmarks/competitive/algo_registry.json (126 ids); run_algo dispatches registry stubs.
4. Open: bench.py merge wall_time_s into summary.
5. Done: ./scripts/validate-sim-summary.sh — schema check (JSON + min JSON + YAML).
6. Open: SimResult JSON serialize from Li when string I/O exists in lic.

Anti-patterns¶