Benchmark Definition Reference (YAML)

The whole benchmark is defined in a YAML file. The following template shows all available options, their types, and whether they are required.

Do note that, if relative paths are used wherever a path is expected, they are relative to the location of the benchmark file itself.

This template is automatically generated from the application's source code and is always up-to-date.

For the full specification — including the rationale, semantics of each section, and design constraints — see the YAML specification design document.

benchmark_template.yaml

# Omnibenchmark YAML Template.
# Generated from Pydantic Benchmark model (omnibenchmark version: 0.6.0.post42+g832944cc1.d20260825)
# DO NOT EDIT THIS FILE
# required (string), Unique identifier
id: example_id

# optional (string), Human-readable name
name: example benchmark

# optional (string), Description of the entity
description: A more verbose description, for humans reading this

# required (string), Benchmark author
benchmarker: Your Name

# optional (array), List of benchmark authors
authors: ...

# required (string), Benchmark version
version: 1.0.0

# required (SoftwareBackendEnum), Software backend, options: apptainer, conda, docker, envmodules, host
software_backend: apptainer

# required (array), Available software environments
software_environments:
  - # Example item:
    # required (string), Unique identifier
    id: example_id

    # optional (string), Human-readable name
    name: example benchmark

    # optional (string), Description of the entity
    description: A more verbose description, for humans reading this

    # optional (Repository)
    repository:
      # required (string), Repository URL
      url: https://example.com

      # required (string), Commit hash
      commit: c0ffee4

      # optional (RepositoryType), Repository type, options: git
      type: git

      # optional (string), Named entrypoint key from the module's omnibenchmark.yaml. Defaults to 'default' when not specified.
      entrypoint: example_entrypoint


    # optional (string), Conda environment file path
    conda: example_conda

    # optional (string), Apptainer/Singularity image path
    apptainer: example_apptainer

    # optional (string), Docker image
    docker: example_docker

    # optional (string), Environment module name
    envmodule: example_envmodule

    # optional (string), EasyBuild config path
    easyconfig: example_easyconfig


# required (array), Benchmark stages
stages:
  - # Example item:
    # required (string), Unique identifier
    id: example_id

    # optional (string), Human-readable name
    name: example benchmark

    # optional (string), Description of the entity
    description: A more verbose description, for humans reading this

    # required (array), Modules in this stage
    modules:
      - # Example item:
        # optional (string), Software environment ID
        software_environment: example_software_environment

        # required (string), Unique identifier
        id: example_id

        # optional (string), Human-readable name
        name: example benchmark

        # optional (string), Description of the entity
        description: A more verbose description, for humans reading this

        # required (Repository), Module repository
        repository:
          # required (string), Repository URL
          url: https://example.com

          # required (string), Commit hash
          commit: c0ffee4

          # optional (RepositoryType), Repository type, options: git
          type: git

          # optional (string), Named entrypoint key from the module's omnibenchmark.yaml. Defaults to 'default' when not specified.
          entrypoint: example_entrypoint


        # optional (array), Module parameters
        parameters: ...

        # optional (array), Module IDs this module must not share an execution path with. Symmetric (declaring it on either module is equivalent), transitive (applies across non-adjacent stages), and OR over the list (a path is pruned if it contains this module together with any listed one). Non-module-id entries are silently ignored.
        exclude: ...

        # optional (array), Module outputs
        outputs: ...

        # optional (object), Lineage gate. Maps label name → required value: this module runs only on execution paths whose upstream lineage carries every label at exactly that value (string equality). Labels are advertised by a stage via `Stage.provides` and valued per node by `Module.provides`, defaulting to the producing module's id — so naming a module id here is the common case, not the contract. Downstream of a fan-in join the lineage is the union over every branch, so a gate may name a label carried by any of them; a gather cuts the chain and exposes only its group key. Non-matching paths prune at DAG-construction time. See docs/design/008-filtering.md §3.5.
        requires: ...

        # optional (object), Explicit benchmark-local label bindings for this module. Maps label name → value, for labels its stage advertises via `Stage.provides`. This is the recommended way to source a label value: it keeps routing out of the module's CLI parameter contract, so the same module stays reusable across benchmarks with different label vocabularies. When a label has no binding here, it defaults to the module id. Requires api_version ≥ 0.7.0.
        provides: ...

        # optional (Resources), Resource requirements (overrides stage-level resources)
        resources:
          # optional (integer), Number of CPU cores
          cores: 1

          # optional (integer), Memory in megabytes
          mem_mb: 1

          # optional (integer), Disk space in megabytes
          disk_mb: 1

          # optional (integer), Expected runtime in minutes
          runtime: 1

          # optional (integer), Number of GPUs
          gpu: 1


        # optional (array), Host capabilities this module needs (e.g. gpu, large_mem). The module is pruned unless every listed capability is passed at run time via --with-capability. Host facts only — not lineage labels.
        requires_capabilities: ...


    # optional (array), Stage inputs
    inputs: ...

    # required (array), Stage outputs
    outputs:
      - # Example item:
        # required (string), Unique identifier
        id: example_id

        # required (string), File path
        path: filename.txt


    # optional (array), Lineage labels this stage advertises to downstream modules. For each label, every node of this stage carries a value, taken from the module's `provides` binding if present, otherwise the module id. Downstream modules gate on these labels via `requires:`; non-matching execution paths prune at DAG-construction time. The builtin labels `name` and `dataset` are reserved and may not be declared here. Requires api_version ≥ 0.7.0.
    provides: ...

    # optional (Resources), Resource requirements for rules in this stage
    resources:
      # optional (integer), Number of CPU cores
      cores: 1

      # optional (integer), Memory in megabytes
      mem_mb: 1

      # optional (integer), Disk space in megabytes
      disk_mb: 1

      # optional (integer), Expected runtime in minutes
      runtime: 1

      # optional (integer), Number of GPUs
      gpu: 1


    # optional (array), Fan-in specs (design 010). Each collects every node producing a shared output id, grouped by a stage they descend from. A gather cuts the lineage chain, so `prefix` is required. Requires api_version ≥ 0.7.0.
    gather: ...

    # optional (string), Filesystem prefix for a gather stage's cut chain (design 010 §3.3). Required when `gather` is set; ignored otherwise.
    prefix: example_prefix


# optional (array), Metric collectors
metric_collectors: ...

# optional (Storage), Remote storage configuration
storage:
  # optional (StorageAPIEnum), Storage API type (currently only S3 is supported), options: S3
  api: S3

  # required (string), Storage endpoint URL
  endpoint: example_endpoint

  # optional (string), Storage bucket name (only needed if using S3)
  bucket_name: example-bucket


# optional (StorageAPIEnum), Storage API type (deprecated, use storage.api)
storage_api: S3

# optional (string), Storage bucket name (deprecated, use storage.bucket_name)
storage_bucket_name: example_storage_bucket_name

# optional (string), Benchmark YAML specification version (deprecated, use api_version)
benchmark_yaml_spec: 0.0

# optional (APIVersion), default: 0.4.0, API version, options: 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.0
api_version: 0.4.0

# optional (Provenance), Lineage metadata
provenance:
  # optional (string), Canonical URL where the authoritative version of this benchmark is published
  canonical_url: example_canonical_url

  # optional (string), Identifier or URL of the benchmark this one was forked/derived from
  derived_from: example_derived_from

  # optional (string), summary_hash() of the parent benchmark of which this is a subset
  subset_of: example_subset_of