Skip to content

Validation

Validation is split into scaffold checks that run without CMSSW and installed-runtime checks that require the completed build.

Scaffold checks

./scripts/test_scaffold.sh

These checks cover:

  • Bash syntax, JSON parsing, and Python compilation;
  • the exact release pins;
  • ignored generated, EOS lock, and local-only tool files;
  • clearing incompatible Python prefix variables;
  • persistent and session-local kernel registration;
  • preservation of the parent server's JUPYTER_PATH and JUPYTER_DATA_DIR;
  • staging a rendered kernel without activating CMSSW in the parent process.

The scaffold workflow runs on every push and pull request.

Installed-runtime checks

./scripts/check_setup.sh

The command activates the pinned environment and runs scripts/smoke_test.py. It verifies:

  1. CMSSW_VERSION is CMSSW_16_0_0;
  2. SCRAM_ARCH is el9_amd64_gcc13;
  3. the Combine checkout is exactly tagged v11.0.0;
  4. PyROOT and HiggsAnalysis.CombinedLimit.PhysicsModel import;
  5. combine, text2workspace.py, and combineTool.py execute;
  6. a temporary counting datacard completes a blind AsymptoticLimits run;
  7. the expected ROOT output exists and is non-empty.

Temporary inputs and outputs are removed after the check.

Reference SWAN execution

The repository's executed smoke-test notebook records a successful run on CERN SWAN on 1 August 2026.

It reports:

Check Recorded result
Python 3.9.14
CMSSW CMSSW_16_0_0
SCRAM architecture el9_amd64_gcc13
ROOT 6.36.07
Combine tag v11.0.0
Smoke-test output PASS

The blind counting-card result was:

Expected  2.5%: r < 0.7635
Expected 16.0%: r < 1.0483
Expected 50.0%: r < 1.5391
Expected 84.0%: r < 2.3059
Expected 97.5%: r < 3.3353

The notebook intentionally retains the reference user's resolved EOS and temporary paths. Those locations vary by user and run; they are evidence of the recorded execution, not paths that another installation must reproduce.

What green means

Validated scope

Green means installation, kernel isolation, imports, Combine utilities, and the repository's reference asymptotic-limit workflow passed in the pinned runtime.

Not validated by this smoke test

The check does not independently validate a user's physics model, datacard construction, nuisance correlations, systematic variations, minimizer behavior for another model, or statistical interpretation. Those remain analysis-specific responsibilities.