[WIP] Benchpark Without Subprocess

[michaelmckinsey1]

Description

• This changes the execution of benchpark from a subprocess executing python main.py ... to python benchpark ..., which results in clearer usage statements.
• Depends on Print Usage When No Command Provided #895

Original goal is to execute from bin/benchpark such that usage statements are benchpark ... instead of main.py ...

$ benchpark list
usage: main.py list [-h] {benchmarks,experiments,systems,modifiers} ...
main.py list: error: the following arguments are required: list_subcommand
Traceback (most recent call last):
  File "/Users/mckinsey1/Documents/repos/benchpark/bin/benchpark", line 23, in <module>
    main()
  File "/Users/mckinsey1/Documents/repos/benchpark/bin/benchpark", line 19, in main
    subprocess.run([sys.executable, main_py] + sys.argv[1:], check=True)
  File "/opt/anaconda3/envs/bp-3.11.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/opt/anaconda3/envs/bp-3.11.11/bin/python3', PosixPath('/Users/mckinsey1/Documents/repos/benchpark/lib/main.py'), 'list']' returned non-zero exit status 2.

$ benchpark list
usage: benchpark list [-h] {benchmarks,experiments,systems,modifiers} ...
benchpark list: error: the following arguments are required: list_subcommand

Adding/modifying a system (docs: Adding a System)

• Add/modify systems/system_name/system.py file
• Add/modify a dry run unit test for system_name in .github/workflows/run.yml
• Add/modify systems/all_hardware_descriptions/hardware_name/hardware_description.yaml which will appear in the docs catalogue

Adding/modifying a benchmark (docs: Adding a Benchmark)

• If modifying the source code of a benchmark: create, self-assign, and link here a follow up issue with a link to the PR in the benchmark repo.
• If package.py upstreamed to Spack is insufficient, add/modify repo/benchmark_name/package.py plus: create, self-assign, and link here a follow up issue with a link to the PR in the Spack repo.
• If application.py upstreamed to Ramble is insufficient, add/modify repo/benchmark_name/application.py plus: create, self-assign, and link here a follow up issue with a link to the PR in the Ramble repo.
• Tags in Ramble's application.py or in repo/benchmark_name/application.py will appear in the docs catalogue
• Add/modify an experiments/benchmark_name/experiment.py to define a single node and multi-node experiments
• Add/modify a dry run unit test in .github/workflows/run.yml

Adding/modifying core functionality, CI, or documentation:

• Update docs
• Update .github/workflows and .gitlab/tests unit tests (if needed)

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.48%. Comparing base (5ac8be9) to head (5bc3ec5).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop     #896   +/-   ##
========================================
  Coverage    41.48%   41.48%           
========================================
  Files           22       22           
  Lines         1509     1509           
  Branches       213      213           
========================================
  Hits           626      626           
  Misses         838      838           
  Partials        45       45           

Flag	Coverage Δ
unittests	41.48% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.