Flux as Resource Scheduler #1647
Description
I was involved today in reviewing the Flux Framework inclusion into HPSF (hpsfoundation/tac#61).
Flux provides very flexible resource management that can be used under slurm, standalone, nested on itself or Kubernetes. Looking at our current scheduling options (base and MPI), I wonder if libEnsemble would benefit implementing resource management and scheduling against the modular and composable Flux Python APIs?
Resources:
- Homepage
- Quickstart
- Launching Work in a Flux Session
- Tutorial that dives deeper and deeper
- Tutorial on starting flux in a container (this currently is tricky/not possible for the libEnsemble MPI Executor, as it depends on srun/mpiexec started from inside a container for sim jobs)