Add support for serial ngen jobs#301
Merged
robertbartel merged 15 commits intoNOAA-OWP:masterfrom Mar 22, 2023
Merged
Conversation
Updating Dockerfiles for data-service and py-sources to explicitly use "latest" tag in FROM statements building off another DMOD internal Docker image.
- Add properties for whether serial or parallel exec was requested - Have data requirements and lazy init of partition config itself consider whether partition config is necessary, which depends on if parallel execution was requested
Updating Launcher class to account for serial ngen execution by not passing any partitioning config dataset when there is only 1 cpu.
Updating manager logic to have single-cpu (i.e., serial exec) jobs be moved to AWAITING_ALLOCATION without a partition config being generated if ever in the AWAITING_PARTITIONING step.
Updating service to move to AWAITING_ALLOCATION step instead of AWAITING_PARTITIONING step after data checks are successful, if serial execution is indicated by a job cpu count of 1.
Updating to new dmod.communication (0.11.0) and dmod.scheduler (0.10.0).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ngenDocker image entrypointNote that this supersedes #278. It is essentially the same change set rebased onto the current master commit.