IGNORE TESTING ONLY Feature/update to spack v1p0p0 test cherry pick#564
Closed
climbfuji wants to merge 1045 commits intoJCSDA:feature/update_to_spack_v1from
Closed
IGNORE TESTING ONLY Feature/update to spack v1p0p0 test cherry pick#564climbfuji wants to merge 1045 commits intoJCSDA:feature/update_to_spack_v1from
climbfuji wants to merge 1045 commits intoJCSDA:feature/update_to_spack_v1from
Conversation
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* Refactor OSError handling in filesystem.py for Python 3 I changed `e.winerror` to `e.errno` in `lib/spack/llnl/util/filesystem.py` for Python 3 compatibility. This change specifically addresses an OSError handling block for Windows where `e.winerror == 5` (ACCESS_DENIED) was checked. The code now uses `e.errno == errno.EACCES` which is the standard way to check for this error in Python 3, while preserving the Windows-specific logic. I ran the existing tests, and they passed, confirming no regressions were introduced by this change. * Refactor OSError handling in filesystem.py for Python 3 (round 2) I simplified a Windows-specific OSError handling case in `lib/spack/llnl/util/filesystem.py` during symlink creation. The check for `e.winerror == 183` (Windows-specific "file exists") was removed in favor of the standard `e.errno == errno.EEXIST`, as Python 3 correctly maps this error code on Windows. This change further removes Python 2 era checks and relies on Python 3's improved error handling consistency across platforms. I ran the existing tests and they passed, confirming no regressions were introduced by this change. * Refactor loop in traverse.py to use `yield from` I replaced a `for item in iterable: yield item` loop with the more idiomatic `yield from iterable` in the `traverse_breadth_first_tree_edges` function within `lib/spack/spack/traverse.py`. This is a standard Python 3.3+ improvement for delegating to a sub-generator. Spack's minimum Python version is 3.6, making this change appropriate and safe. I also updated the associated comment. * I've removed some unnecessary comments from the code. Specifically, I took out the recently added explanatory comments in: - lib/spack/spack/test/llnl/util/lock.py (regarding Barrier exceptions) - lib/spack/spack/traverse.py (regarding `yield from` introduction) You indicated these comments weren't needed. The underlying code functionality, which was refactored previously, remains unchanged. * Remove useless statement * [@spackbot] updating style on behalf of haampie --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: haampie <haampie@users.noreply.github.com>
`spack pkg grep` can construct command lines that are too long for Windows, i.e. command lines that are longer than 32768 characters. This makes `spack pkg grep` respect the Windows limit by default, and gets the unix limit from `sysconfig`. - [x] Add a new `spack.cmd.group_arguments` function to create CLI-safe arg groups - [x] Default to max 500 elements or 32768 chars, whichever comes first - [x] If sysconfig is available, get `SC_ARG_MAX` and use that for max chars - [x] Make `Executable` always return an error if it's not successful - [x] Add test for `group_arguments` --------- Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* py-pycocotools: numpy 2.3 not yet supported * Update homepage * py-pycocotools: add v2.0.9
Adds support for Spack package repositories from external git repos.
```yaml
repos:
my_repo:
git: https://example.com/example/example.git
destination: ~/example # optional
paths: # optional
- subdir/spack_repo/example/x
- subdir/spack_repo/example/y
```
If `destination` is not configured, Spack clones the git repo to `~/.spack/git_repos/{hash(repository)}`.
Package repositories can put a file `spack-repo-index.yaml` in their root with relative paths to roots of package repositories (i.e. directories that contain `repo.yaml`):
```yaml
repo_index:
paths:
- subdir/spack_repo/example/x
- subdir/spack_repo/example/y
```
so users don't have to put that under `paths` in config. The `spack-repo-index.yaml` is simply a list of paths under `repo_index`. The idea is to avoid duplicating data such as "namespace" already specified in `<git repo>/<repo path>/repo.yaml`, to avoid that there are two sources of truth that go out of sync.
Further, `paths` in user config takes precedence, which allows users to enable specific package repositories in case the git monorepo provides multiple.
Remote package repositories are cloned/initialized in:
* `spack.main`
* `spack repo add`
This is process safe due to the lock in `$SPACK_USER_CACHE_PATH/package-repository.lock`; only one process can clone at a time.
The `spack repo add` command has a few new flags and a new positional arg:
```
spack repo add [-h] [--name NAME] [--path PATH] [--scope ...] path_or_repo [destination]
```
The signature is similar to the familiar `git clone <repository> <directory>`.
The `path_or_repo` argument is detected as a remote git repo if it contains a `:` not preceded by a `/`, which is what git does as well. If in the future we would support package repositories other than local file paths and remote git repos, we can resolve ambiguities with a new flag `[--git | --path | --<other-type>]`, but this is currently unnecessary.
The positional `destination` argument allows users to pick their own clone path, and only applies in case of git repos.
The flag `--path` corresponds to `repos:<name>:paths` and can be repeated to select specific package repositories in a git monorepo, and is also required if the git repo does not provide a `spack-repo-index.yaml` file in its root. Spack will never scan for `repo.yaml` files recursively, it only relies on `spack-repo-index.yaml` and `--repo-path` for package repository roots inside a git repo.
The flag `--name` applies to the *config* name/key in `repos.yaml` under `repos:<name>`. This flag is optional in the common case of adding just one package repository: it is set to the package repository's namespace. In case of monorepos with multiple package repositories, the `--name` flag is required.
spack#50656) intel-oneapi-compilers added to optim_opts Transfer cflags, cxxflags from Spack to COPTS, CXXOPTS. --------- Co-authored-by: teabagk7 <teabagk7@users.noreply.github.com>
Using the -d flag leads to unreadable, and possibly truncated, output. See: - https://gitlab.spack.io/spack/spack/-/jobs/16901136 Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Move the functions responsible for retrieving buildcache manifests into the url_buildcache.py module, and generalize the API so that manifests for any type of buildcache component (specs, indices, keys, etc) can be fetched the same way.
spack.repo.PATH is constructed either in: 1. spack.main.setup_main_options (entrypoint of Spack) 2. or spack.repo.PATH.__getattr__ (singleton, in sub-processes that don't run main) The latter was using an old named constructor of RepoPath, which had regressed.
* Bump Trilinos/Kokkos dependency Signed-off-by: Samuel E. Browne <sebrown@sandia.gov> * Pin Kokkos/KokkosKernels more specifically for releases Signed-off-by: Samuel E. Browne <sebrown@sandia.gov> --------- Signed-off-by: Samuel E. Browne <sebrown@sandia.gov>
* set release channel for non nightly builds * style * use .satisfies Co-authored-by: Alec Scott <hi@alecbcs.com> --------- Co-authored-by: Alec Scott <hi@alecbcs.com>
* RepoSplit/tests: sync mock packages for build_systems.py * Switch to use of underscore in package directories
…ued variants section (spack#51079) The packaging guide should always refer to `spack.package` when possible, to prevent people from importing private Spack API. The only thing that is indirectly exposed from internals is the class `DisjointSetsOfValues`. I don't think that's terrible, just that we have to keep in mind that this is effectively also public API... * docs: fix reference spack.directives -> spack.package, and rewords mv variants * Ensure all xrefs point to spack.package in packaging_*.rst --------- Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Currently, differences in `gcc-runtime` or `glibc` packages are hidden by `spack diff`. This is an unintended artifact of how they are specially handled in the solver. This PR adds an option to `SpecSolverSetup.spec_clauses` to include runtime specs that are not unified in the spec. The `spack diff` command calls `spec_clauses` with the new option, to ensure those differences are represented in the `spack diff` output. Includes regression test. --------- Signed-off-by: Gregory Becker <becker33@llnl.gov> Co-authored-by: becker33 <becker33@users.noreply.github.com> Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
…stion" (spack#51086) "beg the question" and "raise the question" are not synonyms. "begging the question" means committing the fallacy of assuming the conclusion in the premises. Signed-off-by: Gregory Becker <becker33@llnl.gov> Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
* Fix spack#51054 Signed-off-by: Till Ehrengruber <till.ehrengruber@cscs.ch> * [@spackbot] updating style on behalf of tehrengruber Signed-off-by: Till Ehrengruber <till.ehrengruber@cscs.ch> --------- Signed-off-by: Till Ehrengruber <till.ehrengruber@cscs.ch> Co-authored-by: tehrengruber <tehrengruber@users.noreply.github.com> Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
* The use of tabs in the Spec Syntax section makes it impossible to `Ctrl+F` because it's hidden, and prevents linking to these sections as they don't appear in the outline. * Fix an issue where the packaging guide introduced single-valued variants incorrectly as multi-valued variants. * Improve the section about conditional variants * Document `self.spec.variants[...].value` for use in build configuration. Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: tldahlgren <dahlgren1@llnl.gov> Signed-off-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* git_fetch: sparse path stages aren't unique For two packages sharing the same commit plus sparse paths the mirror layout is overriding one another See spack#50699 Signed-off-by: psakiev <psakiev@sandia.gov> * Apply changes from @tjfulle Apply changes suggested by @tjfulle in spack#50699 Signed-off-by: psakiev <psakiev@sandia.gov> * Remove superfluous test lines Signed-off-by: psakiev <psakiev@sandia.gov> * Pathlib for git projection Signed-off-by: psakiev <psakiev@sandia.gov> * Update test comment Signed-off-by: psakiev <psakiev@sandia.gov> * Use hash for sparse-path list Signed-off-by: psakiev <psakiev@sandia.gov> * Use reproducible hash algorithm Signed-off-by: psakiev <psakiev@sandia.gov> --------- Signed-off-by: psakiev <psakiev@sandia.gov> Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
…ze (spack#51090) Signed-off-by: Alec Scott <scott112@llnl.gov> Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Alec Scott <scott112@llnl.gov> Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
) * Set the packages fetcher using the fetcher property setter. closes: spack#51107 Signed-off-by: Tim Fuller <tjfulle@sandia.gov> * Update lib/spack/spack/package_base.py Co-authored-by: Phil Sakievich <psakiev@sandia.gov> Signed-off-by: Tim Fuller <tjfulle@sandia.gov> --------- Signed-off-by: Tim Fuller <tjfulle@sandia.gov> Co-authored-by: Phil Sakievich <psakiev@sandia.gov> Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
If not specified, it is python (with errors surpressed) Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
GoogleBot is crawling pages with `?highlight=...` query params as separate pages. Avoid that by using ```html <link rel="canonical" href="<url without query params>" /> ``` Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Use of `::` at the end of a line or `.. code-block::` means "default language", which is Python. Fix cases where Python is the wrong language by using ``` .. code-block:: <lang> ``` Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
…ate_to_spack_v1p0p0
Speed up the concretizer, guided by the profiling results from the development version of clingo v6: - Use a positive fact `concrete(PackageNode)`, instead of `not build(PackageNode)` - Simplify construction of `condition_set` given the current set of rules Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Tracks unsupported targets explicitly to avoid clingo having to figure out a negative statement. This should improve solver performance. Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
… feature/update_to_spack_v1p0p0
In this way trying to solve more input specs is at higher priority than trying to satisfy strong preferences or requirements. Fixes a regression introduced by spack#44373 Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
…ts.yaml to use new script REMOVE_SPACK_STACK_MODS_FOR_CI.sh to revert spack-stack extensions
…/spack into feature/update_to_spack_v1p0p0
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.