Skip to content

depend on libnvjitlink-dev at build time unconditionally#5479

Merged
rapids-bot[bot] merged 5 commits intorapidsai:mainfrom
jameslamb:fix/libnvjitlink-dependencies
Mar 31, 2026
Merged

depend on libnvjitlink-dev at build time unconditionally#5479
rapids-bot[bot] merged 5 commits intorapidsai:mainfrom
jameslamb:fix/libnvjitlink-dependencies

Conversation

@jameslamb
Copy link
Copy Markdown
Member

@jameslamb jameslamb commented Mar 30, 2026

rapidsai/cuvs#1923 switched cuVS to always enabling JIT+LTO, and therefore requiring nvJitLink at build and runtime.

Previously, it had only been required for CUDA 13.

cuGraph pulls cuVS via CPM and builds it from source, but wasn't updated to match, resulting in CUDA 12 builds here failing like this:

 │ │ CMake Error at build/_deps/cuvs-src/cpp/CMakeLists.txt:754 (target_link_libraries):
 │ │   Target "cuvs" links to:
 │ │     CUDA::nvJitLink
 │ │   but the target was not found.  Possible reasons include:
 │ │     * There is a typo in the target name.
 │ │     * A find_package call is missing for an IMPORTED target.
 │ │     * An ALIAS target is missing.
 │ │ CMake Error at build/_deps/cuvs-src/cpp/CMakeLists.txt:812 (target_link_libraries):
 │ │   Target "cuvs_static" links to:
 │ │     CUDA::nvJitLink
 │ │   but the target was not found.  Possible reasons include:
 │ │     * There is a typo in the target name.
 │ │     * A find_package call is missing for an IMPORTED target.
 │ │     * An ALIAS target is missing.

(example build link)

This resolves that, by removing "if CUDA 13" types of guards around this project's libnvJitLink dependency.

Notes for Reviewers

Some builds may still fail here

Like this:

  /__w/cugraph/cugraph/python/libcugraph/build/py3-none-linux_aarch64/_deps/cccl-src/lib/cmake/cub/../../../cub/cub/device/dispatch/kernels/kernel_scan_warpspeed.cuh(455): error: more than one instance of function template "cub::detail::scan::warpReduce" matches the argument list:
              function template "T raft::warpReduce(T, ReduceLambda)" (declared at line 49 of /pyenv/versions/3.14.3/lib/python3.14/site-packages/libraft/include/raft/util/reduction.cuh)
              function template "Tp cub::detail::scan::warpReduce(Tp, ScanOpT &)" (declared at line 247)
              argument types are: (unsigned int, raft::add_op)
     regWarpSum = warpReduce(regThreadSum, scan_op);

Until rapidsai/cuvs#1954 is resolved, which @viclafargue is working on in rapidsai/cuvs#1963

@jameslamb jameslamb requested a review from a team as a code owner March 30, 2026 21:36
@jameslamb jameslamb added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Mar 30, 2026
rapids-bot bot pushed a commit to rapidsai/shared-workflows that referenced this pull request Mar 30, 2026
Proposes allowing all `wheels-build` jobs to run to completion, instead of immediately cancelling all of them when 1 fails.

This should help with narrowing down issues, like "only fails on arm64".

## Motivation

Noticed while working on rapidsai/cugraph#5479, where @ChuckHastings and I were investigating an issue that only occurred on CUDA 12 builds. Noticed that all `conda-cpp-build` jobs ran to completion, but `wheels-build` jobs were cancelled after the first failure.

<img width="1837" height="733" alt="image" src="https://github.com/user-attachments/assets/70459440-e8b0-4a16-b955-b1a22452d11e" />

## Notes for Reviewers

This change will increase CI runner usage, but I think it's worth it:

* CPU-only runners used for builds are generally cheap and plentiful
* failed wheel build jobs are valuable:
  - they provide information to speed up debugging (e.g. "only fails on arm64")
  - they partially fill up the shared build caches and package caches, which other CI jobs benefit from and which might reduce the time-to-resolution for issues

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)
  - Bradley Dice (https://github.com/bdice)

URL: #517
@jameslamb
Copy link
Copy Markdown
Member Author

Now that rapidsai/cuvs#1963 is merged, pushed an empty commit to re-run all CI. Hopefully we will see builds pass here now.

@jameslamb
Copy link
Copy Markdown
Member Author

/merge

@jameslamb
Copy link
Copy Markdown
Member Author

All wheel tests are failing like this:

ImportError while loading conftest '/__w/cugraph/cugraph/python/pylibcugraph/pylibcugraph/tests/conftest.py'.
python/pylibcugraph/pylibcugraph/tests/conftest.py:10: in <module>
    from pylibcugraph.testing import utils
/pyenv/versions/3.11.15/lib/python3.11/site-packages/pylibcugraph/__init__.py:15: in <module>
    import pylibcugraph.comms
/pyenv/versions/3.11.15/lib/python3.11/site-packages/pylibcugraph/comms/__init__.py:4: in <module>
    from .comms_wrapper import init_subcomms
E   ImportError: libcugraph.so: cannot open shared object file: No such file or directory

(build link)

Just seeing that it's in .comms, I strongly suspect this is because there haven't been new nightly libraft / raft-dask wheels published in a while, and there have been ABI breaks in RMM since the last time they were published.

I'll go get some new raft wheels published.

@jameslamb
Copy link
Copy Markdown
Member Author

Once rapidsai/raft#2991 is merged (or any other RAFT PR, if another gets there sooner) and there are new nightly packages, I'll restart CI here.

rapids-bot bot pushed a commit to rapidsai/raft that referenced this pull request Mar 31, 2026
To adapt to recent changes in other RAPIDS projects, we need to produce new nightly wheels from this project (rapidsai/cugraph#5479 (comment)).

Today, that requires merging a new commit (rapidsai/build-planning#218 documents why). Given that we need that anyway, this at least tries to make it a useful one... bumping a few `pre-commit` hook versions and consolidating duplicate uses of `pre-commit/pre-commit-hooks`.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #2991
@jameslamb
Copy link
Copy Markdown
Member Author

New RAFT build finished (https://github.com/rapidsai/raft/actions/runs/23807692461/job/69387358843), just pushed a commit restarting CI here. This PR is already queued to auto-merge, so if CI passes it'll merge in and things should be working in this repo again.

@rapids-bot rapids-bot bot merged commit c520457 into rapidsai:main Mar 31, 2026
80 checks passed
@jameslamb jameslamb deleted the fix/libnvjitlink-dependencies branch March 31, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants