Skip to content

Conversation

@fwyzard
Copy link
Contributor

@fwyzard fwyzard commented Dec 13, 2025

This release implements a new interface, similar to std::transform, that simplifies writing asynchronous parallel algorithms across all back-ends. SYCL support is extended to NVIDIA and AMD GPUs. The release introduces unified memory and expands asynchronous memory allocation to buffers of any dimension. Interoperability with standard C++ is improved through std::span support: alpaka buffers expose a span interface, and any std::span can be used as an alpaka view. It adds compile-time warp-size definitions, extends atomic increment and decrement operations and fixes their behaviour on CPU back-end; it introduces a C++ concept for alpaka accelerators together with new type traits, along with many smaller fixes and improvements.
The CI has been updated to test newer operating systems and compilers, including Clang 20 and ROCm 6.3, 6.4, and 7.0.

The full list of changes is available in the ChangeLog.

@fwyzard
Copy link
Contributor Author

fwyzard commented Dec 13, 2025

enable gpu

@fwyzard
Copy link
Contributor Author

fwyzard commented Dec 13, 2025

please test

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @fwyzard for branch IB/CMSSW_16_0_X/master.

@akritkbehera, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Dec 13, 2025

cms-bot internal usage

@cmsbuild
Copy link
Contributor

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e0c53/49947/summary.html
COMMIT: c14809b
CMSSW: CMSSW_16_0_X_2025-12-12-2300/el8_amd64_gcc13
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10250/49947/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed External Build

I found compilation warning when building: See details on the summary page.

@fwyzard
Copy link
Contributor Author

fwyzard commented Dec 13, 2025

please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e0c53/49948/summary.html
COMMIT: c14809b
CMSSW: CMSSW_16_0_X_2025-12-12-2300/el8_amd64_gcc13
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10250/49948/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed Build

I found compilation error when building:

Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestAtomicPairCounterROCmAsync/libalpakaTestAtomicPairCounterROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestBufferROCmAsync/libalpakaTestBufferROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a to productstore area:
cp: cannot stat 'tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a': No such file or directory
>> Deleted: tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a
gmake: *** [config/SCRAM/GMake/Makefile.rules:1920: tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a] Error 1
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestIndependentKernelROCmAsync/libalpakaTestIndependentKernelROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelROCmAsync/libalpakaTestKernelROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneHistoContainerROCmAsync/libalpakaTestOneHistoContainerROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneRadixSortROCmAsync/libalpakaTestOneRadixSortROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneToManyAssocROCmAsync/libalpakaTestOneToManyAssocROCmAsync_rocm.a to productstore area:


@cmsbuild
Copy link
Contributor

Pull request #10250 was updated.

@fwyzard
Copy link
Contributor Author

fwyzard commented Dec 13, 2025

please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e0c53/49963/summary.html
COMMIT: 8c4996c
CMSSW: CMSSW_16_0_X_2025-12-13-1100/el8_amd64_gcc13
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10250/49963/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed Build

I found compilation error when building:

Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestBufferROCmAsync/libalpakaTestBufferROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestIndependentKernelROCmAsync/libalpakaTestIndependentKernelROCmAsync_rocm.a to productstore area:
cp: cannot stat 'tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a': No such file or directory
>> Deleted: tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a
gmake: *** [config/SCRAM/GMake/Makefile.rules:1920: tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestHistoContainerROCmAsync/libalpakaTestHistoContainerROCmAsync_rocm.a] Error 1
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestKernelROCmAsync/libalpakaTestKernelROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneHistoContainerROCmAsync/libalpakaTestOneHistoContainerROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneRadixSortROCmAsync/libalpakaTestOneRadixSortROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestOneToManyAssocROCmAsync/libalpakaTestOneToManyAssocROCmAsync_rocm.a to productstore area:
Copying tmp/el8_amd64_gcc13/src/HeterogeneousCore/AlpakaInterface/test/alpakaTestPrefixScanROCmAsync/libalpakaTestPrefixScanROCmAsync_rocm.a to productstore area:


@fwyzard
Copy link
Contributor Author

fwyzard commented Dec 14, 2025

please test with #10238

@smuzaffar
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 6, 2026

-1

Failed Tests: nvidia_h100UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e0c53/50446/summary.html
COMMIT: e3c6e9d
CMSSW: CMSSW_16_1_X_2026-01-05-2300/el8_amd64_gcc13
Additional Tests: GPU,NVIDIA_H100
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10250/50446/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e0c53/50446/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e0c53/50446/git-merge-result

Comparison Summary

Summary:

  • You potentially added 17 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 17 differences found in the comparisons
  • Reco comparison had 4 failed jobs
  • DQMHistoTests: Total files compared: 53
  • DQMHistoTests: Total histograms compared: 4280544
  • DQMHistoTests: Total failures: 137
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4280387
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.002 KiB( 52 files compared)
  • DQMHistoSizes: changed ( 18434.0,... ): -0.001 KiB HLT/Filters
  • Checked 227 log files, 198 edm output root files, 53 DQM output files
  • TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

  • You potentially added 6 lines to the logs
  • Reco comparison results: 234 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 38773
  • DQMHistoTests: Total nulls: 5
  • DQMHistoTests: Total successes: 110593
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

@smuzaffar
Copy link
Contributor

test parameters:

  • enable = gpu

@smuzaffar
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2026

-1

Failed Tests: nvidia_h100UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e0c53/50456/summary.html
COMMIT: e3c6e9d
CMSSW: CMSSW_16_1_X_2026-01-06-1100/el8_amd64_gcc13
Additional Tests: GPU,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10250/50456/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 9 differences found in the comparisons
  • Reco comparison had 4 failed jobs
  • DQMHistoTests: Total files compared: 53
  • DQMHistoTests: Total histograms compared: 4280544
  • DQMHistoTests: Total failures: 117
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4280407
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.002 KiB( 52 files compared)
  • DQMHistoSizes: changed ( 18434.0,... ): -0.001 KiB HLT/Filters
  • Checked 227 log files, 198 edm output root files, 53 DQM output files
  • TriggerResults: no differences found

AMD_W7900 Comparison Summary

Summary:

  • You potentially added 13 lines to the logs
  • Reco comparison results: 255 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 30052
  • DQMHistoTests: Total nulls: 12
  • DQMHistoTests: Total successes: 119307
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

  • You potentially added 16 lines to the logs
  • Reco comparison results: 240 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 26949
  • DQMHistoTests: Total nulls: 8
  • DQMHistoTests: Total successes: 122414
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

  • You potentially added 1 lines to the logs
  • Reco comparison results: 257 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 27362
  • DQMHistoTests: Total nulls: 11
  • DQMHistoTests: Total successes: 121998
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

@smuzaffar
Copy link
Contributor

+externals

@smuzaffar smuzaffar merged commit b02fdba into cms-sw:IB/CMSSW_16_1_X/master Jan 7, 2026
45 of 47 checks passed
@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 7, 2026

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_16_1_X/master IBs (test failures were overridden). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @sextonkennedy, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

@smuzaffar
Copy link
Contributor

@fwyzard , we are getting many warnings [a] in CMSSW after this was merged. Can you please look in to this ? If it is easy to fix these warnings then we can wait otherwise I suggest that we revert this change and wait for a CMSSW PR to go with it

[a] https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/el8_amd64_gcc13/CMSSW_16_1_X_2026-01-07-2300/HeterogeneousCore/AlpakaInterface

src/HeterogeneousCore/AlpakaInterface/test/alpaka/testIndependentKernel.dev.cc:133:26:   required from here
  [src/HeterogeneousCore/AlpakaInterface/interface/workdivision.h:26](https://github.com/cms-sw/cmssw/blob/CMSSW_16_1_X_2026-01-07-2300/HeterogeneousCore/AlpakaInterface/interface/workdivision.h#L26):58: warning: 'alpaka::isAccelerator<alpaka::AccGpuUniformCudaHipRt<alpaka::ApiCudaRt, std::integral_constant<long unsigned int, 1>, unsigned int> >' is deprecated: use the alpaka::concepts::Acc instead. [-Wdeprecated-declarations]
    26 |   template <typename TAcc, typename = std::enable_if_t<alpaka::isAccelerator<TAcc>>>
      |                                                  ~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc13/external/alpaka/2.1.1-3caaac8d71f39d400ab2511b2403675a/include/alpaka/acc/Traits.hpp:35:80: note: declared here
   35 |     [[deprecated("use the alpaka::concepts::Acc instead.")]] inline constexpr bool isAccelerator = concepts::Acc<TAcc>;
      |                                                                                ^~~~~~~~~~~~~
src/HeterogeneousCore/AlpakaInterface/interface/workdivision.h: In instantiation of 'struct cms::alpakatools::requires_single_thread_per_block<alpaka::AccGpuUniformCudaHipRt<alpaka::ApiCudaRt, std::integral_constant<long unsigned int, 1>, unsigned int>, void>':
src/HeterogeneousCore/AlpakaInterface/interface/workdivision.h:46:81:   required from 'constexpr const bool cms::alpakatools::requires_single_thread_per_block_v<alpaka::AccGpuUniformCudaHipRt<alpaka::ApiCudaRt, std::integral_constant<long unsigned int, 1>, unsigned int>, void>'
src/HeterogeneousCore/AlpakaInterface/interface/workdivision.h:51:16:   required from 'alpaka_common::WorkDiv<std::integral_constant<long unsigned int, 1> > cms::alpakatools::make_workdiv(alpaka_common::Idx, alpaka_common::Idx) [with TAcc = alpaka::AccGpuUniformCudaHipRt<alpaka::ApiCudaRt, std::integral_constant<long unsigned int, 1>, unsigned int>; <template-parameter-1-2> = void; alpaka_common::WorkDiv<std::integral_constant<long unsigned int, 1> > = alpaka::WorkDivMembers<std::integral_constant<long unsigned int, 1>, unsigned int>; alpaka_common::Idx = unsigned int]'
src/HeterogeneousCore/AlpakaInterface/test/alpaka/testIndependentKernel.dev.cc:97:153:   required from 'void testIndependentWorkKernel(size_t, size_t, size_t, TKernel) [with TKernel = IndependentWorkKernel; size_t = long unsigned int]'
src/HeterogeneousCore/AlpakaInterface/test/alpaka/testIndependentKernel.dev.cc:133:26:   required from here
  [src/HeterogeneousCore/AlpakaInterface/interface/workdivision.h:31](https://github.com/cms-sw/cmssw/blob/CMSSW_16_1_X_2026-01-07-2300/HeterogeneousCore/AlpakaInterface/interface/workdivision.h#L31):108: warning: 'alpaka::isAccelerator<alpaka::AccGpuUniformCudaHipRt<alpaka::ApiCudaRt, std::integral_constant<long unsigned int, 1>, unsigned int> >' is deprecated: use the alpaka::concepts::Acc instead. [-Wdeprecated-declarations]
    31 |   struct requires_single_thread_per_block<alpaka::AccGpuCudaRt<TDim, Idx>> : public std::false_type {};
      |                                                                                                    ~~~~    ^                                                     
/data/cmsbld/jenkins/workspace/build-any-ib/w/el8_amd64_gcc13/external/alpaka/2.1.1-3caaac8d71f39d400ab2511b2403675a/include/alpaka/acc/Traits.hpp:35:80: note: declared here
   35 |     [[deprecated("use the alpaka::concepts::Acc instead.")]] inline constexpr bool isAccelerator = concepts::Acc<TAcc>;
      |                                                                                ^~~~~~~~~~~~~

@fwyzard
Copy link
Contributor Author

fwyzard commented Jan 8, 2026

I'll have a look.

@smuzaffar
Copy link
Contributor

all these warnings are coming from following cmssw sources

     17 HeterogeneousCore/AlpakaInterface/interface/atomicMaxF.h
      7 HeterogeneousCore/AlpakaInterface/interface/atomicMaxPair.h
     16 HeterogeneousCore/AlpakaInterface/interface/OneToManyAssoc.h
    210 HeterogeneousCore/AlpakaInterface/interface/prefixScan.h
   5623 HeterogeneousCore/AlpakaInterface/interface/workdivision.h
      7 PhysicsTools/PyTorchAlpaka/test/alpaka/testSoAToTorch.dev.cc

@fwyzard
Copy link
Contributor Author

fwyzard commented Jan 8, 2026

cms-sw/cmssw#49748 should fix the warnings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants