Skip to content

Alpaka / HIP Cleanup, main branch (2026.03.02.)#1271

Open
krasznaa wants to merge 3 commits intoacts-project:mainfrom
krasznaa:AlpakaHIPCleanup-main-20260302
Open

Alpaka / HIP Cleanup, main branch (2026.03.02.)#1271
krasznaa wants to merge 3 commits intoacts-project:mainfrom
krasznaa:AlpakaHIPCleanup-main-20260302

Conversation

@krasznaa
Copy link
Member

@krasznaa krasznaa commented Mar 2, 2026

This was triggered by @fwyzard pointing out in #1269 that our code is currently not able to use rocThrust from an already installed location. (Picking it up from the local HIP installation for instance.)

But while experimenting with the code, I realized that we should do some additional cleanup as well. So I changed the setup of the Alpaka unit test a little as well along the way. 🤔

@StewMH, how do you do builds with Alpaka+HIP usually? I has to realize that I only ever did such builds on Ubuntu so far, never on RHEL. So on RHEL I had to devise a build setup where:

  • The C/C++ compilers are set up to be amdclang / amdclang++ respectably;
  • (amd)Clang would be told to use GCC 13 for its C++ standard library.

Without this, if I let the host compiler be GCC 13, I was getting into some weird linking errors on traccc_test_alpaka. The kind of linking errors that I also get when mixing GCC + oneAPI in a single build tree. (Some of our templates are not represented in the ABI in the same way between GCC and Clang unfortunately.)

So, do you do the same @StewMH? I.e. do you need to use HIP's Clang as the host compiler in such builds as well?

krasznaa added 3 commits March 2, 2026 14:30
Similar to traccc_add_alpaka_library(...), it makes use of the
helper function provided by Alpaka (alpaka_add_executable(...))
for setting up the code build. Simplifying our own code a little
in the process.
The main thing being that an installed rocthrust version will
only provide the target roc::rocthrust. So we need to create
a target with that same name even when we provide rocthrust
as part of our own build.
While simplifying the configuration of the Alpaka unit tests
a little using traccc_add_alpaka_test(...).
@krasznaa krasznaa added build This relates to the build system hip Changes related to AMD HIP alpaka Changes related to Alpaka labels Mar 2, 2026
@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 2, 2026

@StewMH
Copy link
Contributor

StewMH commented Mar 2, 2026

I always use hipcc + llvm clang, whichever version matches the printout from hipcc -v.

@krasznaa
Copy link
Member Author

krasznaa commented Mar 2, 2026

I always use hipcc + llvm clang, whichever version matches the printout from hipcc -v.

Interesting. 🤔

You see, now that with the latest VecMem version we use CMake's own HIP language implementation, I rather made such a setup on RHEL9:

# Set up GCC.
GCCDIR=/software/gcc/13.3.0/x86_64-el9
source ${GCCDIR}/setup.sh

# Set up where to get HIP from.
export ROCM_PATH=/opt/rocm-6.0.3
export CMAKE_PREFIX_PATH=${ROCM_PATH}${CMAKE_PREFIX_PATH:+:${CMAKE_PREFIX_PATH}}

# Set up the compilers for CMake.
export CC="${ROCM_PATH}/bin/amdclang"
export CFLAGS="${CFLAGS:+${CFLAGS} }--gcc-toolchain=${GCCDIR}"
export CXX="${ROCM_PATH}/bin/amdclang++"
export CXXFLAGS="${CXXFLAGS:+${CXXFLAGS} }--gcc-toolchain=${GCCDIR}"
export HIPCXX="${ROCM_PATH}/bin/amdclang++"
export HIPFLAGS="${HIPFLAGS:+${HIPFLAGS} }--gcc-toolchain=${GCCDIR}"

Previously I was also of course using hipcc, as VecMem only supported using that for building HIP sources. But I was also always afraid of not being able to tell hipcc which GCC version it should tell the underlying Clang executable to use. 🤔

@StewMH
Copy link
Contributor

StewMH commented Mar 2, 2026

I think we expect alpaka to only work with a clang host compiler, given the compatibility matrix here:
https://github.com/alpaka-group/alpaka
but indeed it does seem like it should now (without the vecmem custom cmake code) be happy with amdclang as the device compiler. Locally that seems to work, although I failed to link with rocthrust, which I guess relates to this PR>

@fwyzard
Copy link

fwyzard commented Mar 2, 2026

In CMSSW we mix the g++ and hipcc:

  • host-only files are compiled by g++
  • device files (device-side stuff, as well as kernel launches, etc) are compiled by hipcc
  • the resulting binaries are linked together

Note that we have to tell hipcc both where to find the gcc libraries, and the target "triple" to use:

COMPILER_HOST=$(gcc -dumpmachine)
COMPILER_PATH=$(dirname $(realpath $(which gcc)))
hipcc --target=$COMPILER_HOST --gcc-toolchain=$COMPILER_PATH ...

(not really like this, but it gives you an idea)

fwyzard pushed a commit to cms-externals/traccc that referenced this pull request Mar 3, 2026
* Introduced traccc_add_alpaka_test(...).

  Similar to traccc_add_alpaka_library(...), it makes use of the
  helper function provided by Alpaka (alpaka_add_executable(...))
  for setting up the code build. Simplifying our own code a little
  in the process.

* Modified how rocthrust would be set up for the project.

  The main thing being that an installed rocthrust version will
  only provide the target roc::rocthrust. So we need to create
  a target with that same name even when we provide rocthrust
  as part of our own build.

* Migrated all clients to the roc::rocthrust target name.

  While simplifying the configuration of the Alpaka unit tests
  a little using traccc_add_alpaka_test(...).
@fwyzard
Copy link

fwyzard commented Mar 3, 2026

To confirm, this PR (as of 09d2fc2) works in the context of CMSSW.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

alpaka Changes related to Alpaka build This relates to the build system hip Changes related to AMD HIP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants