Skip to content

Fix linux aggressively drop symbols #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chengcli merged 2 commits into from
Feb 5, 2026
Merged

Fix linux aggressively drop symbols #71

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a2cb4b1
wip
Feb 5, 2026
a153c5a
wip
chengcli Feb 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions cmake/macros/macro_setup_problem.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,8 @@ macro(setup_problem namel)
${namel}.${buildl}
PRIVATE kintera::kintera
$<IF:$<BOOL:${CUDAToolkit_FOUND}>,kintera::kintera_cu,>)

if (UNIX AND NOT APPLE)
target_link_options(${namel}.${buildl} PRIVATE -Wl,--no-as-needed)
endif()
endmacro()
6 changes: 5 additions & 1 deletion cmake/macros/macro_setup_test.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,13 @@ macro(setup_test namel)

target_link_libraries(
${namel}.${buildl}
PRIVATE kintera::kintera
PRIVATE kintera::kintera
$<IF:$<BOOL:${CUDAToolkit_FOUND}>,kintera::kintera_cu,>
gtest_main)

if (UNIX AND NOT APPLE)
target_link_options(${namel}.${buildl} PRIVATE -Wl,--no-as-needed)
endif()
Comment on lines +26 to +28
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --no-as-needed linker flag is being added to setup_test to prevent Linux from aggressively dropping symbols. However, macro_setup_problem.cmake has a similar structure and links the same libraries (kintera::kintera and optionally kintera::kintera_cu) but does not have this flag. For consistency and to prevent similar symbol dropping issues in problem executables, consider adding the same linker flag to setup_problem as well.

Copilot uses AI. Check for mistakes.

add_test(NAME ${namel}.${buildl} COMMAND ${namel}.${buildl})
endmacro()
42 changes: 19 additions & 23 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,15 @@ def parse_library_names(libdir):
library_names.extend(['netcdf'])

# move current library name to first
current = [item for item in library_names if item.startswith('kintera')]
other = [item for item in library_names if not item.startswith('kintera')]
return current + other
#current = [item for item in library_names if item.startswith('kintera')]
#other = [item for item in library_names if not item.startswith('kintera')]
# 1) non-cuda libs first (consumers)
kintera_non_cuda = [l for l in library_names if l.startswith("kintera") and "cuda" not in l]
# 2) cuda libs last (providers)
Comment on lines +26 to +28
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on line 26 describes kintera non-CUDA libraries as "consumers," but this is potentially misleading. In typical linking scenarios, consumers depend on providers, not the other way around. If kintera_cuda libraries are "providers" (line 28), then kintera non-CUDA libraries would be consumers of those CUDA providers. However, placing providers (kintera_cuda) last in the link order is correct for ensuring that dependencies are resolved properly. Consider clarifying these comments to explain that this ordering ensures proper symbol resolution where libraries that depend on CUDA functionality come before the CUDA provider libraries.

Suggested change
# 1) non-cuda libs first (consumers)
kintera_non_cuda = [l for l in library_names if l.startswith("kintera") and "cuda" not in l]
# 2) cuda libs last (providers)
# 1) non-CUDA kintera libs first (they may depend on CUDA symbols)
kintera_non_cuda = [l for l in library_names if l.startswith("kintera") and "cuda" not in l]
# 2) CUDA kintera libs last (they provide the CUDA symbols for earlier deps)

Copilot uses AI. Check for mistakes.
kintera_cuda = [l for l in library_names if l.startswith("kintera") and "cuda" in l]
# 3) everything else
other = [l for l in library_names if not l.startswith("kintera")]
return kintera_non_cuda + other + kintera_cuda

site_dir = sysconfig.get_paths()["purelib"]

Expand Down Expand Up @@ -56,33 +62,23 @@ def parse_library_names(libdir):
]
else:
extra_link_args = [
"-Wl,--no-as-needed",
"-Wl,-rpath,$ORIGIN/lib",
"-Wl,-rpath,$ORIGIN/../torch/lib",
"-Wl,-rpath,$ORIGIN/../pydisort/lib",
"-Wl,-rpath,$ORIGIN/../pyharp/lib",
"-Wl,--as-needed",
]

if torch.cuda.is_available():
ext_module = cpp_extension.CUDAExtension(
name='kintera.kintera',
sources=glob.glob('python/csrc/*.cpp'),
include_dirs=include_dirs,
library_dirs=lib_dirs,
libraries=libraries,
extra_compile_args={'nvcc': ['--extended-lambda'],
'cc': ["-Wno-attributes"]},
extra_link_args=extra_link_args,
ext_module = cpp_extension.CppExtension(
name='kintera.kintera',
sources=glob.glob('python/csrc/*.cpp'),
include_dirs=include_dirs,
library_dirs=lib_dirs,
libraries=libraries,
extra_compile_args=['-Wno-attributes'],
extra_link_args=extra_link_args,
Comment on lines +73 to +80
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --no-as-needed linker flag is being added to test executables in CMake to prevent Linux from aggressively dropping symbols. However, the Python extension in setup.py also links to the same kintera libraries and may suffer from the same symbol dropping issue on Linux. Consider adding -Wl,--no-as-needed to extra_link_args for non-Darwin platforms (lines 64-69) to ensure consistent behavior and prevent potential symbol resolution issues in the Python extension.

Copilot uses AI. Check for mistakes.
)
else:
ext_module = cpp_extension.CppExtension(
name='kintera.kintera',
sources=glob.glob('python/csrc/*.cpp'),
include_dirs=include_dirs,
library_dirs=lib_dirs,
libraries=libraries,
extra_compile_args=['-Wno-attributes'],
extra_link_args=extra_link_args,
)

setup(
package_dir={"kintera": "python"},
Expand Down
Loading