[ROOT6_X, ROOT636_X, RNTUPLE_X] Some AddOn tests failing with SIGSEGV in CLING #49333
Description
Step 3 of the following AddOn tests failed in CMSSW_16_0_ROOT6_X_2025-11-05-2300, CMSSW_16_0_ROOT636_X_2025-11-05-2300, CMSSW_16_0_RNTUPLE_X_2025-11-05-2300 IBs due to SIGSEGV in CLING:
- hlt_data_GRun
- hlt_data_PIon
- hlt_data_PRef
- hlt_data_Special
- hlt_mc_GRun
- hlt_mc_PIon
- hlt_mc_PRef
- hlt_mc_Special
Example stacktrace (full log):
Thread 1 (Thread 0x14e800666580 (LWP 2063281) "cmsRun"):
#0 0x000014e80015bbb1 in poll () from /lib64/libc.so.6
#1 0x000014e7fbedb88e in edm::service::InitRootHandlers::stacktraceFromThread() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/pluginFWCoreServicesPlugins.so
#2 0x000014e7fbedba93 in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/pluginFWCoreServicesPlugins.so
#3 <signal handler called>
#4 0x000014e7effc948b in clang::BaseUsingDecl::removeShadowDecl(clang::UsingShadowDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#5 0x000014e7ecbf7ddb in clang::declvisitor::Base<std::add_pointer, cling::DeclUnloader, bool>::Visit(clang::Decl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#6 0x000014e7ecbf43c5 in cling::DeclUnloader::VisitDeclContext(clang::DeclContext*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#7 0x000014e7ecbf451c in cling::DeclUnloader::VisitFunctionDecl(clang::FunctionDecl*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#8 0x000014e7ecbb6695 in cling::TransactionUnloader::unloadDeserializedDeclarations(cling::Transaction*, cling::DeclUnloader&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#9 0x000014e7ecbb698f in cling::TransactionUnloader::RevertTransaction(cling::Transaction*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#10 0x000014e7ecb94e73 in cling::Interpreter::unload(cling::Transaction&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#11 0x000014e7ecc2e16d in cling::IncrementalParser::commitTransaction(llvm::PointerIntPair<cling::Transaction*, 2u, cling::IncrementalParser::EParseResult, llvm::PointerLikeTypeTraits<cling::Transaction*>, llvm::PointerIntPairInfo<cling::Transaction*, 2u, llvm::PointerLikeTypeTraits<cling::Transaction*> > >&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#12 0x000014e7ecc3108b in cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#13 0x000014e7ecb965c2 in cling::Interpreter::parseForModule(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#14 0x000014e7eca9e9cc in ExecAutoParse(char const*, bool, cling::Interpreter*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#15 0x000014e7ecaa125b in TCling::AutoParseImplRecurse(char const*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#16 0x000014e7ecaaa5e9 in TCling::AutoParse(char const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#17 0x000014e7ecaf0bae in TClingCallbacks::LookupObject(clang::TagDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#18 0x000014e7ecb9df8c in cling::MultiplexInterpreterCallbacks::LookupObject(clang::TagDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#19 0x000014e7eda03689 in clang::MultiplexExternalSemaSource::CompleteType(clang::TagDecl*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#20 0x000014e7ee3f78cd in clang::Sema::RequireCompleteTypeImpl(clang::SourceLocation, clang::QualType, clang::Sema::CompleteTypeKind, clang::Sema::TypeDiagnoser*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#21 0x000014e7ee3f7f96 in clang::Sema::RequireCompleteType(clang::SourceLocation, clang::QualType, clang::Sema::CompleteTypeKind, clang::Sema::TypeDiagnoser&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#22 0x000014e7eda6d6f7 in clang::Sema::RequireCompleteDeclContext(clang::CXXScopeSpec&, clang::DeclContext*&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#23 0x000014e7eda70b07 in clang::Sema::BuildCXXNestedNameSpecifier(clang::Scope*, clang::Sema::NestedNameSpecInfo&, bool, clang::CXXScopeSpec&, clang::NamedDecl*, bool, bool*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#24 0x000014e7eda74141 in clang::Sema::ActOnCXXNestedNameSpecifier(clang::Scope*, clang::Sema::NestedNameSpecInfo&, bool, clang::CXXScopeSpec&, bool*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#25 0x000014e7ed7c9022 in clang::Parser::ParseOptionalCXXScopeSpecifier(clang::CXXScopeSpec&, clang::OpaquePtr<clang::QualType>, bool, bool, bool*, bool, clang::IdentifierInfo**, bool, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#26 0x000014e7ed85952f in clang::Parser::TryAnnotateCXXScopeToken(bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#27 0x000014e7ecba902f in cling::LookupHelper::findScope(llvm::StringRef, cling::LookupHelper::DiagSetting, clang::Type const**, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#28 0x000014e7eca8e41e in TCling::GetClassSharedLibs(char const*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#29 0x000014e7ecaab802 in TClingLookupHelper__ExistingTypeCheck(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCling.so
#30 0x000014e800af3209 in ResolveTypedefProcessType(char const*, unsigned int, unsigned int, bool, unsigned int, unsigned int, unsigned int, bool&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) [clone .isra.0] () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#31 0x000014e800af3959 in ResolveTypedefImpl(char const*, unsigned int, unsigned int&, bool&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#32 0x000014e800af4bfb in TClassEdit::ResolveTypedef[abi:cxx11](char const*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#33 0x000014e800af7395 in TClassEdit::TSplitType::ShortType(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#34 0x000014e800af823d in TClassEdit::GetNormalizedName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::basic_string_view<char, std::char_traits<char> >) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#35 0x000014e800b1c24a in TClass::GetClass(char const*, bool, bool, unsigned long, unsigned long) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#36 0x000014e7fa5e15bc in (anonymous namespace)::GetSourceType(ROOT::TSchemaRule::TSources const*, TStreamerElement const*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libRIO.so
#37 0x000014e7fa5e2710 in (anonymous namespace)::UpdateFromRule(TStreamerInfo const*, ROOT::TSchemaRule::TSources const*, TStreamerElement*) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libRIO.so
#38 0x000014e7fa5fea93 in TStreamerInfo::Build(bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libRIO.so
#39 0x000014e800b0c6fa in TClass::GetStreamerInfoImpl(int, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#40 0x000014e800b0c8ab in TClass::GetStreamerInfo(int, bool) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libCore.so
#41 0x000014e7fa5dd404 in TStreamerInfo::ForceWriteInfo(TFile*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libRIO.so
#42 0x000014e7fbde011e in TTree::BuildStreamerInfo(TClass*, void*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libTree.so
#43 0x000014e7fbdf5151 in TTree::BronchExec(char const*, char const*, void*, bool, int, int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libTree.so
#44 0x000014e7fbdde0d5 in TTree::Branch(char const*, char const*, void*, int, int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/external/el8_amd64_gcc13/lib/libTree.so
#45 0x000014e770f89609 in edm::RootOutputTree::addBranch(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void const*&, int, int, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/libIOPoolOutput.so
#46 0x000014e770f8a849 in edm::RootOutputFile::RootOutputFile(edm::PoolOutputModule*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/libIOPoolOutput.so
#47 0x000014e770f738ef in edm::PoolOutputModule::reallyOpenFile() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/libIOPoolOutput.so
#48 0x000014e770f73a41 in virtual thunk to edm::PoolOutputModule::openFile(edm::FileBlock const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/libIOPoolOutput.so
#49 0x000014e80100d1a0 in edm::Schedule::openOutputFiles(edm::FileBlock&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/libFWCoreFramework.so
#50 0x000014e800f509dc in edm::EventProcessor::openOutputFiles() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/libFWCoreFramework.so
#51 0x000014e800f58606 in edm::EventProcessor::runToCompletion() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02914/el8_amd64_gcc13/cms/cmssw/CMSSW_16_0_ROOT636_X_2025-11-05-2300/lib/el8_amd64_gcc13/libFWCoreFramework.so
#52 0x000000000040916d in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#53 0x000014e800d588c2 in tbb::detail::r1::task_arena_impl::execute (ta=..., d=...) at src/tbb/arena.cpp:860
#54 0x000000000040abfe in main::{lambda()#1}::operator()() const ()
#55 0x0000000000405268 in main ()