[LLD][COFF] Discard .llvmbc and .llvmcmd sections

[HaohaiWen]

Those sections are generated by -fembed-bitcode and do not need to be
kept in executable files.

[llvmbot]

@llvm/pr-subscribers-platform-windows

@llvm/pr-subscribers-lld

Author: Haohai Wen (HaohaiWen)

Changes

Those sections are generated by -fembed-bitcode and do not need to be
kept in executable files.

---

Full diff: https://github.com/llvm/llvm-project/pull/150897.diff

2 Files Affected:

• (modified) lld/COFF/InputFiles.cpp (+5)
• (added) lld/test/COFF/embed-bitcode.test (+30)

diff --git a/lld/COFF/InputFiles.cpp b/lld/COFF/InputFiles.cpp
index 2a6b63cbacca1..c08099b8810bb 100644
--- a/lld/COFF/InputFiles.cpp
+++ b/lld/COFF/InputFiles.cpp
@@ -403,6 +403,11 @@ SectionChunk *ObjFile::readSection(uint32_t sectionNumber,
     return nullptr;
   }
 
+  // Those sections are generated by -fembed-bitcode and do not need to be kept
+  // in executable files.
+  if (name == ".llvmbc" || name == ".llvmcmd")
+    return nullptr;
+
   // Object files may have DWARF debug info or MS CodeView debug info
   // (or both).
   //
diff --git a/lld/test/COFF/embed-bitcode.test b/lld/test/COFF/embed-bitcode.test
new file mode 100644
index 0000000000000..10f88c5c0117d
--- /dev/null
+++ b/lld/test/COFF/embed-bitcode.test
@@ -0,0 +1,30 @@
+# RUN: yaml2obj %s -o %t.obj
+# RUN: lld-link /entry:main /subsystem:console /out:%t.exe %t.obj
+# RUN: llvm-readobj -S %t.exe | FileCheck %s
+
+# CHECK-NOT: Name: .llvmbc
+# CHECK-NOT: Name: .llvmcmd
+
+--- !COFF
+header:
+  Machine:         IMAGE_FILE_MACHINE_AMD64
+
+sections:
+  - Name:            .text
+    Characteristics: [ IMAGE_SCN_CNT_CODE, IMAGE_SCN_MEM_EXECUTE, IMAGE_SCN_MEM_READ ]
+    SectionData:     "C3"
+  - Name:           .llvmbc
+    Characteristics: [ IMAGE_SCN_MEM_DISCARDABLE ]
+    SectionData:     "4243C0DE"
+  - Name:           .llvmcmd
+    Characteristics: [ IMAGE_SCN_MEM_DISCARDABLE ]
+    SectionData:     "2D63633100"
+
+symbols:
+  - Name:            main
+    Value:           0
+    SectionNumber:   1
+    SimpleType:      IMAGE_SYM_TYPE_NULL
+    ComplexType:     IMAGE_SYM_DTYPE_FUNCTION
+    StorageClass:    IMAGE_SYM_CLASS_EXTERNAL
+...

[mstorsjo]

LGTM

[realoriginal]

@mstorsjo Could this be reverted? This ends up breaking -lto-embed-bitcode=<> linker option for embedding a whole-program-bitcode representation of the linked executable, an incredibly useful feature.

[mstorsjo]

@mstorsjo Could this be reverted? This ends up breaking -lto-embed-bitcode=<> linker option for embedding a whole-program-bitcode representation of the linked executable, an incredibly useful feature.

@HaohaiWen who authored this PR, can you comment?

Actually, that makes me wonder what the motivation for this whole PR is to begin with: If building with -fembed-bitcode, but not wanting to have the bitcode included in the binary in the end, what's the motivation for using -fembed-bitcode at all? Isn't the only actual intended effect of that option to include the bitcode in the end binary?

[HaohaiWen]

Some discussion about this: https://discourse.llvm.org/t/end-to-end-fembed-bitcode-llvmbc-and-llvmcmd/56265/3

what the motivation for this whole PR is to begin with:

If we compile the source code with -fembed-bitcode, then,
Without LTO:
Each object file to linker will have a .llvmbc and .llvmcmd sections containing command line and bitcode to recompile this object file.
With LTO.
Each bitcode to linker will also have a .llvmbc and .llvmcmd global variables containing command line and bitcode to recompile this bitcode.

What inside the final executable is the merged .llvmbc and .llvmcmd. There's no guaranteed mapping between the N-th command line and the N-th bitcode module unless the order is preserved. LLVM also doesn't have existing API to split them apart from the final executable. Those two sections will occupy lot of space, making it easier to be greater then 2GB limitation.

I wonder what's motivation to use the merged .llvmbc and .llvmcmd in the final binary? I think use bitcode/object is more friendly and easier.

[realoriginal]

What inside the final executable is the merged .llvmbc and .llvmcmd. There's no guaranteed mapping between the N-th command line and the N-th bitcode module unless the order is preserved. LLVM also doesn't have existing API to split them apart from the final executable. Those two sections will occupy lot of space, making it easier to be greater then 2GB limitation.

However, when I request it to embed bitcode, I need it to embed bitcode. LLVM doesnt intentionally do that unless you ask for it.

[realoriginal]

@mstorsjo I'll submit a quick PR removing this patch seems it appears it brokes more than then just -lto-embed-bitcode but -fembed-bitcode features.

[mrexodia]

Just FYI the -lto-embed-bitcode= is used in production and very useful to obtain bitcode for analysis/rewriting. It seems like a weird thing to remove the effect of two supported flags (only on Windows)? If the user does not want to embed the bitcode they would simply not set the flags?

[HaohaiWen]

They were also removed from wasm already.
I suggest to add a flag to control whether to keep them in the final binary if we really want to use it in the final binary, as some user just want to retrieve them from object/bitcode.

[MaskRay]

I was also wary of this change, as I mentioned in this related comment. The points raised by @realoriginal and @mrexodia reinforce that maintaining concatenated bitcode in the .llvmbc section is used by certain workflows.

lld/ELF’s support for linker scripts allows for fine-grained control, such as using /DISCARD/ to remove unwanted input sections.

[realoriginal]

I've created #188398 for fixing this, so hopefully we can get this merged before the next version if possible