[Bug]: SSE3 variant build contains AVX-512 instructions. It crashes on non-AVX CPUs (Illegal instruction / Segfault)

[freymeyer]

Bug Description

The SSE3 variant build contains AVX-512 instructions from the bundled roaring bitmaps
third-party dependency. This causes an immediate crash on any CPU without AVX support
(Pentium Silver/Gold, Celeron, Atom, older budget x86 chips).

Steps to Reproduce

1. Use any CPU without AVX (e.g. Intel Pentium Silver N6000 has SSE4.2 but no AVX)
2. git clone https://github.com/volcengine/OpenViking.git && cd OpenViking
3. OV_X86_BUILD_VARIANTS=sse3 pip install -e .
4. python -c "import openviking"
   → Crashes immediately

Expected Behavior

The SSE3 variant should run cleanly on any CPU with SSE3/SSE4 support,
with no AVX instructions anywhere in the build including third-party dependencies.

Actual Behavior

Process crashes with "Illegal instruction (core dumped)" or "Segmentation fault
(core dumped)" on import. The official Docker image (ghcr.io/volcengine/openviking:latest)
also crashes the same way, confirming this is not an environment issue.

Minimal Reproducible Example

import openviking  # crashes here on non-AVX CPU

Error Logs

$ python -c "import openviking; print('import OK')"
corrupted size vs. prev_size
Aborted (core dumped)

$ objdump -d openviking/storage/vectordb/engine/_x86_sse3.cpython-312-x86_64-linux-gnu.so | grep -E "ymm|zmm" | head -5
141d2d: 62 f1 fe 48 6f 01    vmovdqu64 (%rcx),%zmm0
141d33: 62 f1 7d 48 74 00    vpcmpeqb (%rax),%zmm0,%k0
141dac: 62 f2 fd 48 7c c8    vpbroadcastq %rax,%zmm1
141db2: 62 f2 fd 48 7c c1    vpbroadcastq %rcx,%zmm0
141dba: 62 f1 fd 48 6f 15 3c vmovdqa64 0xec23c(%rip),%zmm2

Symbol _ZN7roaring8internal appears alongside these AVX-512 instructions,
identifying roaring bitmaps as the source. OV_X86_BUILD_VARIANTS=sse3 does
not propagate to third-party deps.

OpenViking Version

v0.1.12 (main branch, built from source)

Python Version

3.12.9

Operating System

Linux

Model Backend

None

Additional Context

No response

[zhoujh01]

Thanks for the feedback, I'll fix it.

[qin-ptr]

Thanks for the detailed bug report and investigation! I've analyzed the root cause and can confirm this is a legitimate issue.

Root Cause

The problem is that Roaring Bitmaps uses GCC/Clang function multiversioning to generate CPU-specific code paths. Even when compiling with -msse3, functions marked with __attribute__((target("avx512f,..."))) will still emit AVX-512 instructions because the function attribute overrides global compiler flags.

Specifically:

• OpenViking defines OV_DISABLE_AVX512=1 for sse3/avx2 variants (src/CMakeLists.txt:144, 152)
• However, Roaring Bitmaps doesn't recognize this macro
• Roaring Bitmaps needs CROARING_COMPILER_SUPPORTS_AVX512=0 to prevent AVX-512 code compilation (third_party/croaring/roaring.c:404, 1194, 11281+)

The disconnect happens at src/index/detail/scalar/bitmap_holder/bitmap.cpp:7 where roaring.c is directly included.

Proposed Fix

Add CROARING_COMPILER_SUPPORTS_AVX512=0 to the compile definitions for sse3 and avx2 variants in the ov_get_x86_variant_flags function:

# For sse3 variant (around line 144):
list(APPEND OV_DEFS OV_DISABLE_AVX512=1)
list(APPEND OV_DEFS CROARING_COMPILER_SUPPORTS_AVX512=0)

# For avx2 variant (around line 152):
list(APPEND OV_DEFS OV_DISABLE_AVX512=1)
list(APPEND OV_DEFS CROARING_COMPILER_SUPPORTS_AVX512=0)

This will prevent Roaring Bitmaps from compiling any AVX-512 code in these variants, ensuring they run correctly on CPUs without AVX support.

🤖 I am a bot owned by @qin-ctx.

[zhoujh01]

#771