Performance: CMake configure with cached vcpkg takes 1hr 8min on Linux vs 1min on Windows

[kumaakh]

Problem

CMake configuration phase takes dramatically longer on Linux compared to Windows when using vcpkg with a fully built cache.

Timing Comparison

Windows NoCUDA (Build #19787730835)

• Prep phase Configure: 1 minute 8 seconds
• Test phase Configure: 1 minute 8 seconds
• Build: https://github.com/Apra-Labs/ApraPipes/actions/runs/19787730835

Linux NoCUDA (Build #19788331486)

• Prep phase Configure: 1 hour 7 minutes 30 seconds
• Test phase Configure: 1 hour 10 minutes 12 seconds
• Build: https://github.com/Apra-Labs/ApraPipes/actions/runs/19788331486

Analysis

The Linux CMake configure step takes approximately 68x longer than Windows, despite both using cached vcpkg dependencies. This suggests the issue is NOT with building dependencies, but with:

1. CMake find_package() performance: CMake may be scanning the filesystem extensively on Linux
2. vcpkg toolchain file overhead: The vcpkg CMake toolchain may have performance issues on Linux
3. Filesystem performance: Linux runner filesystem may be slower (network-mounted?)
4. CMake version differences: Different CMake versions or configurations between platforms

Impact

• Linux CI builds take 1+ hour just for CMake configuration
• This affects developer productivity and CI costs
• Total Linux build time is 2+ hours vs 20-30 minutes on Windows

Suggested Investigation

1. Enable CMake verbose/trace mode to see what's taking time
2. Check if vcpkg is rebuilding packages instead of using cache
3. Compare CMake versions between Linux and Windows runners
4. Profile the CMake configuration step
5. Check if there are I/O bottlenecks on Linux runners

Environment

• Linux: ubuntu-22.04 GitHub runner, CMake 3.29.6
• Windows: windows-2022 GitHub runner, CMake 3.29.6
• vcpkg: Same baseline across both platforms
• Cache key: Linux-4 / Windows-4

[kumaakh]

🔍 Critical Analysis - Potentially Outdated

Issue: CMake configure takes 1hr+ on Linux vs 1min on Windows

Age: Nov 2025 - data from builds #19787730835 and #19788331486

Critical Questions:

1. Is this still happening with current CI? Check recent Linux build times
2. The 68x difference is extreme - was vcpkg actually rebuilding packages?

Possible Root Causes (in order of likelihood):

1. vcpkg cache miss - Different cache key between runs causing full rebuild
2. vcpkg binary cache not working - Check if VCPKG_BINARY_SOURCES is set correctly
3. Filesystem I/O - GitHub runners can have slow I/O

Verification:
Check a recent CI-Linux run and look for:

• "Restored X package(s)" in vcpkg output
• Total configure time

Recommendation:

• If recent builds are fast (< 10 min configure), close as resolved
• If still slow, focus on vcpkg binary caching, not CMake itself

Priority: High if still occurring, but likely already fixed by cache improvements