Build ODBC statically on macOS #153

alinaliBQ · 2026-01-28T23:37:36Z

Enable ODBC static build on macOS. This will make it easier for end users to handle the dependencies. Users just need to use the one ODBC .DYLIB file to use the driver. New flags that are needed for build:

       # Enable static build 
      ARROW_DEPENDENCY_USE_SHARED: OFF
      ARROW_DEPENDENCY_SOURCE: BUNDLED
       # Workaround for mimalloc run-time error
      ARROW_MIMALLOC: OFF

Fix c-ares build error:

CMake Error at cmake_modules/ThirdpartyToolchain.cmake:3032 (set_target_properties):
  set_target_properties can not be used on an ALIAS target.

Change unit tests to re-use code. Testing on windows is needed. Now seeing error ``` Library not loaded: /usr/local/opt/grpc/lib/libgrpc++.1.76.dylib ``` on Excel. Attempt to unlink arrow_flight.proto Now I see ODBC.dylib has 100MB, and error from Excel is ``` Library not loaded: /usr/local/opt/libiodbc/lib/libiodbc.2.dylib ``` instead of looking for flightsql. `iodbctest` command outside of Excel works. in-progress link arrow statically for `flightsql-odbc` * both ODBC driver and `arrow_odbc_spi_impl` need to link Arrow statically for tests to work. * getting error `File already exists in database: Flight.proto` since the ODBC layer is linking dynamically while I successfully got `arrow_odbc_spi_impl` to link to Arrow statically Draft for resolving `File already exists in database: FlightSql.proto` error Note: make ODBC available on Excel first, test with `iodbctest`, worry about ODBC test executable later. Add unix build (same as Windows build) * Attempted to link static libraries with `arrow_odbc_spi_impl`, failed as it caused linking errors and rendered the ODBC driver unusable. Undo changes to resolve broken driver Resolves dependency errors Attempt to switch to static build Still getting error ``` [iODBC][Driver Manager]dlopen(/Library/ODBC/arrow-odbc/libarrow_flight_sql_odbc.2300.0.0.dylib, 0x0006): Library not loaded: @rpath/libarrow_flight_sql.2300.dylib ``` In-progress attempt to build ODBC statically on unix systems In-progress changes to enable static odbc_impl library pushing the changes just for saving my code. Currently odbc test still has double registration issue, work on this next. I think it should be solvable by either adding library flags or just using odbc spi shared. Because `odbc_spi_impl_test` works. And the ODBC tests worked just fine before I switched to static odbc spi impl. Fix for gtest missing issue Attempting to fix for issue: IMPORTANT NOTICE - DO NOT IGNORE: This test program did NOT call testing::InitGoogleTest() before calling RUN_ALL_TESTS(). This is INVALID. Soon Google Test will start to enforce the valid usage. Please fix it ASAP, or IT WILL START TO FAIL. Issue fixed with adding `${ARROW_TEST_LINK_LIBS}`. It is needed alongside `arrow_flight_testing_shared`. IN-PROGRESS build static test on macOS * Trying to have 2 builds. One `arrow_odbc_spi_impl_static` for static build and one `arrow_odbc_spi_impl_shared` for a shared lib that links Arrow dynamically --- only committing the approach that worked. Trying to work from `flight_sql_odbc_test` to debug the real issue. Link 3rd party dependencies statically Fix 3rd party headers cannot find issue for: - boost xpressive and beast headers - rapidjson headers Resolve unneeded link for ODBC ---- Moving `set(APPEND CMAKE_FIND_LIBRARY_SUFFIXES ".a")` up didn't seem to have any effect ---- Fix for static gRPC connection issue: export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=/etc/ssl/cert.pem Fix cares `cannot modify alias target` issue Make arrow_odbc_spi_impl static lib In progress for getting gprc linked statically Add debug msgs in add_arrow_lib BuildUtils.cmake Remove commented out code in odbc lib Fix build issue from odbc impl tests Added draft code for Link Arrow libs statically on Unix for unit tests Add commented out code`ARROW_DEPENDENCY_USE_SHARED` in ODBC cpp yml Uncomment when it is verified to work. In-progress Fix ODBC test build and make it run. Build executable from scratch Revert "Build executable from scratch" This reverts commit a51dc39. Dummy test dummy test without using ODBC passed without double registration issue Make ODBC test run Issue to resolve: [libprotobuf FATAL /path/to/arrow/cpp/debug-build/_deps/protobuf-src/src/google/protobuf/extension_set.cc:100] Multiple extension registrations for type "google.protobuf.MessageOptions", field number 1000. unknown file: Failure C++ exception with description "Multiple extension registrations for type "google.protobuf.MessageOptions", field number 1000." thrown in SetUp(). Fix: only link test executable directly with ODBC shared dylib. Do not link test executable with odbc spi impl dylib. And change ODBC to link odbc spi impl dylib publicly instead of privately

Can revert later

As dependencies are static and built from source

Resolves the issue of ODBC on macOS being dynamically linked to ODBC

Need to check if Windows CI passes

- Remove dummy tests - Remove unneeded changes

This reverts commit 711b3e0.

Example of failed run in forked repo: https://github.com/Bit-Quill/arrow/actions/runs/21459770235/job/61809740997?pr=153#step:7:540

alinaliBQ · 2026-01-29T18:11:02Z

cpp/cmake_modules/ThirdpartyToolchain.cmake

@@ -3026,6 +3031,10 @@ function(build_cares)
    find_library(LIBRESOLV_LIBRARY NAMES resolv libresolv REQUIRED)


Without this change, we have failed run with c-ares::cares: https://github.com/Bit-Quill/arrow/actions/runs/21459770235/job/61809740997?pr=153#step:7:540

Fixes error:

CMake Error at cmake_modules/ThirdpartyToolchain.cmake:3032 (set_target_properties): set_target_properties can not be used on an ALIAS target.

alinaliBQ · 2026-01-30T00:32:47Z

cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc

  ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_IMP_PARAM_DESC,
-                               static_cast<SQLULEN>(0),
+                               static_cast<SQLULEN>(0), SQL_ERROR,
 #ifdef __APPLE__
-                               // iODBC on MacOS returns SQL_INVALID_HANDLE for this case
-                               SQL_INVALID_HANDLE,
+                               // static iODBC on MacOS returns IM001 for this case
+                               kErrorStateIM001);


@justing-bq Could you kindly let me know if this change breaks tests on ARM macOS?

StatementAttributeTests all pass on my machine with your change.

alinaliBQ · 2026-01-30T00:58:44Z

Converting PR to draft due to new test issues I found

Fixes issue: ``` mimalloc: assertion failed: at "arrow/cpp/debug-build/mimalloc_ep-prefix/src/mimalloc_ep/include/mimalloc/internal.h":658, _mi_ptr_page assertion: "p==NULL || mi_is_in_heap_region(p)" zsh: abort debug/arrow-flight-sql-odbc-test ```

alinaliBQ · 2026-01-30T01:45:32Z

Test issue resolved on Intel mac

justing-bq · 2026-01-30T17:40:38Z

.github/workflows/cpp_extra.yml

      - name: Install Dependencies
        run: |
          brew bundle --file=cpp/Brewfile
+          # We want to use bundled RE2 for static linking. If


Can we add some blank lines for readability?

justing-bq · 2026-01-30T17:41:57Z

cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt

+                               arrow_compute_static
+                               Boost::locale
+                               Boost::headers
+                               RapidJSON)


We don't need iodbc anymore?

justing-bq · 2026-01-30T17:53:24Z

cpp/src/arrow/flight/sql/odbc/tests/statement_attr_test.cc

  ValidateSetStmtAttrErrorCode(this->stmt, SQL_ATTR_IMP_PARAM_DESC,
-                               static_cast<SQLULEN>(0),
+                               static_cast<SQLULEN>(0), SQL_ERROR,
 #ifdef __APPLE__
-                               // iODBC on MacOS returns SQL_INVALID_HANDLE for this case
-                               SQL_INVALID_HANDLE,
+                               // static iODBC on MacOS returns IM001 for this case
+                               kErrorStateIM001);


StatementAttributeTests all pass on my machine with your change.

alinaliBQ and others added 18 commits January 28, 2026 15:02

in-progress changes to accomodate macos CI

9288a6f

Test ODBC CI - to be reverted

711b3e0

Can revert later

Only install ODBC dependencies in mac

eea27ba

As dependencies are static and built from source

Uninstall absl as attempt to resolve build on Intel

f27dad7

Add C++ standard 20

7b72564

Remove absl header from intel

b80234c

Use static linking in dependency install step

51b8526

Use bundled boost for static linking

f6efc09

In-progress Fix Windows build

27f4573

Fix macOS (after Windows build)

f661478

Remove unneeded ODBC link

2eb9a5d

Resolves the issue of ODBC on macOS being dynamically linked to ODBC

Fix ODBC double proto issue on macOS

88c7c96

Need to check if Windows CI passes

Clean up PR

aac357d

- Remove dummy tests - Remove unneeded changes

Revert "Test ODBC CI - to be reverted"

c1de9c8

This reverts commit 711b3e0.

Remove unneeded code

10bf349

Attempt to revert c-ares change

64fe7f9

Fix set_target_properties can not be used on an ALIAS target. error.

7ed2f88

Example of failed run in forked repo: https://github.com/Bit-Quill/arrow/actions/runs/21459770235/job/61809740997?pr=153#step:7:540

alinaliBQ marked this pull request as ready for review January 29, 2026 20:08

Fix descriptor pointer tests on static macOS

87cfd56

alinaliBQ commented Jan 30, 2026

View reviewed changes

alinaliBQ marked this pull request as draft January 30, 2026 00:58

Fix mimalloc issue

89466c0

Fixes issue: ``` mimalloc: assertion failed: at "arrow/cpp/debug-build/mimalloc_ep-prefix/src/mimalloc_ep/include/mimalloc/internal.h":658, _mi_ptr_page assertion: "p==NULL || mi_is_in_heap_region(p)" zsh: abort debug/arrow-flight-sql-odbc-test ```

alinaliBQ marked this pull request as ready for review January 30, 2026 01:45

justing-bq reviewed Jan 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build ODBC statically on macOS #153

Build ODBC statically on macOS #153

Uh oh!

alinaliBQ commented Jan 28, 2026 •

edited

Loading

Uh oh!

alinaliBQ Jan 29, 2026

Uh oh!

alinaliBQ Jan 30, 2026

Uh oh!

justing-bq Jan 30, 2026

Uh oh!

alinaliBQ commented Jan 30, 2026

Uh oh!

alinaliBQ commented Jan 30, 2026

Uh oh!

justing-bq Jan 30, 2026

Uh oh!

justing-bq Jan 30, 2026

Uh oh!

justing-bq Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -3026,6 +3031,10 @@ function(build_cares)
		find_library(LIBRESOLV_LIBRARY NAMES resolv libresolv REQUIRED)

Build ODBC statically on macOS #153

Are you sure you want to change the base?

Build ODBC statically on macOS #153

Uh oh!

Conversation

alinaliBQ commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alinaliBQ Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

alinaliBQ Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

justing-bq Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

alinaliBQ commented Jan 30, 2026

Uh oh!

alinaliBQ commented Jan 30, 2026

Uh oh!

justing-bq Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

justing-bq Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

justing-bq Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alinaliBQ commented Jan 28, 2026 •

edited

Loading