Skip to content

Commit 9259b48

Browse files
authored
FEAT: Build Wheels for MacOS (#85)
### Summary This pull request introduces significant updates to the `eng/pipelines/build-whl-pipeline.yml` file to expand the build pipeline by adding support for macOS alongside existing Windows builds. Key changes include the addition of a macOS-specific job for building Python wheels, enhancements to the build strategy for cross-platform compatibility, and new steps for dependency installation and artifact management. ### macOS Build Pipeline Addition: * Added a new `BuildMacOSWheels` job to the pipeline, which uses a macOS image (`macos-latest`) to build Python wheels for Python versions 3.10 through 3.13 on the x86_64 architecture. This includes a matrix strategy to handle multiple Python versions. * Introduced steps to install macOS-specific dependencies such as `CMake` and `pybind11`, build `.so` files using a `build.sh` script, and package them into wheel files. ### Dependency Management and Artifact Handling: * Added steps to install required Python dependencies and tools (e.g., `pip`, `setuptools`, `wheel`) and to copy and publish the built `.so` and `.whl` files as build artifacts for macOS. * Enhanced the pipeline to include tasks for publishing test results and code coverage reports for macOS builds. ### General Pipeline Adjustments: * Updated the pipeline structure to include the new `BuildMacOSWheels` job alongside the existing `BuildWindowsWheels` job, ensuring cross-platform compatibility. ### Issue Reference Fixes [AB#37752](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/37752) ### Checklist - [x] **Tests Passed** (if applicable) - [x] **Code is formatted** - [x] **Docs Updated** (if necessary) ### Testing Performed <!-- How was this fix tested? --> - [x] Unit Tests` ### Additional Notes <!-- Any extra details or related links -->
1 parent 54b649b commit 9259b48

File tree

2 files changed

+102
-10
lines changed

2 files changed

+102
-10
lines changed

eng/pipelines/build-whl-pipeline.yml

Lines changed: 98 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,12 @@ pr:
1313
include:
1414
- main
1515

16-
# Use Microsoft-hosted Windows VM
17-
pool:
18-
vmImage: 'windows-latest'
19-
2016
jobs:
21-
- job: BuildPYDs
22-
displayName: 'Build -'
17+
- job: BuildWindowsWheels
18+
# Use the latest Windows image for building
19+
pool:
20+
vmImage: 'windows-latest'
21+
displayName: 'Build Windows -'
2322
# Strategy matrix to build all combinations
2423
strategy:
2524
matrix:
@@ -123,7 +122,7 @@ jobs:
123122
inputs:
124123
SourceFolder: '$(Build.SourcesDirectory)\mssql_python\pybind\build\$(targetArch)\py$(shortPyVer)\Release'
125124
Contents: 'ddbc_bindings.cp$(shortPyVer)-*.pyd'
126-
TargetFolder: '$(Build.ArtifactStagingDirectory)\all-pyds'
125+
TargetFolder: '$(Build.ArtifactStagingDirectory)\ddbc-bindings'
127126
displayName: 'Place PYD file into artifacts directory'
128127

129128
# Copy the built .pdb files to staging folder for artifacts
@@ -154,12 +153,15 @@ jobs:
154153
- task: PublishBuildArtifacts@1
155154
condition: succeededOrFailed()
156155
inputs:
157-
PathtoPublish: '$(Build.ArtifactStagingDirectory)\all-pyds'
158-
ArtifactName: 'mssql-python-pyds'
156+
PathtoPublish: '$(Build.ArtifactStagingDirectory)\ddbc-bindings'
157+
ArtifactName: 'mssql-python-ddbc-bindings'
159158
publishLocation: 'Container'
160159
displayName: 'Publish all PYDs as artifacts'
161160

162161
# Publish the python arm64 libraries as build artifacts for next builds if ARM64
162+
# We publish them only for ARM64 builds since we cannot install arm64 Python on x64 host
163+
# This allows us to reuse the libraries in future ARM64 builds
164+
# Publishing will retain the libraries in the build artifacts
163165
- task: PublishBuildArtifacts@1
164166
condition: eq(variables['targetArch'], 'arm64')
165167
inputs:
@@ -176,3 +178,90 @@ jobs:
176178
ArtifactName: 'mssql-python-wheels-dist'
177179
publishLocation: 'Container'
178180
displayName: 'Publish all wheels as artifacts'
181+
182+
- job: BuildMacOSWheels
183+
# Use the latest macOS image for building
184+
pool:
185+
vmImage: 'macos-latest'
186+
# Display name for the job in Azure DevOps UI
187+
displayName: 'Build macOS - '
188+
strategy:
189+
matrix:
190+
# Python 3.13 (only x86_64 for builds)
191+
py313_x86_64:
192+
pythonVersion: '3.13'
193+
shortPyVer: '313'
194+
# targetArch is not used anywhere since build.sh does auto-detection, kept for cross-compilation of ARM64 Builds
195+
targetArch: 'x86_64'
196+
steps:
197+
# Use correct Python version and architecture for the current job
198+
- task: UsePythonVersion@0
199+
inputs:
200+
versionSpec: '$(pythonVersion)'
201+
addToPath: true
202+
displayName: 'Use Python $(pythonVersion) (x86_64)'
203+
204+
# Install CMake on macOS
205+
- script: |
206+
brew update
207+
brew install cmake
208+
displayName: 'Install CMake'
209+
210+
# Install required packages: pip, CMake, pybind11
211+
- script: |
212+
python -m pip install --upgrade pip
213+
pip install -r requirements.txt
214+
pip install cmake pybind11
215+
displayName: 'Install dependencies'
216+
217+
# Build the .so file by calling build.sh
218+
- script: |
219+
echo "Python Version: $(pythonVersion)"
220+
echo "Short Tag: $(shortPyVer)"
221+
echo "Architecture: Target=$(targetArch)"
222+
cd "$(Build.SourcesDirectory)/mssql_python/pybind"
223+
# Call build.sh to build the .so file
224+
./build.sh
225+
displayName: 'Build .so file'
226+
227+
# Copy the built .so file to staging folder for artifacts
228+
- task: CopyFiles@2
229+
inputs:
230+
SourceFolder: '$(Build.SourcesDirectory)/mssql_python'
231+
Contents: '*.so'
232+
TargetFolder: '$(Build.ArtifactStagingDirectory)/ddbc-bindings'
233+
displayName: 'Place .so file into artifacts directory'
234+
235+
# Build wheel package for the current architecture
236+
- script: |
237+
python -m pip install --upgrade pip
238+
pip install wheel setuptools
239+
set ARCHITECTURE=$(targetArch)
240+
python setup.py bdist_wheel
241+
displayName: 'Build wheel package for Python $(pythonVersion) ($(targetArch))'
242+
243+
# Copy the wheel file to the artifacts
244+
- task: CopyFiles@2
245+
inputs:
246+
SourceFolder: '$(Build.SourcesDirectory)/dist'
247+
Contents: '*.whl'
248+
TargetFolder: '$(Build.ArtifactStagingDirectory)/dist'
249+
displayName: 'Collect wheel package'
250+
251+
# Publish the collected .so file(s) as build artifacts
252+
- task: PublishBuildArtifacts@1
253+
condition: succeededOrFailed()
254+
inputs:
255+
PathtoPublish: '$(Build.ArtifactStagingDirectory)/ddbc-bindings'
256+
ArtifactName: 'mssql-python-ddbc-bindings'
257+
publishLocation: 'Container'
258+
displayName: 'Publish all .so files as artifacts'
259+
260+
# Publish the collected wheel file(s) as build artifacts
261+
- task: PublishBuildArtifacts@1
262+
condition: succeededOrFailed()
263+
inputs:
264+
PathtoPublish: '$(Build.ArtifactStagingDirectory)/dist'
265+
ArtifactName: 'mssql-python-wheels-dist'
266+
publishLocation: 'Container'
267+
displayName: 'Publish all wheels as artifacts'

setup.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,8 +81,11 @@ def finalize_options(self):
8181
if platform.machine() == 'arm64':
8282
arch = 'arm64'
8383
platform_tag = 'macosx_15_0_arm64'
84+
elif platform.machine() == 'x86_64':
85+
arch = 'x86_64'
86+
platform_tag = 'macosx_15_0_x86_64'
8487
else:
85-
raise Exception("Unsupported architecture for macOS. Please set the ARCHITECTURE environment variable to 'arm64'.")
88+
raise Exception("Unsupported architecture for macOS.")
8689

8790
# Add architecture-specific packages for macOS
8891
packages.extend([

0 commit comments

Comments
 (0)