Skip to content

[ABLD-28] Backport upstream macOS multiarch support #240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
rdesgroppes merged 2 commits into from
Jul 18, 2025
Merged

[ABLD-28] Backport upstream macOS multiarch support #240

rdesgroppes merged 2 commits into from
Jul 18, 2025

Conversation

@rdesgroppes
Copy link

@rdesgroppes rdesgroppes commented Jul 16, 2025 •
edited
Loading

Description

While looking at the macOS installation script, I realized no architecture was passed to the dd-agent query string:

curl "https://s3.amazonaws.com/dd-agent?prefix=datadog-agent-${major_minor}."

There was no architecture disambiguation so far because our PKGs and DMGs had been primarily targeting x86_64/AMD64, and relying on Rosetta 2 translation for AArch64/ARM64.

The present change is therefore to backport (git cherry-pick) upstream efforts on the matter, i.e.:

  1. chef/omnibus@2b1b503
  2. chef/omnibus@6173b5f

Without/with an architecture qualifier:

-datadog-agent-7.70.0-devel.git.75.d018f55.pipeline.70720451-1.dmg
+datadog-agent-7.70.0-devel.git.79.1816262.pipeline.70752546-1.x86_64.dmg

(examples from development pipelines)

⚠️ depends on:

@rdesgroppes rdesgroppes requested a review from a team as a code owner July 16, 2025 16:58
@rdesgroppes rdesgroppes changed the title [ABLD-28] Backport macOS multiarch support [ABLD-28] Backport upstream macOS multiarch support Jul 16, 2025
aiuto
aiuto reviewed Jul 16, 2025
View reviewed changes
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 16, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
e9819e0 
The script doesn't need to wait. Quite the other way around as we're
going to add an architecture qualifier to DMGs, see:
DataDog/omnibus-ruby#240

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
@rdesgroppes rdesgroppes mentioned this pull request Jul 16, 2025
Merged
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 16, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
8d8d137 
The script doesn't need to wait for AArch64/ARM64 to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 16, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
4f6b362 
The script doesn't need to wait for AArch64/ARM64 to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 16, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
5f9ead0 
The script doesn't need to wait for AArch64/ARM64 to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 16, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
279355c 
The script doesn't need to wait for AArch64/ARM64 DMGs to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
@rdesgroppes rdesgroppes force-pushed the regis.desgroppes/backport-macos-multiarch-support branch from bb4f723 to 0af1fed Compare July 16, 2025 21:51
@rdesgroppes rdesgroppes requested review from a team and aiuto July 16, 2025 22:02
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 16, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
85f3a32 
The script doesn't need to wait for AArch64/ARM64 DMGs to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 16, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
65440db 
The script doesn't need to wait for AArch64/ARM64 DMGs to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 17, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
fc434b4 
The script doesn't need to wait for AArch64/ARM64 DMGs to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 17, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
20dc753 
The script doesn't need to wait for AArch64/ARM64 DMGs to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
@tduffield @rdesgroppes
Add support for macOS 11 and multiple architectures
421b589 
* Provide the correct platform version (11 vs 11.x) for macOS.
* Add the architecture (x86_64, arm64) to the file name

Cherry-picked-from: chef/omnibus@2b1b503
Co-authored-by: Régis Desgroppes <regis.desgroppes@datadoghq.com>
@rdesgroppes rdesgroppes force-pushed the regis.desgroppes/backport-macos-multiarch-support branch from 0af1fed to 0dce646 Compare July 17, 2025 14:01
@rdesgroppes rdesgroppes mentioned this pull request Jul 17, 2025
Merged
25 tasks
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 17, 2025
@rdesgroppes
Update macOS installation script for AArch64/ARM64
02daac8 
The script doesn't need to wait for AArch64/ARM64 DMGs to be publicly
released - quite the other way around as we're going to add an
architecture qualifier to DMGs, see: DataDog/omnibus-ruby#240.

The present change therefore adapts the script to the foreseeable
future, by considering the optional presence of an architecture
qualifier.
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 18, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
56a61be 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- #38498
- DataDog/omnibus-ruby#240
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
@rdesgroppes rdesgroppes force-pushed the regis.desgroppes/backport-macos-multiarch-support branch from 0dce646 to 71e05f5 Compare July 18, 2025 13:32
@rdesgroppes rdesgroppes merged commit fb400d5 into datadog-5.5.0 Jul 18, 2025
1 check passed
@rdesgroppes rdesgroppes deleted the regis.desgroppes/backport-macos-multiarch-support branch July 18, 2025 13:34
@rdesgroppes rdesgroppes mentioned this pull request Jul 18, 2025
Merged
1 task
@rdesgroppes rdesgroppes added this to the 7.70.0 milestone Jul 18, 2025
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 18, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
c5ea2b1 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- #38498
- DataDog/omnibus-ruby#240
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 18, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
a7df877 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- #38498
- DataDog/omnibus-ruby#240
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 18, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
25ef159 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- #38498
- DataDog/omnibus-ruby#240
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 21, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
eaa3f37 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- #38498
- DataDog/omnibus-ruby#240
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 21, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
b00e20c 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- #38498
- DataDog/omnibus-ruby#240
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 21, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
477bd36 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- DataDog/omnibus-ruby#240
- #38498
- #39082
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
- DataDog/agent-release-management#365
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 21, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
aabd73b 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- DataDog/omnibus-ruby#240
- #38498
- #39082
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
- DataDog/agent-release-management#365
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 22, 2025
@rdesgroppes
Add macOS build for AArch64/ARM64
9938ed8 
This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- DataDog/omnibus-ruby#240
- #38498
- #39082
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20812
- DataDog/integrations-core#20815
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
- DataDog/agent-release-management#365
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 22, 2025
@rdesgroppes
[ABLD-28] Add macOS build for AArch64/ARM64 (#37676)
87d5d66 
Add macOS build for AArch64/ARM64

This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- DataDog/omnibus-ruby#240
- #38498
- #39082
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20812
- DataDog/integrations-core#20815
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
- DataDog/agent-release-management#365
@rdesgroppes rdesgroppes mentioned this pull request Jul 23, 2025
Merged
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jul 28, 2025
@rdesgroppes
[ABLD-28] Add macOS build for AArch64/ARM64 (#37676)
2042230 
Add macOS build for AArch64/ARM64

This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- DataDog/omnibus-ruby#240
- #38498
- #39082
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20812
- DataDog/integrations-core#20815
- DataDog/integrations-core#20645
- DataDog/ci-platform-machine-images#343
- DataDog/agent-release-management#365
lucastemb pushed a commit to DataDog/datadog-agent that referenced this pull request Aug 7, 2025
@rdesgroppes @lucastemb
[ABLD-28] Add macOS build for AArch64/ARM64 (#37676)
d0ec76d 
Add macOS build for AArch64/ARM64

This is the payload for adding macOS "native" build support for the
AArch64/ARM64 architecture to the Agent, that is by removing the need
for Rosetta 2 translation at runtime.

Groundwork enabling the present payload:
- DataDog/integrations-core#20455
- DataDog/integrations-core#20577
- DataDog/integrations-core#20699
- DataDog/omnibus-ruby#239
- DataDog/omnibus-ruby#240
- #38498
- #39082
- DataDog/agent-release-management#364

Some earlier scouting:
- DataDog/integrations-core#19193 (credits to @Pythyu)
- DataDog/omnibus-software#553 (credits to @Pythyu)
- #31787 (credits to @Pythyu)
- DataDog/ci-platform-machine-images#321

Some follow-ups:
- DataDog/datadog-agent-dev#137
- DataDog/datadog-agent-buildimages#888
- #38446
- DataDog/integrations-core#20812
- DataDog/integrations-core#20815
- DataDog/integrations-core#20645
- Datadog/ci-platform-machine-images#343
- DataDog/agent-release-management#365
rdesgroppes added a commit to DataDog/ansible-datadog that referenced this pull request Oct 8, 2025
@rdesgroppes @fanny-jiang
[ABLD-27] Prepare deployment of Agent v7.70+ for macOS (arm64/x86_64) (…
6616ffc 
…#664)

* Prepare deployment of Agent v7.70+ for macOS (arm64/x86_64)

This change is meant to take into account how agent deliverables for
macOS embed architecture qualifiers (name & metadata) starting from
version 7.70 and some of the practical implications.

A word of caution: I'm far from being fluent with `ansible`, let alone
with the local setup and conventions, so the change is unlikely to work
out of the box nor to be exhaustive.

Following upstream changes might be of interest to reviewers:
- architecture qualifiers:
  - DataDog/omnibus-ruby#240
  - DataDog/agent-release-management#364
- new build for AArch64/ARM64:
  - DataDog/datadog-agent#37676
  - DataDog/agent-release-management#365

* Fix CI-reported failures

Altogether:
- use method `GET` w/ `return_content: false` instead of `HEAD` (not
  necessarily implemented)
- move `uri`-unsupported `retries` to upper level
- pass `status_code: 200` to populate
  `datadog_macos_agent7_latest_arch_check.status`
- make `ansible-lint` happy (at the expense of readability)

* Apply Fanny's suggestion

To prevent an undefined variable error.

Co-authored-by: Fanny Jiang <fanny.jiang@datadoghq.com>

* Make `ansible-lint` happy again

Use a folded block scalar to overcome the 160-char line limitation.

---------

Co-authored-by: Fanny Jiang <fanny.jiang@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aiuto aiuto aiuto approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

7.70.0

Development

Successfully merging this pull request may close these issues.

4 participants

@rdesgroppes @aiuto @tduffield