Skip to content

Conversation

@AliDatadog
Copy link

Description

Instead of using git clone, we now fetch the right tag directly. It is impossible to clone to a specific revision so we need to fetch and checkout.
Used as part of this PR https://gitlab.ddbuild.io/DataDog/datadog-agent/-/pipelines/53787106.
Job logs show the new log https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/777524284/raw.


/cc @chef/omnibus-maintainers <- This ensures the Omnibus Maintainers team are notified to review this PR.

@AliDatadog AliDatadog requested a review from a team as a code owner January 23, 2025 13:16
@AliDatadog AliDatadog changed the title Clone directly to the right tag [BARX-775] Clone directly to the right tag Jan 23, 2025
git("checkout #{resolved_version}")
if clone_submodules?
git("submodule update --init --recursive")
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about git clone --single-branch --branch <branch> ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it doesn't work with revisions

Copy link
Member

@FlorentClarret FlorentClarret Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❯ git clone --single-branch --branch 7.61.0 git@github.com:DataDog/omnibus-ruby.git
Cloning into 'omnibus-ruby'...
remote: Enumerating objects: 15051, done.
remote: Counting objects: 100% (2139/2139), done.
remote: Compressing objects: 100% (311/311), done.
remote: Total 15051 (delta 1978), reused 1907 (delta 1828), pack-reused 12912 (from 2)
Receiving objects: 100% (15051/15051), 32.65 MiB | 16.85 MiB/s, done.
Resolving deltas: 100% (9421/9421), done.
Note: switching to 'd365e483ee05a13e55eeb5208d11452c5b65afbb'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah it works with tags but not hashes 😢

@AliDatadog AliDatadog changed the title [BARX-775] Clone directly to the right tag [BARX-775] Clone directly to the right revision Jan 23, 2025
Copy link
Member

@FlorentClarret FlorentClarret left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay let's try that

@AliDatadog AliDatadog merged commit 49ba118 into datadog-5.5.0 Jan 23, 2025
3 checks passed
@AliDatadog AliDatadog deleted the ali/clone-right-tag branch January 23, 2025 14:32
Pythyu added a commit that referenced this pull request Mar 21, 2025
* Support an explicit target install location for macOS packages (#215)

* configure: use the expected default value for windows prefix (#216)

* use the expected default value for windows prefix

* allow --build to be ommited from configure args

* replace unless with if not

* Always ship source/source offers (#217)

* software: always ship the source or source offer

Even when restoring the build from cache

* attempt clean instead of deployed

deploy isn't available for all fetchers

* [BARX-775] Clone directly to the right revision (#218)

* Replace git clone by git fetch to the right version

* add log

* feat(security): Pin github actions (#219)

* add a disable_version_manifest accessor (#220)

* add a disable_version_manifest accessor

* allow version manifests to be disabled

* remove an extra blank line to keep rubocop happy

* fix syntax

* project: fix typo preventing overriding version manifest file (#222)

* project: provide a default value for disable_version_manifest (#221)

* cli: rely on the existing use_s3_caching option to enable caching (#223)

* cli: rely on the existing use_s3_caching option to enable caching

* cli: remove now unused populate_s3_cache option

* don't attempt to populate cache when we only have read access

* add a comment to clarify the logic

* feat(bump): Remove homemade workflow to use renovate instead (#224)

* [PathFetcher] Improve speed by removing unnecessary digest operations (#225)

Improves the PathFetcher class' speed by removing some digest operations that are not neccesary, since they are quite costly (10-15s for a large directory like the datadog-agent repository).

Specifically:
- if the target directory (in the omnibus project directories) is empty, force a fetch without computing the digests of the source and the target: in this case, we know a fetch is required.
- after a fetch, do not pre-compute the digest of the target directory. In practice, it is a useless operation because this digest is never reused in the same omnibus run.

I expect these improvements to reduce the duration of all Agent builds by about 30s.

---------

Co-authored-by: Alex Lopez <alex.lopez.zorzano@gmail.com>
Co-authored-by: Hugo Beauzée-Luyssen <hugo.beauzee@datadoghq.com>
Co-authored-by: AliDatadog <125997632+AliDatadog@users.noreply.github.com>
Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>
Co-authored-by: Kylian Serrania <kylian.serrania@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants