Skip to content

feat: add archive extraction and custom URL support to install_tools framework #326

@endavis

Description

@endavis

Description

The install_tools.py framework currently only supports downloading single binary files from GitHub releases via download_github_release_binary(). Many tools distribute as tar.gz or zip archives, requiring custom install functions in downstream projects.

Current Limitations

  1. No tar.gz support — Tools like age distribute as age-v{version}-linux-amd64.tar.gz containing multiple binaries (age, age-keygen)
  2. No zip support — Tools like Terraform and OpenTofu distribute as zip archives containing a single binary
  3. GitHub-only download URLsdownload_github_release_binary() constructs URLs from repo + version, but Terraform uses releases.hashicorp.com instead of GitHub releases

Downstream Impact

InfraFoundry needs age, sops, terraform, opentofu, and ansible. Only sops and direnv work with the generic framework — the rest require ~100 lines of custom install functions because of archive extraction.

Proposed Solution

Extend install_tool() and create_install_task() with two new parameters:

1. extract_binaries — archive extraction support

create_install_task(
    name="age",
    repo="FiloSottile/age",
    asset_patterns={"linux": "age-v{version}-linux-amd64.tar.gz"},
    extract_binaries=["age", "age-keygen"],  # Extract these from archive
)

create_install_task(
    name="terraform",
    repo="hashicorp/terraform",
    asset_patterns={"linux": "terraform_{version}_linux_{arch}.zip"},
    extract_binaries=["terraform"],  # Extract from zip
)

When extract_binaries is set:

  • Download the archive to a temp location
  • Detect format from extension (.tar.gz → tarfile, .zip → zipfile)
  • Extract only the named binaries to ~/.local/bin/
  • chmod 755
  • Clean up temp archive

When extract_binaries is not set (current behavior):

  • Download directly as a binary (existing download_github_release_binary behavior)

2. url_template — custom download URLs

create_install_task(
    name="terraform",
    repo="hashicorp/terraform",  # Still used for version lookup via GitHub API
    url_template="https://releases.hashicorp.com/terraform/{version}/terraform_{version}_{os}_{arch}.zip",
    extract_binaries=["terraform"],
)

When url_template is set:

  • Use it instead of constructing a GitHub releases URL
  • Supports {version}, {os}, {arch} placeholders
  • {os} = platform.system().lower() (linux, darwin)
  • {arch} = mapped from platform.machine() (x86_64→amd64, aarch64→arm64)
  • repo is still used for get_latest_github_release() version lookup

Implementation Details

Add a download_github_release_archive() function alongside the existing download_github_release_binary():

def download_and_extract_archive(
    url: str, extract_binaries: list[str], dest_dir: Path
) -> list[Path]:
    """Download an archive and extract specific binaries."""
    # Detect format from URL extension
    # Download to temp file
    # Extract named binaries using tarfile/zipfile
    # chmod 755
    # Return paths to extracted binaries

Update install_tool() to use the new parameters:

  • If extract_binaries is set → download archive, extract
  • If url_template is set → use custom URL instead of GitHub releases URL
  • Otherwise → existing binary download behavior

Success Criteria

  • create_install_task() supports extract_binaries parameter
  • create_install_task() supports url_template parameter
  • tar.gz and zip archives handled automatically based on file extension
  • Existing binary-only installs still work unchanged
  • Architecture mapping (x86_64→amd64, aarch64→arm64) included
  • Tests added for archive extraction
  • Documentation updated

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions