From b2ee9c458d2eaef90dc90ecea8956f873f4a7536 Mon Sep 17 00:00:00 2001 From: Mo King Date: Wed, 4 Jun 2025 12:17:09 -0400 Subject: [PATCH 1/3] Add GH build args --- serverless/workers/github-integration.mdx | 232 +++++++++++++--------- 1 file changed, 134 insertions(+), 98 deletions(-) diff --git a/serverless/workers/github-integration.mdx b/serverless/workers/github-integration.mdx index 1a0a1578..a356ebe2 100644 --- a/serverless/workers/github-integration.mdx +++ b/serverless/workers/github-integration.mdx @@ -13,25 +13,20 @@ RunPod's GitHub integration simplifies your workflow by pulling your code and Do To deploy a worker from GitHub, you need: -* A working [handler function](/serverless/workers/handler-functions) in a GitHub repository. -* A Dockerfile in your repository. See [Creating a Dockerfile](/serverless/workers/deploy#creating-a-dockerfile) for details. -* A GitHub account. +- A working [handler function](/serverless/workers/handler-functions) in a GitHub repository. +- A Dockerfile in your repository. See [Creating a Dockerfile](/serverless/workers/deploy#creating-a-dockerfile) for details. +- A GitHub account. ## Authorizing RunPod with GitHub Before deploying from GitHub, you need to authorize RunPod to access your repositories: 1. Open the [settings page](http://runpod.io/console/user/settings) in the RunPod console. - 2. Find the **GitHub** card under **Connections** and click **Connect**. - 3. Sign in using the GitHub authorization flow. This will open your GitHub account settings page. - 4. Choose which repositories RunPod can access: - - * **All repositories:** Access to all current and future repositories. - * **Only select repositories:** Choose specific repositories. - + - **All repositories:** Access to all current and future repositories. + - **Only select repositories:** Choose specific repositories. 5. Click **Save**. You can manage this connection using RunPod settings or GitHub account settings, in the **Applications** tab. @@ -41,45 +36,87 @@ You can manage this connection using RunPod settings or GitHub account settings, To deploy a worker from a GitHub repository: 1. Go to the [Serverless section](https://www.runpod.io/console/serverless) of the RunPod console - 2. Click **New Endpoint** - 3. Under **Custom Source**, select **GitHub Repo**, then click **Next** - 4. Use the search bar or menu to select the repository containing your code. This menu is populated with all repos connected to your account (repos you've forked/created, or owned by your GitHub organizations). - 5. Configure your deployment options: - - * **Branch:** Select which branch to deploy from. - * **Dockerfile:** Specify the path to your Dockerfile (if not in root). - + - **Branch:** Select which branch to deploy from. + - **Dockerfile:** Specify the path to your Dockerfile (if not in root). 6. (Optional) Enter a custom name for your endpoint. - 7. Configure your compute resources (GPU type, worker count, etc.). - 8. Click **Create Endpoint**. RunPod will build your Docker image and deploy it to your endpoint automatically. You'll be redirected to the endpoint details page when complete. +## GitHub build arguments + +When deploying a worker from a GitHub repo, you can use specify build arguments for your Dockerfile using `ARG`, allowing you to parameterize values in your Dockerfile instructions. Unlike environment variables, these variables do not persist into the final container, so they won't affect your handler function code. + +You can specify build arguments either in your Dockerfile, or using the RunPod UI. + +:::note + +For comprehensive information about using `ARG`, refer to the [official Docker documention](https://docs.docker.com/build/building/variables/). + +::: + +### Using build arguments in your Dockerfile + +To use build arguments in your Dockerfile, declare variables using the `ARG` keyword, then reference them using `$` syntax. + +For example: + +```dockerfile +# Base image from RunPod's container registry +ARG CUDA_VERSION=11.8.0 +ARG BASE_VERSION=0.6.3 + +# Use build arguments to specify the base image version +FROM runpod/base:${BASE_VERSION}-cuda${CUDA_VERSION} +``` + +You must declare an ARG in your Dockerfile before you can reference it, even if you've defined it in the RunPod UI. + +### Setting build arguments in the RunPod UI + +To customize your build arguments without modifying your Dockerfile, you can specify them using the RunPod console. + +When [deploying a worker](#deploying-from-github) from GitHub: + +1. During the endpoint configuration process, find the **Build Args** section (below **Public Environment Variables**). +2. Enter key-value pairs for your build arguments. +3. These will be passed to your Docker build process. + +You must declare an `ARG` in your Dockerfile before you can reference it, even if you've defined it in the RunPod UI: + +```dockerfile +# Build arguments defined in the UI must still be declared in your Dockerfile +ARG CUDA_VERSION +ARG BASE_VERSION + +# Use build arguments to specify the base image version +FROM runpod/base:${BASE_VERSION}-cuda${CUDA_VERSION} +``` + ## Monitoring build status You can monitor your build status in the **Builds** tab of your endpoint detail page. Builds progress through these statuses: -| Status | Description | -| --------- | --------------------------------------------------- | -| Pending | RunPod is scheduling the build. | -| Building | RunPod is building your container. | +| Status | Description | +|--------|-------------| +| Pending | RunPod is scheduling the build. | +| Building | RunPod is building your container. | | Uploading | RunPod is uploading your container to the registry. | -| Testing | RunPod is testing your Serverless worker. | -| Completed | RunPod completed the build and upload. | -| Failed | Something went wrong (check build logs). | +| Testing | RunPod is testing your Serverless worker. | +| Completed | RunPod completed the build and upload. | +| Failed | Something went wrong (check build logs). | ## Managing multiple environments GitHub integration enables streamlined development workflows by supporting multiple environments: -* Production endpoint tracking the `main` branch. -* Staging endpoint tracking the `dev` branch. +- Production endpoint tracking the `main` branch. +- Staging endpoint tracking the `dev` branch. To set up multiple environments: @@ -97,67 +134,66 @@ You can enhance your workflow with GitHub Actions for testing before deployment: 1. Create a workflow file at `.github/workflows/test-and-deploy.yml`: - ```yml - name: Test and Deploy - - on: - push: - branches: [ main ] - pull_request: - branches: [ main ] - - jobs: - test-and-deploy: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v3 - - - name: Build and push Docker image - uses: docker/build-push-action@v4 - with: - context: . - push: true - tags: [DOCKER_USERNAME]/[WORKER_NAME]:${{ github.sha }} - - - name: Run Tests - uses: runpod/runpod-test-runner@v1 - with: - image-tag: [DOCKER_USERNAME]/[WORKER_NAME]:${{ github.sha }} - runpod-api-key: ${{ secrets.RUNPOD_API_KEY }} # Add your API key to a GitHub secret - test-filename: .github/tests.json - request-timeout: 300 - ``` + ```yaml + name: Test and Deploy - - TIP + on: + push: + branches: [ main ] + pull_request: + branches: [ main ] - To add your RunPod API key to a GitHub secret, see [Using secrets in GitHub Actions](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions). - - + jobs: + test-and-deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Build and push Docker image + uses: docker/build-push-action@v4 + with: + context: . + push: true + tags: [DOCKER_USERNAME]/[WORKER_NAME]:${{ github.sha }} + + - name: Run Tests + uses: runpod/runpod-test-runner@v1 + with: + image-tag: [DOCKER_USERNAME]/[WORKER_NAME]:${{ github.sha }} + runpod-api-key: ${{ secrets.RUNPOD_API_KEY }} # Add your API key to a GitHub secret + test-filename: .github/tests.json + request-timeout: 300 + ``` + + :::tip + + To add your RunPod API key to a GitHub secret, see [Using secrets in GitHub Actions](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions). + + ::: 2. Create test cases for your repository at `.github/tests.json`: - ```json - [ - { - "input": { - "prompt": "Test input 1" - }, - "expected_output": { - "status": "COMPLETED" - } - }, - { - "input": { - "prompt": "Test input 2", - "parameter": "value" + ```json + [ + { + "input": { + "prompt": "Test input 1" + }, + "expected_output": { + "status": "COMPLETED" + } }, - "expected_output": { - "status": "COMPLETED" + { + "input": { + "prompt": "Test input 2", + "parameter": "value" + }, + "expected_output": { + "status": "COMPLETED" + } } - } - ] - ``` + ] + ``` ## Known limitations @@ -167,12 +203,12 @@ RunPod's GitHub integration has a few limitations to be aware of: When using GitHub integration with RunPod, be aware of these important limitations: -* **Build time limit**: Builds must complete within 160 minutes (2.5 hours). Optimize your Dockerfile for efficiency with large images to avoid timeouts. -* **Image size restriction**: Docker images cannot exceed 100 GB. Plan your image requirements accordingly, particularly when including large model weights or dependencies. -* **Base image limitations**: The integration doesn't support privately hosted images as base images. Consider incorporating essential components directly into your Dockerfile instead. -* **Hardware-specific builds**: Builds requiring GPU access during construction (such as those using GPU-compiled versions of libraries like `bitsandbytes`) are not supported. -* **Platform exclusivity**: Images built through RunPod's image builder service are designed exclusively for RunPod's infrastructure and cannot be pulled or executed on other platforms. -* **Single GitHub connection**: Each RunPod account can link to only one GitHub account. This connection cannot be shared among team members, requiring separate RunPod accounts for collaborative projects. +- **Build time limit**: Builds must complete within 160 minutes (2.5 hours). Optimize your Dockerfile for efficiency with large images to avoid timeouts. +- **Image size restriction**: Docker images cannot exceed 100 GB. Plan your image requirements accordingly, particularly when including large model weights or dependencies. +- **Base image limitations**: The integration doesn't support privately hosted images as base images. Consider incorporating essential components directly into your Dockerfile instead. +- **Hardware-specific builds**: Builds requiring GPU access during construction (such as those using GPU-compiled versions of libraries like `bitsandbytes`) are not supported. +- **Platform exclusivity**: Images built through RunPod's image builder service are designed exclusively for RunPod's infrastructure and cannot be pulled or executed on other platforms. +- **Single GitHub connection**: Each RunPod account can link to only one GitHub account. This connection cannot be shared among team members, requiring separate RunPod accounts for collaborative projects. ## Disconnecting GitHub @@ -188,17 +224,17 @@ To disconnect your GitHub account from RunPod: If your worker fails to deploy or process requests: -* Check the build logs in the RunPod console for error messages. -* Verify your Dockerfile is properly configured. -* Ensure your handler function works correctly in local testing. -* Check that your repository structure matches what's expected in your Dockerfile. -* Verify you have the necessary permissions on the GitHub repository. +- Check the build logs in the RunPod console for error messages. +- Verify your Dockerfile is properly configured. +- Ensure your handler function works correctly in local testing. +- Check that your repository structure matches what's expected in your Dockerfile. +- Verify you have the necessary permissions on the GitHub repository. ## Next steps After deploying your worker from GitHub, you can: -* [Send API requests to your endpoint.](/serverless/endpoints/send-requests) -* [Create more advanced handler functions.](/serverless/workers/handler-functions) -* [Optimize your endpoint configurations.](/serverless/endpoints/endpoint-configurations) -* [Learn how to deploy workers from Docker Hub.](/serverless/workers/deploy) +- [Send API requests to your endpoint.](/serverless/endpoints/send-requests) +- [Create more advanced handler functions.](/serverless/workers/handler-functions) +- [Optimize your endpoint configurations.](/serverless/endpoints/endpoint-configurations) +- [Learn how to deploy workers from Docker Hub.](/serverless/workers/deploy) \ No newline at end of file From ff5f9f265caff4a62e50c9812d87667f69ea113c Mon Sep 17 00:00:00 2001 From: Mo King Date: Wed, 4 Jun 2025 12:20:21 -0400 Subject: [PATCH 2/3] Fix tips --- serverless/workers/github-integration.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/serverless/workers/github-integration.mdx b/serverless/workers/github-integration.mdx index a356ebe2..b0af69d4 100644 --- a/serverless/workers/github-integration.mdx +++ b/serverless/workers/github-integration.mdx @@ -54,11 +54,11 @@ When deploying a worker from a GitHub repo, you can use specify build arguments You can specify build arguments either in your Dockerfile, or using the RunPod UI. -:::note + For comprehensive information about using `ARG`, refer to the [official Docker documention](https://docs.docker.com/build/building/variables/). -::: + ### Using build arguments in your Dockerfile @@ -165,11 +165,11 @@ You can enhance your workflow with GitHub Actions for testing before deployment: request-timeout: 300 ``` - :::tip + To add your RunPod API key to a GitHub secret, see [Using secrets in GitHub Actions](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions). - ::: + 2. Create test cases for your repository at `.github/tests.json`: From bb0769a26d1b57eec0bc0b809017081ca0218797 Mon Sep 17 00:00:00 2001 From: Mo King Date: Wed, 4 Jun 2025 12:22:12 -0400 Subject: [PATCH 3/3] Update --- serverless/workers/github-integration.mdx | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/serverless/workers/github-integration.mdx b/serverless/workers/github-integration.mdx index b0af69d4..e223856d 100644 --- a/serverless/workers/github-integration.mdx +++ b/serverless/workers/github-integration.mdx @@ -195,13 +195,9 @@ You can enhance your workflow with GitHub Actions for testing before deployment: ] ``` -## Known limitations +## Limitations -RunPod's GitHub integration has a few limitations to be aware of: - -## GitHub integration limitations - -When using GitHub integration with RunPod, be aware of these important limitations: +RunPod's GitHub integration has a few limitations you should be aware of: - **Build time limit**: Builds must complete within 160 minutes (2.5 hours). Optimize your Dockerfile for efficiency with large images to avoid timeouts. - **Image size restriction**: Docker images cannot exceed 100 GB. Plan your image requirements accordingly, particularly when including large model weights or dependencies.