diff --git a/.docs/cve_scan.md b/.docs/cve_scan.md index e7bafd5..f45e565 100644 --- a/.docs/cve_scan.md +++ b/.docs/cve_scan.md @@ -1,70 +1,70 @@ # CVE Scan CI Integration ## Description -This CI file will run a Trivy CVE scan against the module images and its submodule images, and then upload the reports to DefectDojo. -The script will detect release or dev tag of module image is used and then construct registry location by itself. If your module located in registries by not standart paths - you may want to define custom path by *module_prod_registry_custom_path* and *module_dev_registry_custom_path* variables. -CI Use cases: -- Scan by scheduler - - Scan main branch and several latest releases 2-3 times a week -- Scan on PR - - Scan images on pull request to check if no new vulnerabilities are present or to ensure if they are closed. -- Manual scan - - Scan specified release by entering semver minor version of target release in *release_branch* variable. - - Scan main branch and several latest releases by setting *scan_several_lastest_releases* to "true" and optionally defining amount of latest minor releases by setting a number into *latest_releases_amount* variable. - - Scan only main branch just by running pipeline - -## Variables - -### workflow_dispatch level -``` -release_branch - Optional. Set minor version of release you want to scan. e.g.: 1.23 -scan_several_lastest_releases - Optional. Whether to scan last several releases or not. true/false. For scheduled pipelines it is always true. Default is: false. -latest_releases_amount - Optional. Number of latest releases to scan. Default is: 3 -severity - Optional. Vulnerabilities severity to scan. Default is: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL -``` +This CI runs a Trivy CVE and License scan against module images and their submodule images, then uploads reports to DefectDojo. +The action clones the [cve-scan](https://github.com/deckhouse/cve-scan) scripts and runs them. The script resolves the registry and tag (release vs dev) from *source_tag* and *case*. If your module uses non-standard paths in registries, set *module_prod_registry_custom_path* and *module_dev_registry_custom_path*. -### Job level +**CI use cases:** +- **Scheduled** — scan main and several latest releases (e.g. 2–3 times a week) +- **On PR** — scan images for the PR to ensure no new vulnerabilities or that known ones are closed +- **Manual** — scan a specific release (*source_tag* as semver minor, e.g. `1.23`), or main + several latest releases (*scan_several_latest_releases* = true, *latest_releases_amount*), or only main by running the workflow -#### Mandatory -``` -tag - module image tag -module_name - module name -dd_url - URL to defectDojo -dd_token - token of defectDojo to upload reports -deckhouse_private_repo - url to private repo -prod_registry - Must be deckhouse prod registry, used to get trivy databases and release images -prod_registry_user - Username to log in to deckhouse prod registry -prod_registry_password - Password to log in to deckhouse prod registry -dev_registry - Must be deckhouse dev registry, used to get dev images -dev_registry_user - Username to log in to deckhouse dev registry -dev_registry_password - Password to log in to deckhouse dev registry -``` -#### Optional -``` -scan_several_lastest_releases - true/false. Whether to scan last several releases or not. For scheduled pipelines override will not work as value is always true -latest_releases_amount - Number of latest minor releases to scan. Latest patch versions of latest N minor versions will be taken. Default is: 3 -severity - Vulnerabilities severity to scan. Default is: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL -module_prod_registry_custom_path - Module custom path in prod registry. Example: flant/modules -module_dev_registry_custom_path - Module custom path in dev registry. Example: flant/modules +## Action reference +Use the reusable action: +```yaml +deckhouse/modules-actions/cve_scan@main ``` +(Replace `main` with the branch or tag your repo uses.) -### GitHub Masked variables -``` -dd_url - URL to defectDojo -dd_token - token of defectDojo to upload reports -deckhouse_private_repo - url to private repo -prod_registry - Must be deckhouse prod registry, used to get trivy databases and release images -prod_registry_user - Username to log in to deckhouse prod registry -prod_registry_password - Password to log in to deckhouse prod registry -dev_registry - Must be deckhouse dev registry, used to get dev images -dev_registry_user - Username to log in to deckhouse dev registry -dev_registry_password - Password to log in to deckhouse dev registry -``` +## Inputs + +### workflow_dispatch (optional inputs) +| Input | Description | +|-------|-------------| +| `release_branch` / source_tag | Optional. Minor version to scan, e.g. `1.23`, or leave empty for default branch | +| `scan_several_latest_releases` | Optional. Scan several latest releases. `true`/`false`. For scheduled runs it is always true. Default: `false` | +| `latest_releases_amount` | Optional. Number of latest releases to scan. Default: `3` | +| `release_in_dev` | Optional. If `true`, release tag is taken from dev registry. Default: `false` | +| `trivy_reports_log_output` | Optional. Trivy report verbosity in logs: `0` — off, `1` — CVE/FS only, `2` — CVE + License. Default: `1` | +| `external_module_name` | Optional. For *case* = "External Modules": the module name whose tag is scanned | + +### Job level — mandatory (action inputs) +| Input | Description | +|-------|-------------| +| `source_tag` | Tag to scan: e.g. `main`, `v1.74.3`, `pr123`, `release-1.73` | +| `case` | Scan type: `deckhouse` \| `External Modules` \| `CSE` | +| `dd_url` | DefectDojo API URL | +| `dd_token` | DefectDojo API token | +| `prod_registry` | Prod registry host (e.g. for release images and Trivy DB) | +| `prod_registry_user` | Prod registry username | +| `prod_registry_password` | Prod registry password | +| `dev_registry` | Dev registry host (e.g. for branch/PR images) | +| `dev_registry_user` | Dev registry username | +| `dev_registry_password` | Dev registry password | +| `codeowners_repo_token` | Token for CODEOWNERS configmap | +| `deckhouse_private_repo` | Deckhouse private repo URL (e.g. GitLab for Trivy binary and configmap) | +| `cve_test_repo_git` | cve-scan repo clone URL (SSH) | +| `cve_ssh_private_key` | SSH key for cloning cve-scan repo | + +### Job level — optional +| Input | Description | +|-------|-------------| +| `external_module_name` | Required when *case* = "External Modules". Module name in registry path | +| `scan_several_latest_releases` | `True`/`False`. For scheduled runs this is typically overridden to true | +| `latest_releases_amount` | Number of latest minor releases to scan. Default: `3` | +| `release_in_dev` | If true, look for release tag in dev registry. Default: `False` | +| `trivy_reports_log_output` | `0` \| `1` \| `2`. Default: `1` | +| `module_prod_registry_custom_path` | Custom path for module in prod registry. Default: `deckhouse/fe/modules` | +| `module_dev_registry_custom_path` | Custom path for module in dev registry. Default: `sys/deckhouse-oss/modules` | +| `workdir` | Working directory for scan artifacts. Default: `cve-scan` | + +### Sensitive inputs (mask in logs) +`dd_url`, `dd_token`, `deckhouse_private_repo`, registry credentials, `codeowners_repo_token`, `cve_ssh_private_key`. ## How to include -Set trigger rules with scheduler, manual and pull requests in your cve scan CI file: -``` +### Triggers +```yaml on: schedule: - cron: '0 01 * * 0,3' @@ -76,62 +76,116 @@ on: workflow_dispatch: inputs: release_branch: - description: 'Optional. Set minor version of release you want to scan. e.g.: 1.23' + description: 'Optional. Minor version to scan. e.g.: 1.23' required: false - scan_several_lastest_releases: - description: 'Optional. Whether to scan last several releases or not. true/false. For scheduled pipelines it is always true. Default is: false.' + scan_several_latest_releases: + description: 'Optional. Scan several latest releases. true/false. Default: false.' required: false latest_releases_amount: - description: 'Optional. Number of latest releases to scan. Default is: 3' + description: 'Optional. Number of latest releases to scan. Default: 3' + required: false + release_in_dev: + description: 'If true, release tag is taken from dev registry. Default: false' required: false - severity: - description: 'Optional. Vulnerabilities severity to scan. Default is: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL' + trivy_reports_log_output: + description: 'Optional. 0=off, 1=CVE only, 2=CVE+License. Default: 1' + required: false + external_module_name: + description: 'For External Modules: module name to scan' required: false ``` -Put the following jobs into required place of you GitHub Action file (usually after build step/job if exist): -``` +### Example: External Modules (e.g. csi-ceph) +Secrets are often provided via BOB (`hashicorp/vault-action`); use `steps.secrets.outputs.*` for registry and DefectDojo credentials. + +```yaml cve_scan_on_pr: if: github.event_name == 'pull_request' name: CVE scan for PR + runs-on: [self-hosted, regular] needs: [build_dev] + permissions: + contents: read + id-token: write steps: - uses: actions/checkout@v4 - - uses: deckhouse/modules-actions/cve_scan@v3 + # Optional: split repo name, import secrets from Vault (see your repo’s pattern) + - uses: deckhouse/modules-actions/cve_scan@main with: - tag: pr${{ github.event.number }} - module_name: ${{ vars.MODULE_NAME }} - dd_url: ${{ secrets.DEFECTDOJO_HOST }} - dd_token: ${{ secrets.DEFECTDOJO_API_TOKEN }} - prod_registry: ${{ secrets.PROD_READ_REGISTRY }} - prod_registry_user: ${{ secrets.PROD_MODULES_READ_REGISTRY_USER }} - prod_registry_password: ${{ secrets.PROD_MODULES_READ_REGISTRY_PASSWORD }} - dev_registry: ${{ secrets.DEV_REGISTRY }} - dev_registry_user: ${{ secrets.DEV_MODULES_REGISTRY_USER }} - dev_registry_password: ${{ secrets.DEV_MODULES_REGISTRY_PASSWORD }} - deckhouse_private_repo: ${{ secrets.DECKHOUSE_PRIVATE_REPO }} + source_tag: 'pr${{ github.event.number }}' + case: "External Modules" + external_module_name: ${{ vars.MODULE_NAME }} + dd_url: ${{ steps.secrets.outputs.DEFECTDOJO_URL }} + dd_token: ${{ steps.secrets.outputs.DEFECTDOJO_API_TOKEN }} + prod_registry: ${{ steps.secrets.outputs.PROD_READ_REGISTRY }} + prod_registry_user: ${{ steps.secrets.outputs.PROD_READ_REGISTRY_USER }} + prod_registry_password: ${{ steps.secrets.outputs.PROD_READ_REGISTRY_PASSWORD }} + dev_registry: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_HOST }} + dev_registry_user: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_USER }} + dev_registry_password: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_PASSWORD }} + deckhouse_private_repo: ${{ steps.secrets.outputs.DECKHOUSE_PRIVATE_REPO }} + codeowners_repo_token: ${{ steps.secrets.outputs.CODEOWNERS_REPO_TOKEN }} + cve_test_repo_git: ${{ steps.secrets.outputs.CVE_TEST_REPO_GIT }} + cve_ssh_private_key: ${{ steps.secrets.outputs.CVE_TEST_SSH_PRIVATE_KEY }} + trivy_reports_log_output: "1" + cve_scan: if: github.event_name != 'pull_request' name: Regular CVE scan + runs-on: [self-hosted, regular] + needs: [build_dev] + permissions: + contents: read + id-token: write steps: - uses: actions/checkout@v4 - - uses: deckhouse/modules-actions/cve_scan@v3 + # Optional: import secrets from Vault + - uses: deckhouse/modules-actions/cve_scan@main with: - tag: ${{ github.event.inputs.release_branch || github.event.repository.default_branch }} - module_name: ${{ vars.MODULE_NAME }} - dd_url: ${{ secrets.DEFECTDOJO_HOST }} - dd_token: ${{ secrets.DEFECTDOJO_API_TOKEN }} - prod_registry: ${{ secrets.PROD_READ_REGISTRY }} - prod_registry_user: ${{ secrets.PROD_MODULES_READ_REGISTRY_USER }} - prod_registry_password: ${{ secrets.PROD_MODULES_READ_REGISTRY_PASSWORD }} - dev_registry: ${{ secrets.DEV_REGISTRY }} - dev_registry_user: ${{ secrets.DEV_MODULES_REGISTRY_USER }} - dev_registry_password: ${{ secrets.DEV_MODULES_REGISTRY_PASSWORD }} - deckhouse_private_repo: ${{ secrets.DECKHOUSE_PRIVATE_REPO }} - scan_several_lastest_releases: ${{ github.event.inputs.scan_several_lastest_releases }} + source_tag: ${{ github.event.inputs.release_branch || github.event.repository.default_branch }} + case: "External Modules" + external_module_name: ${{ vars.MODULE_NAME }} + dd_url: ${{ steps.secrets.outputs.DEFECTDOJO_URL }} + dd_token: ${{ steps.secrets.outputs.DEFECTDOJO_API_TOKEN }} + prod_registry: ${{ steps.secrets.outputs.PROD_READ_REGISTRY }} + prod_registry_user: ${{ steps.secrets.outputs.PROD_READ_REGISTRY_USER }} + prod_registry_password: ${{ steps.secrets.outputs.PROD_READ_REGISTRY_PASSWORD }} + dev_registry: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_HOST }} + dev_registry_user: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_USER }} + dev_registry_password: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_PASSWORD }} + deckhouse_private_repo: ${{ steps.secrets.outputs.DECKHOUSE_PRIVATE_REPO }} + scan_several_latest_releases: ${{ github.event.inputs.scan_several_latest_releases || 'True' }} latest_releases_amount: ${{ github.event.inputs.latest_releases_amount || '3' }} - severity: ${{ github.event.inputs.severity }} + release_in_dev: ${{ github.event.inputs.release_in_dev || 'False' }} + codeowners_repo_token: ${{ steps.secrets.outputs.CODEOWNERS_REPO_TOKEN }} + cve_test_repo_git: ${{ steps.secrets.outputs.CVE_TEST_REPO_GIT }} + cve_ssh_private_key: ${{ steps.secrets.outputs.CVE_TEST_SSH_PRIVATE_KEY }} + trivy_reports_log_output: "1" ``` -Usage example can be found [here](../.examples/cve_scan.yml) +### Example: Deckhouse core (case: deckhouse) +For the main Deckhouse repo, *case* is `deckhouse` and *source_tag* is set from workflow (e.g. from a previous step like `steps.scan_type.outputs.tag`). Prod registry credentials may come from repo secrets or BOB. + +```yaml + - uses: deckhouse/modules-actions/cve_scan@new_cve_scan + with: + source_tag: ${{ steps.scan_type.outputs.tag }} + case: "deckhouse" + prod_registry: ${{ secrets.DECKHOUSE_REGISTRY_READ_HOST }} + prod_registry_user: ${{ steps.secrets.outputs.PROD_READ_REGISTRY_USER }} + prod_registry_password: ${{ steps.secrets.outputs.PROD_READ_REGISTRY_PASSWORD }} + dev_registry: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_HOST }} + dev_registry_user: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_USER }} + dev_registry_password: ${{ steps.secrets.outputs.DECKHOUSE_DEV_REGISTRY_PASSWORD }} + codeowners_repo_token: ${{ steps.secrets.outputs.CODEOWNERS_REPO_TOKEN }} + deckhouse_private_repo: ${{ steps.secrets.outputs.DECKHOUSE_PRIVATE_REPO }} + dd_url: ${{ steps.secrets.outputs.DD_URL }} + dd_token: ${{ steps.secrets.outputs.DD_TOKEN }} + cve_test_repo_git: ${{ steps.secrets.outputs.CVE_TEST_REPO_GIT }} + cve_ssh_private_key: ${{ steps.secrets.outputs.CVE_SSH_PRIVATE_KEY }} + scan_several_latest_releases: ${{ steps.scan_type.outputs.scan_several_latest_releases }} + latest_releases_amount: ${{ steps.scan_type.outputs.latest_releases_amount }} + trivy_reports_log_output: "2" +``` +Full workflow examples: [.examples/cve_scan.yml](../.examples/cve_scan.yml); real usage: [csi-ceph](https://github.com/deckhouse/csi-ceph/blob/main/.github/workflows/trivy_image_check.yaml), [deckhouse cve-pr](https://github.com/deckhouse/deckhouse/blob/main/.github/workflows/cve-pr.yml), [deckhouse cve-weekly](https://github.com/deckhouse/deckhouse/blob/main/.github/workflows/cve-weekly.yml). diff --git a/.examples/cve_scan.yml b/.examples/cve_scan.yml index 83855e9..7014fb6 100644 --- a/.examples/cve_scan.yml +++ b/.examples/cve_scan.yml @@ -1,3 +1,9 @@ +# Example CVE/License scan workflow for a repo using deckhouse/modules-actions/cve_scan. +# For External Modules (e.g. csi-ceph): use case "External Modules" and external_module_name. +# For Deckhouse core: use case "deckhouse" and set source_tag from your workflow (e.g. steps.scan_type.outputs.tag). +# In production, secrets are often provided via hashicorp/vault-action → steps.secrets.outputs.* +# See: .docs/cve_scan.md and real usage in csi-ceph, deckhouse (cve-pr.yml, cve-weekly.yml). + name: CVE Scan on: @@ -10,67 +16,81 @@ on: - main workflow_dispatch: inputs: - tag_type: - type: choice - description: Tag type - required: false - options: - - release - - dev - tag_name: - description: "release version in semver minor format (example: 1.68) or specified tag from dev registry" + release_branch: + description: "Optional. Minor version to scan (e.g. 1.68) or leave empty for default branch" required: false - scan_several_lastest_releases: - description: 'Optional. Whether to scan last several releases or not. true/false. For scheduled pipelines it is always true. Default is: false.' + scan_several_latest_releases: + description: 'Optional. Scan several latest releases. true/false. Default: false.' required: false latest_releases_amount: - description: 'Optional. Number of latest releases to scan. Default is: 3' + description: 'Optional. Number of latest releases to scan. Default: 3' + required: false + release_in_dev: + description: 'If true, release tag is taken from dev registry. Default: false' required: false - severity: - description: 'Optional. Vulnerabilities severity to scan. Default is: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL' + trivy_reports_log_output: + description: 'Optional. 0=off, 1=CVE only, 2=CVE+License. Default: 1' + required: false + external_module_name: + description: 'For External Modules: module name whose tag will be scanned' required: false jobs: cve_scan_on_pr: if: github.event_name == 'pull_request' name: CVE scan for PR - runs-on: ubuntu-latest + runs-on: [self-hosted, regular] + permissions: + contents: read + id-token: write steps: - uses: actions/checkout@v4 - - uses: deckhouse/modules-actions/cve_scan@v5 + - uses: deckhouse/modules-actions/cve_scan@new_cve_scan with: - tag: pr${{ github.event.number }} - module_name: ${{ vars.MODULE_NAME }} - dd_url: ${{ secrets.DEFECTDOJO_HOST }} + source_tag: 'pr${{ github.event.number }}' + case: "External Modules" + external_module_name: ${{ vars.MODULE_NAME }} + dd_url: ${{ secrets.DEFECTDOJO_URL }} dd_token: ${{ secrets.DEFECTDOJO_API_TOKEN }} prod_registry: ${{ secrets.PROD_READ_REGISTRY }} - prod_registry_user: ${{ secrets.PROD_READ_REGISTRY_LOGIN }} + prod_registry_user: ${{ secrets.PROD_READ_REGISTRY_USER }} prod_registry_password: ${{ secrets.PROD_READ_REGISTRY_PASSWORD }} - dev_registry: ${{ secrets.DEV_REGISTRY }} - dev_registry_user: ${{ secrets.DEV_REGISTRY_LOGIN }} - dev_registry_password: ${{ secrets.DEV_REGISTRY_PASSWORD }} + dev_registry: ${{ secrets.DECKHOUSE_DEV_REGISTRY_HOST }} + dev_registry_user: ${{ secrets.DECKHOUSE_DEV_REGISTRY_USER }} + dev_registry_password: ${{ secrets.DECKHOUSE_DEV_REGISTRY_PASSWORD }} + codeowners_repo_token: ${{ secrets.CODEOWNERS_REPO_TOKEN }} deckhouse_private_repo: ${{ secrets.DECKHOUSE_PRIVATE_REPO }} - severity: "HIGH,CRITICAL" + cve_test_repo_git: ${{ secrets.CVE_TEST_REPO_GIT }} + cve_ssh_private_key: ${{ secrets.CVE_TEST_SSH_PRIVATE_KEY }} + trivy_reports_log_output: ${{ github.event.inputs.trivy_reports_log_output || '1' }} + cve_scan: if: github.event_name != 'pull_request' name: Regular CVE scan - runs-on: ubuntu-latest + runs-on: [self-hosted, regular] + permissions: + contents: read + id-token: write steps: - uses: actions/checkout@v4 - - uses: deckhouse/modules-actions/cve_scan@v5 + - uses: deckhouse/modules-actions/cve_scan@new_cve_scan with: - tag: ${{ github.event.inputs.tag_name || github.event.repository.default_branch }} - tag_type: ${{ github.event.inputs.tag_type }} - module_name: ${{ vars.MODULE_NAME }} - dd_url: ${{ secrets.DEFECTDOJO_HOST }} + source_tag: ${{ github.event.inputs.release_branch || github.event.repository.default_branch }} + case: "External Modules" + external_module_name: ${{ vars.MODULE_NAME }} + dd_url: ${{ secrets.DEFECTDOJO_URL }} dd_token: ${{ secrets.DEFECTDOJO_API_TOKEN }} prod_registry: ${{ secrets.PROD_READ_REGISTRY }} - prod_registry_user: ${{ secrets.PROD_READ_REGISTRY_LOGIN }} + prod_registry_user: ${{ secrets.PROD_READ_REGISTRY_USER }} prod_registry_password: ${{ secrets.PROD_READ_REGISTRY_PASSWORD }} - dev_registry: ${{ secrets.DEV_REGISTRY }} - dev_registry_user: ${{ secrets.DEV_REGISTRY_LOGIN }} - dev_registry_password: ${{ secrets.DEV_REGISTRY_PASSWORD }} + dev_registry: ${{ secrets.DECKHOUSE_DEV_REGISTRY_HOST }} + dev_registry_user: ${{ secrets.DECKHOUSE_DEV_REGISTRY_USER }} + dev_registry_password: ${{ secrets.DECKHOUSE_DEV_REGISTRY_PASSWORD }} + codeowners_repo_token: ${{ secrets.CODEOWNERS_REPO_TOKEN }} deckhouse_private_repo: ${{ secrets.DECKHOUSE_PRIVATE_REPO }} - scan_several_lastest_releases: ${{ github.event.inputs.scan_several_lastest_releases }} + cve_test_repo_git: ${{ secrets.CVE_TEST_REPO_GIT }} + cve_ssh_private_key: ${{ secrets.CVE_TEST_SSH_PRIVATE_KEY }} + scan_several_latest_releases: ${{ github.event.inputs.scan_several_latest_releases || 'True' }} latest_releases_amount: ${{ github.event.inputs.latest_releases_amount || '3' }} - severity: ${{ github.event.inputs.severity }} + release_in_dev: ${{ github.event.inputs.release_in_dev || 'False' }} + trivy_reports_log_output: ${{ github.event.inputs.trivy_reports_log_output || '1' }} diff --git a/cve_scan/action.yml b/cve_scan/action.yml index 934f0b7..da204c5 100644 --- a/cve_scan/action.yml +++ b/cve_scan/action.yml @@ -1,328 +1,144 @@ name: 'Trivy CVE Scan' description: 'Build Deckhouse module' inputs: - tag_type: - description: 'Module tag type' - required: true - tag: - description: 'Module image tag' - required: true - module_name: - description: 'Module name' - required: true - dd_url: - description: 'URL to defectDojo' - required: true - dd_token: - description: 'Token of defectDojo to upload reports' - required: true prod_registry: - description: 'Must be deckhouse prod registry, used to get trivy databases and release images' + description: 'Prod registry host (e.g., registry.deckhouse.io)' required: true prod_registry_user: - description: 'Username to log in to deckhouse prod registry' + description: 'Username for prod registry authentication' required: true prod_registry_password: - description: 'Password to log in to deckhouse prod registry' + description: 'Password for prod registry authentication' required: true dev_registry: - description: 'Must be deckhouse dev registry, used to get dev images' + description: 'Dev-registry host (e.g., dev-registry.deckhouse.io)' required: true dev_registry_user: - description: 'Username to log in to deckhouse dev registry' + description: 'Username for dev-registry authentication' required: true dev_registry_password: - description: 'Password to log in to deckhouse dev registry' + description: 'Password for dev-registry authentication' + required: true + codeowners_repo_token: + description: 'Fox token for downloading CODEOWNERS configmap' required: true deckhouse_private_repo: - description: 'URL to private repo to get Trivy from' + description: 'Deckhouse private repository' + required: true + dd_url: + description: 'DefectDojo API URL' required: true - scan_several_lastest_releases: - description: 'true/false. Whether to scan last several releases or not. For scheduled pipelines override will not work as value is always true' + dd_token: + description: 'DefectDojo API token' + required: true + source_tag: + description: 'Tag to scan (e.g., main, v1.74.3, pr123, release-1.73)' + required: true + case: + description: 'Scan type: deckhouse | external_modules | CSE' + required: true + cve_test_repo_git: + description: 'cve_scan repo' + required: true + cve_ssh_private_key: + description: 'cve_scan repo key' + required: true + external_module_name: + description: 'External module name (required when case=External Modules)' required: false - latest_releases_amount: - description: 'Number of latest releases to scan. Default is: 3' + default: '' + scan_several_latest_releases: + description: 'Scan multiple latest releases (True/False)' required: false - severity: - description: 'Vulnerabilities severity to scan. Default is: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL' + default: 'False' + latest_releases_amount: + description: 'Number of latest releases to scan when scan_several_latest_releases=true' required: false + default: '3' module_prod_registry_custom_path: - description: 'Module custom path in prod registry. Example: flant/modules' + description: 'Custom path for external modules in production registry' required: false + default: 'deckhouse/fe/modules' module_dev_registry_custom_path: - description: 'Module custom path in dev registry. Example: flant/modules' + description: 'Custom path for external modules in development registry' + required: false + default: 'sys/deckhouse-oss/modules' + release_in_dev: + description: 'If true, release tag will be searched in dev registry instead of prod' + required: false + default: 'False' + digest_from_werf: + description: 'Path to werf images tags file (for CSE external modules)' required: false + default: 'images_tags_werf' + scan_users: + description: 'Enable user validation scan for CSE (True/False)' + required: false + default: 'False' + workdir: + description: 'Working directory for temporary files' + required: false + default: 'cve-scan' + trivy_reports_log_output: + description: 'Trivy log output level (0=off, 1=CVE only, 2=CVE+License)' + required: false + default: '1' runs: using: "composite" steps: + - name: Start ssh-agent + uses: webfactory/ssh-agent@v0.9.0 + with: + ssh-private-key: ${{ inputs.cve_ssh_private_key }} + + - name: Add host to known_hosts + shell: bash + run: | + HOST=$(echo "${{ inputs.cve_test_repo_git }}" | sed -E 's/.*@([^:]+).*/\1/') + mkdir -p ~/.ssh + ssh-keyscan -H "$HOST" >> ~/.ssh/known_hosts 2>/dev/null + + - name: Clone repository + shell: bash + run: | + rm -rf /tmp/cve-scripts + git clone --depth 1 ${{ inputs.cve_test_repo_git }} /tmp/cve-scripts + cp /tmp/cve-scripts/* ./ + - name: Run Trivy CVE Scan shell: bash env: TRIVY_BIN_VERSION: "v0.67.2" TRIVY_REPO_ID: "2181" - TRIVY_DB_URL: "${{inputs.prod_registry}}/deckhouse/ee/security/trivy-db:2" - TRIVY_JAVA_DB_URL: "${{inputs.prod_registry}}/deckhouse/ee/security/trivy-java-db:1" - DECKHOUSE_PRIVATE_REPO: ${{inputs.deckhouse_private_repo}} - IMAGES_DIGESTS_PATH: "/images_digests.json" - TAG: "${{inputs.tag}}" - TAG_TYPE: "${{inputs.tag_type}}" - MODULE_NAME: "${{inputs.module_name}}" - DD_URL: "${{inputs.dd_url}}" - DD_TOKEN: "${{inputs.dd_token}}" - TRIVY_POLICY_URL: "${{inputs.prod_registry}}/deckhouse/ee/security/trivy-bdu:1" - SCAN_SEVERAL_LASTEST_RELEASES: "${{inputs.scan_several_lastest_releases}}" - LATEST_RELEASES_AMOUNT: "${{inputs.latest_releases_amount || '3'}}" - SEVERITY: "${{inputs.severity || 'UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL'}}" - MODULE_PROD_REGISTRY_PATH: "${{inputs.module_prod_registry_custom_path || 'deckhouse/fe/modules'}}" - MODULE_DEV_REGISTRY_PATH: "${{inputs.module_dev_registry_custom_path || 'sys/deckhouse-oss/modules'}}" + TRIVY_PROD_REGISTRY: "registry.deckhouse.io" + TRIVY_DEV_REGISTRY: "dev-registry.deckhouse.io" + TRIVY_DB_URL: "${{ inputs.prod_registry }}/deckhouse/ee/security/trivy-db:2" + TRIVY_JAVA_DB_URL: "${{ inputs.prod_registry }}/deckhouse/ee/security/trivy-java-db:1" + TRIVY_POLICY_URL: "${{ inputs.prod_registry }}/deckhouse/ee/security/trivy-bdu:1" + TRIVY_REPORTS_LOG_OUTPUT: "${{ inputs.trivy_reports_log_output }}" + PROD_REGISTRY: "${{ inputs.prod_registry }}" + PROD_REGISTRY_USER: "${{ inputs.prod_registry_user }}" + PROD_REGISTRY_PASSWORD: "${{ inputs.prod_registry_password }}" + DEV_REGISTRY: "${{ inputs.dev_registry }}" + DEV_REGISTRY_USER: "${{ inputs.dev_registry_user }}" + DEV_REGISTRY_PASSWORD: "${{ inputs.dev_registry_password }}" + SOURCE_TAG: "${{ inputs.source_tag }}" + CASE: "${{ inputs.case }}" + EXTERNAL_MODULE_NAME: "${{ inputs.external_module_name }}" + RELEASE_IN_DEV: "${{ inputs.release_in_dev }}" + SCAN_USERS: "${{ inputs.scan_users }}" + SCAN_SEVERAL_LATEST_RELEASES: "${{ inputs.scan_several_latest_releases }}" + LATEST_RELEASES_AMOUNT: "${{ inputs.latest_releases_amount }}" + MODULE_PROD_REGISTRY_CUSTOM_PATH: "${{ inputs.module_prod_registry_custom_path }}" + MODULE_DEV_REGISTRY_CUSTOM_PATH: "${{ inputs.module_dev_registry_custom_path }}" + DIGEST_FROM_WERF: "${{ inputs.digest_from_werf }}" + DD_URL: "${{ inputs.dd_url }}" + DD_TOKEN: "${{ inputs.dd_token }}" + CODEOWNERS_REPO_TOKEN: "${{ inputs.codeowners_repo_token }}" + DECKHOUSE_PRIVATE_REPO: "${{ inputs.deckhouse_private_repo }}" + CONFIGMAP_PROJECT_ID: "4352" + WORKDIR: "${{ github.workspace }}/${{ inputs.workdir }}" run: | - echo "Creating workdir" - workdir="trivy_scan" - # remove workdir in case it was not removed on previous run - rm -rf "${workdir}" - mkdir "${workdir}" - echo - echo "=======================================================" - echo - echo "Preparing DOCKER_CONFIG and log in to registries" - mkdir -p "${workdir}/docker" - export DOCKER_CONFIG="${workdir}/docker" - echo "${{inputs.prod_registry_password}}" | docker login --username="${{inputs.prod_registry_user}}" --password-stdin ${{inputs.prod_registry}} - echo "${{inputs.dev_registry_password}}" | docker login --username="${{inputs.dev_registry_user}}" --password-stdin ${{inputs.dev_registry}} - echo - echo "=======================================================" - echo - echo "Get Trivy" - echo "Trivy version: ${TRIVY_BIN_VERSION}" - mkdir -p "${workdir}/bin/trivy-${TRIVY_BIN_VERSION}" - curl -L -s --fail-with-body https://${DECKHOUSE_PRIVATE_REPO}/api/v4/projects/${TRIVY_REPO_ID}/packages/generic/trivy-${TRIVY_BIN_VERSION}/${TRIVY_BIN_VERSION}/trivy -o ${workdir}/bin/trivy-${TRIVY_BIN_VERSION}/trivy - chmod u+x ${workdir}/bin/trivy-${TRIVY_BIN_VERSION}/trivy - ln -s ${PWD}/${workdir}/bin/trivy-${TRIVY_BIN_VERSION}/trivy ${workdir}/bin/trivy - - echo "Updating Trivy Data Bases" - mkdir -p "${workdir}/bin/trivy_cache" - ${workdir}/bin/trivy image --timeout 15m --username "${{inputs.prod_registry_user}}" --password "${{inputs.prod_registry_password}}" --download-db-only --db-repository "${TRIVY_DB_URL}" --cache-dir "${workdir}/bin/trivy_cache" - ${workdir}/bin/trivy image --timeout 15m --username "${{inputs.prod_registry_user}}" --password "${{inputs.prod_registry_password}}" --download-java-db-only --java-db-repository "${TRIVY_JAVA_DB_URL}" --cache-dir "${workdir}/bin/trivy_cache" - echo - echo "=======================================================" - echo - # Defining functions - - trivy_scan() { - ${workdir}/bin/trivy i --timeout 15m --vex oci --show-suppressed --config-check "${TRIVY_POLICY_URL}" --cache-dir "${workdir}/bin/trivy_cache" --skip-db-update --skip-java-db-update --exit-code 0 --severity "${SEVERITY}" --ignorefile "${module_workdir}/.trivyignore" --format ${1} ${2} ${3} --quiet ${4} --username "${trivy_registry_user}" --password "${trivy_registry_pass}" --image-src remote - } - - send_report() { - dd_scan_type="${1}" - dd_report_file_path="${2}" - dd_module_name="${3}" - dd_image_name="${4}" - dd_engagement_name="[$(echo "${dd_scan_type}" | tr '[:lower:]' '[:upper:]')] [IMAGES] [${dd_branch}]" - tags_string="\"external_modules\",\"images\",\"${dd_scan_type}\",\"${dd_release_or_dev_tag}\",\"${dd_image_version}\"" - if [[ -n "${dd_short_release_tag}" && -n "${dd_full_release_tag}" ]]; then - tags_string+=",\"${dd_short_release_tag}\",\"${dd_full_release_tag}\"" - fi - echo "" - echo "Uploading trivy ${dd_branch} report for image \"${dd_image_name}\" of \"${dd_module_name}\" module" - echo "" - dd_upload_response=$(curl -sw "%{http_code}" -X POST \ - --retry 10 \ - --retry-delay 20 \ - --retry-all-errors \ - ${DD_URL}/api/v2/reimport-scan/ \ - -H "accept: application/json" \ - -H "Authorization: Token ${DD_TOKEN}" \ - -F "auto_create_context=True" \ - -F "minimum_severity=Info" \ - -F "active=true" \ - -F "verified=true" \ - -F "scan_type=Trivy Scan" \ - -F "close_old_findings=true" \ - -F "do_not_reactivate=false" \ - -F "push_to_jira=false" \ - -F "file=@${dd_report_file_path}" \ - -F "product_type_name=External Modules" \ - -F "product_name=${dd_module_name}" \ - -F "scan_date=${date_iso}" \ - -F "engagement_name=${dd_engagement_name}" \ - -F "service=${dd_module_name} / ${dd_image_name}" \ - -F "group_by=component_name+component_version" \ - -F "deduplication_on_engagement=false" \ - -F "tags=external_module,${dd_scan_type},module:${dd_module_name},image:${dd_image_name},branch:${dd_branch},${dd_short_release_tag},${dd_full_release_tag},${dd_default_branch_tag},${dd_release_or_dev_tag}" \ - -F "test_title=[${dd_module_name}]: ${dd_image_name}:${dd_image_version}" \ - -F "version=${dd_image_version}" \ - -F "build_id=${IMAGE_HASH}" \ - -F "commit_hash=${CI_COMMIT_SHA}" \ - -F "branch_tag=${module_tag}" \ - -F "apply_tags_to_findings=true") - - dd_return_code="${dd_upload_response: -3}" - dd_return_body="${dd_upload_response:0: -3}" - if [ ${dd_return_code} -eq 201 ]; then - dd_engagement_id=$(echo ${dd_return_body} | jq ".engagement_id" ) - echo "dd_engagement_id: ${dd_engagement_id}" - echo "Update with tags: ${tags_string}" - # Updating engagement - dd_eng_patch_response=$(curl -sw "%{http_code}" -X "PATCH" \ - --retry 10 \ - --retry-delay 20 \ - --retry-all-errors \ - "${DD_URL}/api/v2/engagements/${dd_engagement_id}/" \ - -H "accept: application/json" \ - -H "Authorization: Token ${DD_TOKEN}" \ - -H "Content-Type: application/json" \ - -d "{ - \"tags\": ["${tags_string}"], - \"version\": \"${dd_image_version}\", - \"branch_tag\": \"${dd_branch}\" - }") - if [ ${dd_eng_patch_response: -3} -eq 200 ]; then - echo "Engagemet \"${dd_engagement_name}\" updated successfully" - else - echo "!!!WARNING!!!" - echo "Engagemet \"${dd_engagement_name}\" WAS NOT UPDATED" - echo "HTTP_CODE: ${dd_eng_patch_response: -3}" - echo "DD_RESPONSE: ${dd_eng_patch_response:0: -3}" - fi - else - echo "!!!WARNING!!!" - echo "Report for image \"${dd_image_name}\" of \"${dd_module_name}\" module WAS NOT UPLOADED" - echo "HTTP_CODE: ${dd_return_code}" - echo "DD_RESPONSE: ${dd_return_body}" - fi - } - if [ "${{ github.event_name }}" == "schedule" ]; then - SCAN_SEVERAL_LASTEST_RELEASES="true" - fi - - echo "Setting up registry path for module" - PROD_REGISTRY_MODULE_BASEDIR="${{inputs.prod_registry}}/${MODULE_PROD_REGISTRY_PATH}" - DEV_REGISTRY_MODULE_BASEDIR="${{inputs.dev_registry}}/${MODULE_DEV_REGISTRY_PATH}" - - echo "Getting tags to scan" - module_tags=("${TAG}") - # Check if provided tag for manual run is for release - if [ "${{ github.event_name }}" != "pull_request" ]; then - if [ "${TAG}" != "${{ github.event.repository.default_branch }}" ]; then - if [ "${TAG_TYPE}" == "release" ]; then - # if some specific release is defined - scan only it - if echo "${TAG}"|grep -qE "^[0-9]+\.[0-9]+$"; then - module_tags=($(crane ls "${PROD_REGISTRY_MODULE_BASEDIR}/${MODULE_NAME}" | grep "^v${TAG}\.[0-9]*$" | sort -V -r | head -n 1)) - else - echo "ERROR: Please specify required release in the following format: [0-9]+\.[0-9]+" - exit 1 - fi - elif [ "${TAG_TYPE}" == "dev" ]; then - if [ $(crane ls "${DEV_REGISTRY_MODULE_BASEDIR}/${MODULE_NAME}" | grep "^${TAG}$" | wc -l) -eq 1 ]; then - module_tags=("${TAG}") - else - echo "ERROR: Provided tag \"${TAG}\" is not found in dev registry" - exit 1 - fi - else - echo "ERROR: TAG TYPE is not defined!" - exit 1 - fi - fi - fi - if [ "${SCAN_SEVERAL_LASTEST_RELEASES}" == "true" ]; then - # Get release tags by regexp, sort by sevmer desc, cut to get minor version, uniq and get 3 latest - releases=($(crane ls "${PROD_REGISTRY_MODULE_BASEDIR}/${MODULE_NAME}" | grep "^v[0-9]*\.[0-9]*\.[0-9]*" | sort -V -r)) - latest_minor_releases=($(printf '%s\n' "${releases[@]}"| cut -d "." -f -2 | uniq | head -n ${LATEST_RELEASES_AMOUNT})) - for r in "${latest_minor_releases[@]}"; do - module_tags+=($(printf '%s\n' "${releases[@]}" | grep "${r}" | sort -V -r|head -n 1)) - done - fi - - echo "CVE Scan will be applied to the following tags of ${MODULE_NAME}" - echo "${module_tags[*]}" - # Scan in loop for provided list of tags - for module_tag in ${module_tags[*]}; do - dd_default_branch_tag="" - dd_short_release_tag="" - dd_full_release_tag="" - dd_release_or_dev_tag="dev" - dd_image_version="${module_tag}" - dd_branch="${module_tag}" - date_iso=$(date -I) - module_image="${DEV_REGISTRY_MODULE_BASEDIR}/${MODULE_NAME}" - trivy_registry_user="${{inputs.dev_registry_user}}" - trivy_registry_pass="${{inputs.dev_registry_password}}" - if [ "${module_tag}" == "${{ github.event.repository.default_branch }}" ]; then - dd_default_branch_tag="default_branch" - fi - # If we are scanning release images - we need to redefine image path to prod registry - if echo "${module_tag}" | grep -q "^v[0-9]*\.[0-9]*\.[0-9]*" && [[ "${{ github.event_name }}" != "pull_request" ]]; then - module_image="${PROD_REGISTRY_MODULE_BASEDIR}/${MODULE_NAME}" - trivy_registry_user="${{inputs.prod_registry_user}}" - trivy_registry_pass="${{inputs.prod_registry_password}}" - dd_short_release_tag="release:$(echo ${module_tag} | cut -d '.' -f -2 | sed 's/^v//')" - dd_full_release_tag="image_release_tag:${module_tag}" - dd_release_or_dev_tag="release" - dd_image_version="$(echo ${dd_short_release_tag} | sed 's/^release\://')" - fi - module_workdir="${workdir}/${MODULE_NAME}_${module_tag}" - module_reports="${module_workdir}/reports" - mkdir -p "${module_reports}" - touch ${module_workdir}/.trivyignore - echo "Run Trivy scan" - echo "Image to check: ${module_image}:${module_tag}" - echo "Severity: ${SEVERITY}" - echo "----------------------------------------------" - echo "" - echo "Getting module image" - crane export "${module_image}:${module_tag}" "${MODULE_NAME}.tar" - tar xf "${MODULE_NAME}.tar" -C "${module_workdir}/" - echo "Preparing images list to scan" - digests=$(cat "${module_workdir}${IMAGES_DIGESTS_PATH}") - # Main module images to scan - digests=$(echo "${digests}"|jq --arg i "${MODULE_NAME}" --arg s "${module_tag}" '. += { ($i): ($s) }') - echo "Images to scan:" - echo "${digests}" - while read -r line; do - IMAGE_NAME=$(jq -rc '.key' <<< "${line}") - if [[ "${IMAGE_NAME}" == "trivy" ]]; then - continue - fi - # Set flag if additional image to use tag instead of hash - additional_image_detected=false - if [ "${IMAGE_NAME}" == "${MODULE_NAME}" ]; then - additional_image_detected=true - fi - echo "----------------------------------------------" - echo "👾 Scaning image \"${IMAGE_NAME}\" of module \"${MODULE_NAME}\" for tag \"${module_tag}\"" - echo "" - IMAGE_HASH="$(jq -rc '.value' <<< "$line")" - - if [ "$additional_image_detected" == true ]; then - if [ "${TRIVY_REPORTS_LOG_OUTPUT}" != "false" ]; then - # CVE Scan - trivy_scan "table" "--scanners vuln" "" "${module_image}:${module_tag}" - # License scan - trivy_scan "table" "--scanners license --license-full" "" "${module_image}:${module_tag}" - fi - # CVE Scan - trivy_scan "json" "--scanners vuln" "--output ${module_reports}/ext_${MODULE_NAME}_${IMAGE_NAME}_report.json" "${module_image}:${module_tag}" - # License scan - trivy_scan "json" "--scanners license --license-full" "--output ${module_reports}/ext_${MODULE_NAME}_${IMAGE_NAME}_report_license.json" "${module_image}:${module_tag}" - else - if [ "${TRIVY_REPORTS_LOG_OUTPUT}" != "false" ]; then - # CVE Scan - trivy_scan "table" "--scanners vuln" "" "${module_image}@${IMAGE_HASH}" - # License scan - trivy_scan "table" "--scanners license --license-full" "" "${module_image}@${IMAGE_HASH}" - fi - # CVE Scan - trivy_scan "json" "--scanners vuln" "--output ${module_reports}/ext_${MODULE_NAME}_${IMAGE_NAME}_report.json" "${module_image}@${IMAGE_HASH}" - # License scan - trivy_scan "json" "--scanners license --license-full" "--output ${module_reports}/ext_${MODULE_NAME}_${IMAGE_NAME}_report_license.json" "${module_image}@${IMAGE_HASH}" - fi - echo " Done" - - send_report "CVE" "${module_reports}/ext_${MODULE_NAME}_${IMAGE_NAME}_report.json" "${MODULE_NAME}" "${IMAGE_NAME}" - send_report "License" "${module_reports}/ext_${MODULE_NAME}_${IMAGE_NAME}_report_license.json" "${MODULE_NAME}" "${IMAGE_NAME}" - done < <(jq -rc 'to_entries[]' <<< "${digests}") - done - rm -r ${workdir} + ./cve_scan.sh