-
Notifications
You must be signed in to change notification settings - Fork 1
feat: enable log forwarding through otel collector #186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
ffd8772
14514f2
77f12cf
f41fd56
11631ad
b2cff4d
564ff2c
7b44fd0
5961c47
1bdb99d
32eb82f
f7f4147
0539529
16a1ff7
9934f4e
8337420
9590fd5
3f3fd59
f15564a
bc2a1e4
5016dcb
a920d9a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| name: Enable Log Forwarding Action Tests | ||
|
|
||
| on: | ||
| pull_request: | ||
| workflow_call: | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| detect-changes: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| action: ${{ steps.filter.outputs.action }} | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: dorny/paths-filter@v4 | ||
| id: filter | ||
| with: | ||
| filters: | | ||
| action: | ||
| - 'actions/enable_log_forwarding/**' | ||
| - '.github/workflows/enable_log_forwarding_action_tests.yaml' | ||
|
|
||
| test-action: | ||
| needs: detect-changes | ||
| if: ${{ needs.detect-changes.outputs.action == 'true' }} | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Set up uv | ||
| uses: astral-sh/setup-uv@v8.1.0 | ||
|
|
||
| - name: Install tox | ||
| run: uv tool install tox --with tox-uv | ||
|
|
||
| - name: Run lint, static checks, and unit tests | ||
| run: tox -e actions-lint,actions-static,actions-unit | ||
|
|
||
| smoke-test-self-hosted: | ||
| needs: detect-changes | ||
| if: ${{ needs.detect-changes.outputs.action == 'true' }} | ||
| runs-on: [self-hosted-linux-amd64-noble-edge] | ||
| env: | ||
| TEST_CONFIG_FILE: 98-enable-log-forwarding-smoke-${{ github.run_id }}-${{ github.run_attempt }}.yaml | ||
|
yanksyoon marked this conversation as resolved.
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Run enable log forwarding action | ||
| uses: ./actions/enable_log_forwarding | ||
| with: | ||
| files: | | ||
| /var/log/syslog | ||
| config-file-name: ${{ env.TEST_CONFIG_FILE }} | ||
| otlp-endpoint: 127.0.0.1:4317 | ||
|
|
||
| - name: Verify generated config file exists | ||
| run: | | ||
| sudo test -f /etc/otelcol/config.d/${TEST_CONFIG_FILE} | ||
|
|
||
| - name: Verify generated config contains expected receiver | ||
| run: | | ||
| sudo grep -q '"filelog/github_runner_optin"' /etc/otelcol/config.d/${TEST_CONFIG_FILE} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Copyright 2026 Canonical Ltd. | ||
| # See LICENSE file for licensing details. | ||
| name: Enable log forwarding | ||
| description: Opt in to forward selected log files from a self-hosted GitHub runner to Loki. | ||
|
|
||
| inputs: | ||
| files: | ||
| description: | | ||
| Newline or comma separated list of file paths or glob patterns to forward. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nitpick, avoid double "or", so There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reading the code, we support Comma separated, or list separated AND globbing. So you can have multiple globbing |
||
| Example: /var/log/chrony/*.log | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Provide an example for each supported content (newline, comma separated, and glob) |
||
| required: true | ||
| otlp-endpoint: | ||
| description: | | ||
| Optional OTLP/gRPC endpoint for upstream OpenTelemetry Collector logs export. | ||
| When not set, the action falls back to ACTION_OTEL_EXPORTER_OTLP_ENDPOINT. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Be more specific about where this env var comes from like: |
||
| Example: otel-gateway.internal:4317 | ||
| required: false | ||
| default: "" | ||
| config-file-name: | ||
| description: File name for the generated collector fragment. | ||
|
Comment on lines
+19
to
+20
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Be specific about the usecase, renaming might helpful for priority mainly if they want this file to be evaluated before or after some other rules |
||
| required: false | ||
| default: 90-github-runner-log-forwarding.yaml | ||
|
yanksyoon marked this conversation as resolved.
|
||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Configure collector for opt-in file log forwarding | ||
| shell: bash | ||
| env: | ||
| INPUT_FILES: ${{ inputs.files }} | ||
| INPUT_OTLP_ENDPOINT: ${{ inputs.otlp-endpoint }} | ||
| INPUT_CONFIG_FILE_NAME: ${{ inputs.config-file-name }} | ||
| run: python3 "${{ github.action_path }}/enable_log_forwarding.py" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| { | ||
| "receivers": { | ||
| "filelog/github_runner_optin": { | ||
| "include": {{ include_files }}, | ||
| "start_at": "end" | ||
| } | ||
| }, | ||
| {{ exporters_section }} | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do a test to see if it won't cause a conflict if the exporter is not optional and there are two config files has the same exporter |
||
| "processors": { | ||
| "resource/github_runner_optin": { | ||
| "attributes": {{ resource_attributes }} | ||
| } | ||
| }, | ||
| "service": { | ||
| "pipelines": { | ||
| "logs/github_runner_optin": { | ||
| "receivers": ["filelog/github_runner_optin"], | ||
| "processors": ["resource/github_runner_optin", "batch"], | ||
| "exporters": [{{ exporter_name }}] | ||
| } | ||
| } | ||
| } | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.