-
Notifications
You must be signed in to change notification settings - Fork 1.5k
N8N integration #21835
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
HadhemiDD
wants to merge
21
commits into
master
Choose a base branch
from
hadhemi/n8n_initial
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
N8N integration #21835
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
10999db
n8n initial draft
HadhemiDD 08e8203
fix validation
HadhemiDD 6c7ada3
lint
HadhemiDD abce4d6
fix
HadhemiDD 8a35317
commit
HadhemiDD 020e48f
labeler
HadhemiDD a798858
fix tests
HadhemiDD b8d7c78
fix pr number
HadhemiDD 70e3951
Update n8n/README.md
HadhemiDD fdb6776
Update n8n/assets/configuration/spec.yaml
HadhemiDD 7c0fee8
Update n8n/assets/configuration/spec.yaml
HadhemiDD eb345a6
Update n8n/assets/configuration/spec.yaml
HadhemiDD 543ad67
readme
HadhemiDD 8b27840
cleanup
HadhemiDD d1929d6
add metrics
HadhemiDD a6b4623
commit
HadhemiDD 71d162e
metrics types
HadhemiDD 15120e9
lint
HadhemiDD a308cc4
lint
HadhemiDD 17b3dbe
Merge branch 'master' into hadhemi/n8n_initial
HadhemiDD 430fc2a
count metrics
HadhemiDD File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # CHANGELOG - n8n | ||
|
|
||
| <!-- towncrier release notes start --> | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| # Agent Check: n8n | ||
|
|
||
| ## Overview | ||
|
|
||
| This check monitors [n8n][1] through the Datadog Agent. | ||
|
|
||
| Collect n8n metrics including: | ||
| - Cache metrics: Hit and miss statistics. | ||
| - Message event bus metrics: Event-related metrics. | ||
| - Workflow metrics: Can include workflow ID labels. | ||
| - Node metrics: Can include node type labels. | ||
| - Credential metrics: Can include credential type labels. | ||
| - Queue Metrics | ||
|
|
||
|
|
||
| ## Setup | ||
|
|
||
| Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions. | ||
|
|
||
| ### Installation | ||
|
|
||
| The n8n check is included in the [Datadog Agent][2] package. | ||
| No additional installation is needed on your server. | ||
|
|
||
| ### Configuration | ||
|
|
||
| The `/metrics` endpoint is disabled by default and must be enabled by setting `N8N_METRICS`=`true`. You can also customize the metric prefix using `N8N_METRICS_PREFIX` (default is `n8n_`). | ||
|
|
||
| Note that the `/metrics` endpoint is only available for self-hosted instances and is not available on n8n Cloud. | ||
|
|
||
| For the datadog agent to collect metrics, you will need to follow the instructions provided [here][10]. | ||
|
|
||
| 1. Edit the `n8n.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your n8n performance data. See the [sample n8n.d/conf.yaml][4] for all available configuration options. | ||
|
|
||
| 2. [Restart the Agent][5]. | ||
|
|
||
| ### Validation | ||
|
|
||
| [Run the Agent's status subcommand][6] and look for `n8n` under the Checks section. | ||
|
|
||
| ## Data Collected | ||
|
|
||
| ### Metrics | ||
|
|
||
| See [metadata.csv][7] for a list of metrics provided by this integration. | ||
|
|
||
| ### Events | ||
|
|
||
| The n8n integration does not include any events. | ||
|
|
||
| ### Service Checks | ||
|
|
||
| See [service_checks.json][8] for a list of service checks provided by this integration. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| Need help? Contact [Datadog support][9]. | ||
|
|
||
|
|
||
| [1]: https://n8n.io/ | ||
| [2]: https://app.datadoghq.com/account/settings/agent/latest | ||
| [3]: https://docs.datadoghq.com/containers/kubernetes/integrations/ | ||
| [4]: https://github.com/DataDog/integrations-core/blob/master/n8n/datadog_checks/n8n/data/conf.yaml.example | ||
| [5]: https://docs.datadoghq.com/agent/configuration/agent-commands/#start-stop-and-restart-the-agent | ||
| [6]: https://docs.datadoghq.com/agent/configuration/agent-commands/#agent-status-and-information | ||
| [7]: https://github.com/DataDog/integrations-core/blob/master/n8n/metadata.csv | ||
| [8]: https://github.com/DataDog/integrations-core/blob/master/n8n/assets/service_checks.json | ||
| [9]: https://docs.datadoghq.com/help/ | ||
| [10]: https://docs.n8n.io/hosting/configuration/configuration-examples/prometheus/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| name: n8n | ||
| files: | ||
| - name: n8n.yaml | ||
| options: | ||
| - template: init_config | ||
| options: | ||
| - template: init_config/default | ||
| - template: instances | ||
| options: | ||
| - template: instances/openmetrics | ||
| overrides: | ||
| openmetrics_endpoint.required: true | ||
| openmetrics_endpoint.hidden: false | ||
| openmetrics_endpoint.display_priority: 1 | ||
| openmetrics_endpoint.value.example: http://localhost:5678 | ||
| openmetrics_endpoint.description: | | ||
| Endpoint exposing the n8n's metrics in the OpenMetrics format. For more information, refer to: | ||
| https://docs.n8n.io/hosting/logging-monitoring/monitoring/ | ||
| https://docs.n8n.io/hosting/configuration/environment-variables/endpoints/ | ||
| raw_metric_prefix.description: | | ||
| The prefix prepended to all metrics from n8n. | ||
| If not set, the default prefix is used. | ||
| The default prefix is 'n8n'. | ||
| If you are using a custom prefix in n8n through N8N_METRICS_PREFIX, you can set it here. | ||
| raw_metric_prefix.value: | ||
| display_default: n8n | ||
| type: string | ||
| example: n8n | ||
| raw_metric_prefix.hidden: false | ||
| - name: server_port | ||
| description: | | ||
| The port exposing the HTTP endpoint of the n8n API. | ||
| value: | ||
| display_default: 5678 | ||
| type: integer | ||
| - template: logs | ||
| example: | ||
| - type: file | ||
| path: /var/log/n8n/*.log | ||
| source: n8n | ||
| service: <SERVICE> | ||
| - type: docker | ||
| source: n8n | ||
| service: <SERVICE> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "title": "N8N Overview Dashboard", | ||
| "description": "N8N Overview Dashboard", | ||
| "widgets": [], | ||
| "template_variables": [], | ||
| "layout_type": "ordered", | ||
| "notify_list": [], | ||
| "reflow_type": "fixed" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| [ | ||
| { | ||
| "agent_version": "7.75.0", | ||
| "integration": "n8n", | ||
| "check": "n8n.openmetrics.health", | ||
| "statuses": ["ok", "critical"], | ||
| "groups": [], | ||
| "name": "n8n OpenMetrics Health", | ||
| "description": "Returns `CRITICAL` if the check cannot access the metrics endpoint. Returns `OK` otherwise." | ||
| }, | ||
| { | ||
| "agent_version": "7.75.0", | ||
| "integration": "n8n", | ||
| "check": "n8n.health.status", | ||
| "statuses": ["ok", "critical"], | ||
| "groups": [], | ||
| "name": "n8n Health Status", | ||
| "description": "Returns `CRITICAL` if the health check endpoint returns an unhealthy status. Returns `OK` otherwise." | ||
| }, | ||
| { | ||
| "agent_version": "7.75.0", | ||
| "integration": "n8n", | ||
| "check": "n8n.readiness.status", | ||
| "statuses": ["ok", "critical"], | ||
| "groups": [], | ||
| "name": "n8n Readiness Status", | ||
| "description": "Returns `CRITICAL` if the readiness check endpoint returns an unready status. Returns `OK` otherwise." | ||
| } | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Initial Release |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # (C) Datadog, Inc. 2025-present | ||
| # All rights reserved | ||
| # Licensed under a 3-clause BSD style license (see LICENSE) | ||
| __version__ = '0.0.1' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # (C) Datadog, Inc. 2025-present | ||
| # All rights reserved | ||
| # Licensed under a 3-clause BSD style license (see LICENSE) | ||
| from .__about__ import __version__ | ||
| from .check import N8nCheck | ||
|
|
||
| __all__ = ['__version__', 'N8nCheck'] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # (C) Datadog, Inc. 2025-present | ||
| # All rights reserved | ||
| # Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
|
||
| from urllib.parse import urljoin, urlparse # noqa: F401 | ||
|
|
||
| from datadog_checks.base import AgentCheck, OpenMetricsBaseCheckV2 | ||
| from datadog_checks.n8n.metrics import METRIC_MAP | ||
|
|
||
| DEFAULT_READY_ENDPOINT = '/healthz/readiness' | ||
| DEFAULT_HEALTH_ENDPOINT = '/healthz' | ||
| DEFAULT_VERSION_ENDPOINT = '/rest/settings' | ||
|
|
||
|
|
||
| class N8nCheck(OpenMetricsBaseCheckV2): | ||
| __NAMESPACE__ = 'n8n' | ||
| DEFAULT_METRIC_LIMIT = 0 | ||
|
|
||
| def __init__(self, name, init_config, instances=None): | ||
| super(N8nCheck, self).__init__( | ||
| name, | ||
| init_config, | ||
| instances, | ||
| ) | ||
| self.openmetrics_endpoint = self.instance["openmetrics_endpoint"] | ||
| self.tags = self.instance.get('tags', []) | ||
| self._ready_endpoint = DEFAULT_READY_ENDPOINT | ||
| self._health_endpoint = DEFAULT_HEALTH_ENDPOINT | ||
| self._version_endpoint = DEFAULT_VERSION_ENDPOINT | ||
| # Get the N8N API port if specified, otherwise use the default 5678. | ||
| self.server_port = str(self.instance.get('server_port', 5678)) | ||
| self.raw_metric_prefix = self.instance.get('raw_metric_prefix', 'n8n') | ||
|
|
||
| def get_default_config(self): | ||
| # If raw_metric_prefix is 'n8n', metrics start with 'n8n' | ||
| if self.raw_metric_prefix == 'n8n': | ||
| namespace = 'n8n' | ||
| else: | ||
| namespace = f'n8n.{self.raw_metric_prefix}' | ||
|
|
||
| return {'namespace': namespace, 'metrics': [METRIC_MAP]} | ||
|
|
||
| @AgentCheck.metadata_entrypoint | ||
| def _submit_version_metadata(self): | ||
| endpoint = urljoin(self.openmetrics_endpoint, self._version_endpoint) | ||
| response = self.http.get(endpoint) | ||
|
|
||
| if response.ok: | ||
| data = response.json() | ||
| version = data.get("versionCli", "") | ||
| version_split = version.split(".") | ||
| if len(version_split) >= 3: | ||
| major = version_split[0] | ||
| minor = version_split[1] | ||
| patch = version_split[2] | ||
|
|
||
| version_raw = f'{major}.{minor}.{patch}' | ||
|
|
||
| version_parts = { | ||
| 'major': major, | ||
| 'minor': minor, | ||
| 'patch': patch, | ||
| } | ||
| self.set_metadata('version', version_raw, scheme='semver', part_map=version_parts) | ||
| else: | ||
| self.log.debug("Malformed N8N Server version format: %s", version) | ||
| else: | ||
| self.log.debug("Could not retrieve version metadata.") | ||
|
|
||
| def _check_n8n_health(self): | ||
| endpoint = urljoin(self.openmetrics_endpoint, self._health_endpoint) | ||
| response = self.http.get(endpoint) | ||
|
|
||
| # Any 4xx or 5xx response from the API endpoint (/healthz) means the n8n process is not responding | ||
| if 400 <= response.status_code and response.status_code < 600: | ||
| self.service_check('health.status', AgentCheck.CRITICAL, self.tags) | ||
| if response.status_code == 200: | ||
| self.service_check('health.status', AgentCheck.OK, self.tags) | ||
| else: | ||
| self.service_check('health.status', AgentCheck.UNKNOWN, self.tags) | ||
HadhemiDD marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| def _check_n8n_readiness(self): | ||
| endpoint = urljoin(self.openmetrics_endpoint, self._ready_endpoint) | ||
| response = self.http.get(endpoint) | ||
|
|
||
| # Any 4xx or 5xx response from the API endpoint (/healthz/readiness) | ||
| # means the n8n is not ready to accept requests | ||
| if 400 <= response.status_code and response.status_code < 600: | ||
| self.service_check('health.status', AgentCheck.CRITICAL, self.tags) | ||
| if response.status_code == 200: | ||
| self.service_check('health.status', AgentCheck.OK, self.tags) | ||
| else: | ||
| self.service_check('health.status', AgentCheck.UNKNOWN, self.tags) | ||
|
|
||
| def check(self, instance): | ||
| super().check(instance) | ||
| self._submit_version_metadata() | ||
| self._check_n8n_health() | ||
| self._check_n8n_readiness() | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.