From 21ec3cd6d52a1256064033d2336bd83a890dc880 Mon Sep 17 00:00:00 2001 From: Debanitrkl Date: Tue, 5 Aug 2025 08:24:10 +0530 Subject: [PATCH 1/5] Add Parseable output plugin documentation - Add comprehensive documentation for Parseable integration using OpenTelemetry plugin - Include configuration parameters table with required headers (X-P-Stream, X-P-Log-Source) - Provide examples for logs, metrics, and traces in both YAML and classic config formats - Document stream configuration and authentication setup - Add references to Parseable documentation and Fluent Bit integration guide Signed-off-by: Debanitrkl --- pipeline/outputs/parseable.md | 156 ++++++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 pipeline/outputs/parseable.md diff --git a/pipeline/outputs/parseable.md b/pipeline/outputs/parseable.md new file mode 100644 index 000000000..bfdfb0a57 --- /dev/null +++ b/pipeline/outputs/parseable.md @@ -0,0 +1,156 @@ +--- +description: Send logs, metrics, and traces to Parseable +--- + +# Parseable + +Stream logs, metrics, and traces to [Parseable](https://www.parseable.com) by utilizing the [OpenTelemetry plugin](opentelemetry.md) to send telemetry data to Parseable's OpenTelemetry-compatible Ingestor endpoints. + +## Configuration parameters + +| Key | Description | Default | +| -------------------------- | ----------- | ------- | +| `host` | Your Parseable Ingestor hostname or IP address. | `parseable` | +| `port` | TCP port of your Parseable Ingestor. | `8000` | +| `http_user` | Username for HTTP basic authentication. | `admin` | +| `http_passwd` | Password for HTTP basic authentication. | `admin` | +| `header` | **Required headers for Parseable integration.** Must include `X-P-Stream` (stream name) and `X-P-Log-Source` (telemetry type: `otel-logs`, `otel-metrics`, or `otel-traces`). | **Required** | +| `add_label` | Add custom labels to the telemetry data. | None | +| `log_response_payload` | Log the response payload from the server for debugging. | `False` | +| `metrics_uri` | Specify the HTTP URI for the target web server listening for metrics | `/v1/metrics` | +| `logs_uri` | Specify the HTTP URI for the target web server listening for logs | `/v1/logs` | +| `traces_uri` | Specify the HTTP URI for the target web server listening for traces | `/v1/traces` | +| `tls` | Enable or disable TLS/SSL encryption. | `Off` | + +### TLS / SSL + +The OpenTelemetry output plugin supports TLS/SSL. For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md). + +## Get started + +To get started with sending telemetry data to Parseable: + +1. Ensure your Parseable Ingestor is running and accessible. +2. Configure authentication credentials (username/password). +3. In your main Fluent Bit configuration file append the following: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + outputs: + # Send logs to Parseable + - name: opentelemetry + match: v1_logs + host: parseable + port: 8000 + logs_uri: /v1/logs + log_response_payload: true + tls: off + http_user: admin + http_passwd: admin + header: X-P-Stream otellogs + header: X-P-Log-Source otel-logs + add_label: app fluent-bit + + # Send metrics to Parseable + - name: opentelemetry + match: v1_metrics + host: parseable + port: 8000 + metrics_uri: /v1/metrics + log_response_payload: true + tls: off + http_user: admin + http_passwd: admin + header: X-P-Stream otelmetrics + header: X-P-Log-Source otel-metrics + add_label: app fluent-bit + + # Send traces to Parseable + - name: opentelemetry + match: v1_traces + host: parseable + port: 8000 + traces_uri: /v1/traces + log_response_payload: true + tls: off + http_user: admin + http_passwd: admin + header: X-P-Stream oteltraces + header: X-P-Log-Source otel-traces + add_label: app fluent-bit +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +# Send logs to Parseable +[OUTPUT] + Name opentelemetry + Match v1_logs + Host parseable + Port 8000 + Logs_uri /v1/logs + Log_response_payload True + Tls Off + Http_User admin + Http_Passwd admin + Header X-P-Stream otellogs + Header X-P-Log-Source otel-logs + Add_label app fluent-bit + +# Send metrics to Parseable +[OUTPUT] + Name opentelemetry + Match v1_metrics + Host parseable + Port 8000 + Metrics_uri /v1/metrics + Log_response_payload True + Tls Off + Http_User admin + Http_Passwd admin + Header X-P-Stream otelmetrics + Header X-P-Log-Source otel-metrics + Add_label app fluent-bit + +# Send traces to Parseable +[OUTPUT] + Name opentelemetry + Match v1_traces + Host parseable + Port 8000 + Traces_uri /v1/traces + Log_response_payload True + Tls Off + Http_User admin + Http_Passwd admin + Header X-P-Stream oteltraces + Header X-P-Log-Source otel-traces + Add_label app fluent-bit +``` + +{% endtab %} +{% endtabs %} + +## Stream Configuration + +Parseable uses streams to organize your telemetry data. The `X-P-Stream` header specifies which stream the data should be sent to: + +- **otellogs**: Stream for log data +- **otelmetrics**: Stream for metrics data +- **oteltraces**: Stream for trace data + +The `X-P-Log-Source` header helps identify the source of the telemetry data for better organization and filtering. And this has to be set to `otel-logs`, `otel-metrics`, or `otel-traces` based on the telemetry type. + +## Authentication + +Parseable supports HTTP basic authentication. Configure the `http_user` and `http_passwd` parameters with your Parseable server credentials. + +## References + +- [Parseable documentation](https://www.parseable.com/docs) +- [Parseable Fluent-bit integration](https://www.parseable.com/docs/datasource/log-agents/fluent-bit) From 624c7bf54ffe4ae22c053aa0bd6653842d1e6f0f Mon Sep 17 00:00:00 2001 From: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> Date: Tue, 5 Aug 2025 22:14:02 +0530 Subject: [PATCH 2/5] Update pipeline/outputs/parseable.md Co-authored-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> Signed-off-by: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> --- pipeline/outputs/parseable.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pipeline/outputs/parseable.md b/pipeline/outputs/parseable.md index bfdfb0a57..283837c81 100644 --- a/pipeline/outputs/parseable.md +++ b/pipeline/outputs/parseable.md @@ -31,8 +31,8 @@ The OpenTelemetry output plugin supports TLS/SSL. For more details about the pro To get started with sending telemetry data to Parseable: 1. Ensure your Parseable Ingestor is running and accessible. -2. Configure authentication credentials (username/password). -3. In your main Fluent Bit configuration file append the following: +1. Configure authentication credentials (username/password). +1. In your main Fluent Bit configuration file append the following: {% tabs %} {% tab title="fluent-bit.yaml" %} From 4bd663f48ab611b22d3ab882b545c6bf39f50c14 Mon Sep 17 00:00:00 2001 From: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> Date: Tue, 5 Aug 2025 22:14:09 +0530 Subject: [PATCH 3/5] Update pipeline/outputs/parseable.md Co-authored-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> Signed-off-by: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> --- pipeline/outputs/parseable.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pipeline/outputs/parseable.md b/pipeline/outputs/parseable.md index 283837c81..d21f312a4 100644 --- a/pipeline/outputs/parseable.md +++ b/pipeline/outputs/parseable.md @@ -140,9 +140,9 @@ pipeline: Parseable uses streams to organize your telemetry data. The `X-P-Stream` header specifies which stream the data should be sent to: -- **otellogs**: Stream for log data -- **otelmetrics**: Stream for metrics data -- **oteltraces**: Stream for trace data +- `otellogs`: Stream for log data +- `otelmetrics`: Stream for metrics data +- `oteltraces`: Stream for trace data The `X-P-Log-Source` header helps identify the source of the telemetry data for better organization and filtering. And this has to be set to `otel-logs`, `otel-metrics`, or `otel-traces` based on the telemetry type. From c04f030379a8f096cd191d156906b4a61653a36f Mon Sep 17 00:00:00 2001 From: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> Date: Tue, 5 Aug 2025 22:14:24 +0530 Subject: [PATCH 4/5] Update pipeline/outputs/parseable.md Co-authored-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> Signed-off-by: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> --- pipeline/outputs/parseable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/parseable.md b/pipeline/outputs/parseable.md index d21f312a4..ab95b810a 100644 --- a/pipeline/outputs/parseable.md +++ b/pipeline/outputs/parseable.md @@ -144,7 +144,7 @@ Parseable uses streams to organize your telemetry data. The `X-P-Stream` header - `otelmetrics`: Stream for metrics data - `oteltraces`: Stream for trace data -The `X-P-Log-Source` header helps identify the source of the telemetry data for better organization and filtering. And this has to be set to `otel-logs`, `otel-metrics`, or `otel-traces` based on the telemetry type. +The `X-P-Log-Source` header helps identify the source of the telemetry data for better organization and filtering. The header must be set to `otel-logs`, `otel-metrics`, or `otel-traces` based on the telemetry type. ## Authentication From f75b076e29e07e4d077ddede75c5a87eb7be63b7 Mon Sep 17 00:00:00 2001 From: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> Date: Tue, 5 Aug 2025 22:14:32 +0530 Subject: [PATCH 5/5] Update pipeline/outputs/parseable.md Co-authored-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> Signed-off-by: Debabrata Panigrahi <50622005+Debanitrkl@users.noreply.github.com> --- pipeline/outputs/parseable.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/pipeline/outputs/parseable.md b/pipeline/outputs/parseable.md index ab95b810a..a71a15a08 100644 --- a/pipeline/outputs/parseable.md +++ b/pipeline/outputs/parseable.md @@ -14,12 +14,12 @@ Stream logs, metrics, and traces to [Parseable](https://www.parseable.com) by ut | `port` | TCP port of your Parseable Ingestor. | `8000` | | `http_user` | Username for HTTP basic authentication. | `admin` | | `http_passwd` | Password for HTTP basic authentication. | `admin` | -| `header` | **Required headers for Parseable integration.** Must include `X-P-Stream` (stream name) and `X-P-Log-Source` (telemetry type: `otel-logs`, `otel-metrics`, or `otel-traces`). | **Required** | -| `add_label` | Add custom labels to the telemetry data. | None | +| `header` | Required headers for Parseable integration. Must include `X-P-Stream` (stream name) and `X-P-Log-Source` (telemetry type: `otel-logs`, `otel-metrics`, or `otel-traces`). | _none_ | +| `add_label` | Add custom labels to the telemetry data. | _none_ | | `log_response_payload` | Log the response payload from the server for debugging. | `False` | -| `metrics_uri` | Specify the HTTP URI for the target web server listening for metrics | `/v1/metrics` | -| `logs_uri` | Specify the HTTP URI for the target web server listening for logs | `/v1/logs` | -| `traces_uri` | Specify the HTTP URI for the target web server listening for traces | `/v1/traces` | +| `metrics_uri` | Specify the HTTP URI for the target web server listening for metrics. | `/v1/metrics` | +| `logs_uri` | Specify the HTTP URI for the target web server listening for logs. | `/v1/logs` | +| `traces_uri` | Specify the HTTP URI for the target web server listening for traces. | `/v1/traces` | | `tls` | Enable or disable TLS/SSL encryption. | `Off` | ### TLS / SSL