-
Notifications
You must be signed in to change notification settings - Fork 90
Description
I am trying to setup logstash top read the content of azure blob storage account and write the logs into elasticsearch.
Here are specifications:
VM: Ubuntu 24.04
Logstash Version: 8.19.9
Azure blob storage plugin Version: 0.12.10
Version Details:
ctuser@elasticlogstash:$ /usr/share/logstash/bin/logstash --version$ /usr/share/logstash/bin/logstash-plugin list --verbose azure_blob_storage
Using bundled JDK: /usr/share/logstash/jdk
logstash 8.19.9
ctuser@elasticlogstash:
Using bundled JDK: /usr/share/logstash/jdk
logstash-input-azure_blob_storage (0.12.10)
ctuser@elasticlogstash:~$
Input content of logstash config file
input {
azure_blob_storage {
storageaccount => "saccount1"
access_key => "Valid storageaccountkey"
container => "sacontainer"
prefix => "*"
registry_path => "/var/lib/logstash/local_registry.dat"
# Point registry to a clean local file to avoid the 404 error
interval => 300
codec => "plain"
# We removed 'registry_create_policy' to let the plugin
# initialize naturally without forcing a 'start_over' state
}
}
Upon running the logstash service, it fails with below error:
alse[2025-12-30T11:05:44,620][WARN ][logstash.filters.grok ][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated [2025-12-30T11:05:44,697][INFO ][logstash.filters.kv ][main] ECS compatibility is enabled buttargetoption was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set thetargetoption to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message) [2025-12-30T11:05:44,699][WARN ][logstash.filters.grok ][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated [2025-12-30T11:05:44,736][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/azure-xshield-cef.conf"], :thread=>"#<Thread:0x7363e079 /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:138 run>"} [2025-12-30T11:05:44,832][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>9, :ecs_compatibility=>:v8} [2025-12-30T11:05:45,617][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.88} [2025-12-30T11:05:45,622][INFO ][logstash.inputs.azureblobstorage][main] === azure_blob_storage 0.12.10 / main / db192f / ruby 3.1.4p0 === [2025-12-30T11:05:45,623][INFO ][logstash.inputs.azureblobstorage][main] If this plugin doesn't work, please raise an issue in https://github.com/janmg/logstash-input-azure_blob_storage [2025-12-30T11:05:45,625][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"} [2025-12-30T11:05:45,638][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} [2025-12-30T11:05:45,779][ERROR][logstash.inputs.azureblobstorage][main]**[db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] caught: undefined local variable or methodpath' for #LogStash::Inputs::AzureBlobStorage:0x3876fc4d**
[2025-12-30T11:05:45,780][ERROR][logstash.inputs.azureblobstorage][main][db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] loading registry failed for attempt 1 of 3
[2025-12-30T11:05:45,822][ERROR][logstash.inputs.azureblobstorage][main][db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] caught: undefined local variable or method path' for #<LogStash::Inputs::AzureBlobStorage:0x3876fc4d> [2025-12-30T11:05:45,823][ERROR][logstash.inputs.azureblobstorage][main][db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] loading registry failed for attempt 2 of 3 [2025-12-30T11:05:45,853][ERROR][logstash.inputs.azureblobstorage][main][db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] caught: undefined local variable or method path' for #LogStash::Inputs::AzureBlobStorage:0x3876fc4d
[2025-12-30T11:05:45,853][ERROR][logstash.inputs.azureblobstorage][main][db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] loading registry failed for attempt 3 of 3
[2025-12-30T11:05:46,498][ERROR][logstash.inputs.azureblobstorage][main][db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] caught: Index -1 out of bounds for length 315 while trying to list blobs
[2025-12-30T11:10:45,553][ERROR][logstash.inputs.azureblobstorage][main][db192f409656da65ec3314716f05a2d9f26996363077ea7322a0fae28c5b9cf6] caught: Index -1 out of bounds for length 315 while trying to list blobs
Please help us resolve the issue.