I am getting an error when trying to use a list in my pillar file. The use case is that one of the fields I want to add to my logstash record is a tag, which can have multiple values.
When I run Logstash locally, I can use the following filter:
filter {
if [path] =~ "nginx" {
grok {
match => { "message" => "%{DATESTAMP} \[%{LOGLEVEL:loglevel}\] %{GREEDYDATA:message}" }
}
mutate {
add_field => { "my_string_field" => "this-works" }
add_field => { "my_list_field" => ['this-also-works','so-does-this'] }
}
}
}
My corresponding pillar file for doing this with this salt formula:
-
plugin_name: grok
cond: 'else if [type] == "nginx"'
match:
message: '%{DATESTAMP} \[%{LOGLEVEL:loglevel}\] %{GREEDYDATA:message}'
add_field:
my_string_field: 'this-works'
my_list_field: ['this-does-not-work','neither-does-this']
The error I receive in Salt:
Comment: Unable to manage file: Jinja error: cannot concatenate 'str' and 'list' objects
/var/cache/salt/minion/files/base/logstash/map.jinja(43):
---
[...]
{{- output_indented(col, (key + ' => "' + value|string + '"')) }}
{%- elif value is mapping %}
{{- output_indented(col, (key + ' => {')) }}
{%- set col = col + logstash.indent %}
{%- for attr_key, attr_value in value.items() %}
{{- output_indented(col, (attr_key + ' => "' + attr_value + '"')) }} <======================
{%- endfor %}
{%- set col = col - logstash.indent %}
{{- output_indented(col, '}') }}
{%- elif value is iterable %}
{{- output_indented(col, key + ' => [') }}
[...]
---
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 368, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "<template>", line 2, in top-level template code
File "/usr/lib/python2.7/dist-packages/jinja2/runtime.py", line 551, in _invoke
rv = self._func(*arguments)
File "/var/cache/salt/minion/files/base/logstash/map.jinja", line 43, in template
{{- output_indented(col, (attr_key + ' => "' + attr_value + '"')) }}
TypeError: cannot concatenate 'str' and 'list' objects
It seems like the config file only expects strings, not list values. Ideally it should accept any input that works in Logstash. I'm not totally sure how the formatting works, but perhaps it could do something like this:
{%- for attr_key, attr_value in value.items() %}
{% if attr_value is iterable %}{{- output_indented(col, (attr_key + ' => ' + attr_value + '')) }}
{% else %}{{- output_indented(col, (attr_key + ' => "' + attr_value + '"')) }} {% endif %}
{%- endfor %}
I am getting an error when trying to use a list in my pillar file. The use case is that one of the fields I want to add to my logstash record is a tag, which can have multiple values.
When I run Logstash locally, I can use the following filter:
My corresponding pillar file for doing this with this salt formula:
The error I receive in Salt:
It seems like the config file only expects strings, not list values. Ideally it should accept any input that works in Logstash. I'm not totally sure how the formatting works, but perhaps it could do something like this: