Skip to content

Unable to manage map file: Jinja cannot concatenate 'str' and 'list' objects #28

@ninapavlich

Description

@ninapavlich

I am getting an error when trying to use a list in my pillar file. The use case is that one of the fields I want to add to my logstash record is a tag, which can have multiple values.

When I run Logstash locally, I can use the following filter:

filter {
  if [path] =~ "nginx" {
    grok {
      match => { "message" => "%{DATESTAMP} \[%{LOGLEVEL:loglevel}\] %{GREEDYDATA:message}" }
    }
    mutate {
      add_field => { "my_string_field" => "this-works" }
      add_field => { "my_list_field" => ['this-also-works','so-does-this'] }
    }
  }
}

My corresponding pillar file for doing this with this salt formula:

-
  plugin_name: grok
  cond: 'else if [type] == "nginx"'
  match:
    message: '%{DATESTAMP} \[%{LOGLEVEL:loglevel}\] %{GREEDYDATA:message}'
  add_field:
    my_string_field: 'this-works'
    my_list_field: ['this-does-not-work','neither-does-this']

The error I receive in Salt:

Comment: Unable to manage file: Jinja error: cannot concatenate 'str' and 'list' objects
          /var/cache/salt/minion/files/base/logstash/map.jinja(43):
          ---
          [...]
                {{- output_indented(col, (key + ' => "' + value|string + '"')) }}
              {%- elif value is mapping %}
                {{- output_indented(col, (key + ' => {')) }}
                {%- set col = col + logstash.indent %}
                {%- for attr_key, attr_value in value.items() %}
                  {{- output_indented(col, (attr_key + ' => "' + attr_value + '"')) }}    <======================
                {%- endfor %}
                {%- set col = col - logstash.indent %}
                {{- output_indented(col, '}') }}
              {%- elif value is iterable %}
                {{- output_indented(col, key + ' => [') }}
          [...]
          ---
          Traceback (most recent call last):
            File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 368, in render_jinja_tmpl
              output = template.render(**decoded_context)
            File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
              return self.environment.handle_exception(exc_info, True)
            File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
              reraise(exc_type, exc_value, tb)
            File "<template>", line 2, in top-level template code
            File "/usr/lib/python2.7/dist-packages/jinja2/runtime.py", line 551, in _invoke
              rv = self._func(*arguments)
            File "/var/cache/salt/minion/files/base/logstash/map.jinja", line 43, in template
              {{- output_indented(col, (attr_key + ' => "' + attr_value + '"')) }}
          TypeError: cannot concatenate 'str' and 'list' objects

It seems like the config file only expects strings, not list values. Ideally it should accept any input that works in Logstash. I'm not totally sure how the formatting works, but perhaps it could do something like this:

 {%- for attr_key, attr_value in value.items() %}
      {% if attr_value is iterable %}{{- output_indented(col, (attr_key + ' => ' + attr_value + '')) }} 
      {% else %}{{- output_indented(col, (attr_key + ' => "' + attr_value + '"')) }} {% endif %}
 {%- endfor %}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions