Skip to content

Conversation

@isaiahfrantz
Copy link

@isaiahfrantz isaiahfrantz commented Jan 28, 2026

Add String Manipulation and Pattern Matching Interpolation Functions

Summary

This PR adds a suite of new interpolation functions to Hiera, enabling powerful string manipulation, pattern matching, and version comparison directly within data values and hierarchy paths.
This change will solve a major pain point with hera by providing a way to construct data file names that can be more easily matched against without needing unnecessary new facts or global variables.

New Features

1. substring(string, start[, count])

Extract a portion of a string using 0-based indexing.

slice: "%{substring('abcdef', 2)}"      # Returns 'cdef'
slice: "%{substring('abcdef', 2, 3)}"   # Returns 'cde'
last3: "%{substring('abcdef', -3)}"     # Returns 'def' (negative index)

# In hierarchy paths
path: "nodes/%{substring($certname, 0, 4)}.yaml"

2. match(string, pattern)

Test if a string matches a regex or contains a substring. Returns 'true' or 'false'.

is_web: "%{match($role, '/^web/')}"
has_db: "%{match($hostname, 'db')}"
case_insensitive: "%{match('HELLO', '/hello/i')}"

# In hierarchy paths
path: "nodes/%{match($certname, 'db')}.yaml"

Supports regex flags: i (case-insensitive), m (multiline), x (extended)

3. grep_captures(string, '/regex/'[, separator][, [groups]])

Extract capture groups from a regex match and join them.

# All groups concatenated (default)
all: "%{grep_captures('abc-123-def', '/([a-z]+)-(\d+)-([a-z]+)/')}"
# Returns: 'abc123def'

# All groups with separator
sep: "%{grep_captures('abc-123-def', '/([a-z]+)-(\d+)-([a-z]+)/', '-')}"
# Returns: 'abc-123-def'

# Specific groups (3rd arg auto-detected as array)
selected: "%{grep_captures('abc-123-def', '/([a-z]+)-(\d+)-([a-z]+)/', [1,3])}"
# Returns: 'abcdef'

# Specific groups with separator
custom: "%{grep_captures('abc-123-def', '/([a-z]+)-(\d+)-([a-z]+)/', '_', [1,3])}"
# Returns: 'abc_def'

# In hierarchy paths
path: "env/%{grep_captures($hostname, '/\w+-(\w+)-\w+/')}.yaml"

Key features:

  • Default separator: '' (empty string)
  • Default groups: all capture groups
  • Third argument auto-detected: array = groups, quoted string = separator

4. Version Comparison Functions

Compare semantic versions using Gem::Version:

is_newer: "%{version_gt($app_version, '2.0.0')}"
is_compatible: "%{version_gte($ruby_version, '2.7.0')}"
needs_upgrade: "%{version_lt($os_version, '8.0')}"
is_current: "%{version_lte($version, '3.0.0')}"

# In hierarchy paths
path: "versions/%{version_gte($app_version, '2.0.0')}.yaml"
  • Handles semantic versioning correctly (1.10.0 > 1.9.0)
  • Supports prerelease versions (1.0.0.alpha < 1.0.0)

Technical Changes

Changed

  • RX_METHOD_AND_ARG regex updated to support function arguments containing parentheses
  • Argument parser now handles array literals [1,2,3] for capture group indices

Fixed

  • Regex literals with escaped slashes (e.g., /a\/b/) now parsed correctly
  • Empty quoted strings ('' or "") handled correctly as function arguments

Single-Quoted Regex Patterns

All regex patterns should use single quotes ('/pattern/') for:

  • Curly brace quantifier support: '/\d{3,5}/'
  • No YAML double-escaping needed: '/\d+/' instead of /\\d+/

Files Changed

  • lib/hiera/interpolate.rb - Core implementation
  • spec/unit/interpolate_spec.rb - Tests
  • spec/unit/fixtures/interpolate/data/weird_keys.yaml - Test fixtures
  • README.md - Documentation
  • CHANGELOG.md - Changelog
  • docs/tutorials/interpolation_functions.md - Tutorial

Testing

bundle exec rspec spec/unit/interpolate_spec.rb
# 86 examples, 0 failures

Use Cases

All functions can be used in both data values and hierarchy paths.

Dynamic hierarchy paths using all functions:

# hiera.yaml
:hierarchy:
  # substring - group nodes by certname prefix
  - name: "node prefix"
    path: "nodes/%{substring($certname, 0, 4)}.yaml"

  # grep_captures - extract environment from structured hostname (web01-prod-dc1)
  - name: "environment from hostname"
    path: "env/%{grep_captures($hostname, '/\w+-(\w+)-\w+/')}.yaml"

  # match - conditional path based on pattern (returns 'true' or 'false')
  - name: "production check"
    path: "is_prod/%{match($environment, '/^prod/')}.yaml"

  # version functions - path based on version comparison
  - name: "version compatibility"
    path: "compat/%{version_gte($app_version, '2.0.0')}.yaml"

  - name: "common"
    path: "common.yaml"

Data value examples:

# Hostname: web01-prod-dc1.example.com
env: "%{grep_captures($hostname, '/\w+-(\w+)-\w+/')}"  # Returns 'prod'
dc: "%{grep_captures($hostname, '/\w+-\w+-(\w+)/')}"   # Returns 'dc1'
short_name: "%{substring($certname, 0, 8)}"
is_webserver: "%{match($role, '/^web/')}"
use_new_api: "%{version_gte($app_version, '2.0.0')}"

@isaiahfrantz isaiahfrantz requested a review from a team as a code owner January 28, 2026 19:27
@CLAassistant
Copy link

CLAassistant commented Jan 28, 2026

CLA assistant check
All committers have signed the CLA.

Adds substring, match, grep_captures, and version comparison functions
for string manipulation and pattern matching in Hiera interpolations.

- substring(string, start, count) extracts portions of strings
- match(string, pattern) tests string matching with regex or literal
- grep_captures(string, regex, separator, groups) extracts regex captures
- version_gt/gte/lt/lte(a, b) compares semantic versions using Gem::Version

Updates interpolation parser to handle complex
@isaiahfrantz isaiahfrantz force-pushed the add-path-interpolation-functions branch from 14bae4a to be65420 Compare January 28, 2026 20:06
@isaiahfrantz isaiahfrantz changed the title (maint) Add interpolation functions to Hiera feat: Add interpolation functions to Hiera Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants