Skip to content

Extend postgresql_conf to support multiple config files using ParsedFile #1542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Extend postgresql_conf to support multiple config files using ParsedFile #1542

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d4054b4
Revert "New postgresql_conf provider, handles multiple configs"
ekohl Oct 23, 2023
d4e1c70
Extend postgresql_conf to support multiple config files
ekohl Oct 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 5 additions & 32 deletions REFERENCE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1526,7 +1526,6 @@ The following parameters are available in the `postgresql::server::config_entry`
* [`key`](#-postgresql--server--config_entry--key)
* [`value`](#-postgresql--server--config_entry--value)
* [`path`](#-postgresql--server--config_entry--path)
* [`comment`](#-postgresql--server--config_entry--comment)

##### <a name="-postgresql--server--config_entry--ensure"></a>`ensure`

Expand Down Expand Up @@ -1560,14 +1559,6 @@ Path for postgresql.conf

Default value: `$postgresql::server::postgresql_conf_path`

##### <a name="-postgresql--server--config_entry--comment"></a>`comment`

Data type: `Optional[String[1]]`

Defines the comment for the setting. The # is added by default.

Default value: `undef`

### <a name="postgresql--server--database"></a>`postgresql::server::database`

Define for creating a database.
Expand Down Expand Up @@ -4309,12 +4300,6 @@ This type allows puppet to manage postgresql.conf parameters.

The following properties are available in the `postgresql_conf` type.

##### `comment`

Valid values: `%r{^[\w\W]+$}`

The comment to set for this parameter.

##### `ensure`

Valid values: `present`, `absent`
Expand All @@ -4323,46 +4308,34 @@ The basic property that the resource should be in.

Default value: `present`

##### `value`
##### `target`

Valid values: `%r{^\S(.*\S)?$}`
The path to postgresql.conf

##### `value`

The value to set for this parameter.

#### Parameters

The following parameters are available in the `postgresql_conf` type.

* [`key`](#-postgresql_conf--key)
* [`name`](#-postgresql_conf--name)
* [`provider`](#-postgresql_conf--provider)
* [`target`](#-postgresql_conf--target)

##### <a name="-postgresql_conf--key"></a>`key`

Valid values: `%r{^[\w.]+$}`

The Postgresql parameter to manage.

##### <a name="-postgresql_conf--name"></a>`name`

Valid values: `%r{^[\w.]+$}`

namevar

A unique title for the resource.
The postgresql parameter name to manage.

##### <a name="-postgresql_conf--provider"></a>`provider`

The specific backend to use for this `postgresql_conf` resource. You will seldom need to specify this --- Puppet will
usually discover the appropriate provider for your platform.

##### <a name="-postgresql_conf--target"></a>`target`

Valid values: `%r{^/\S+[a-z0-9(/)-]*\w+.conf$}`

The path to the postgresql config file

### <a name="postgresql_conn_validator"></a>`postgresql_conn_validator`

Verify that a connection can be successfully established between a node
Expand Down
20 changes: 20 additions & 0 deletions examples/multiple_confs.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
postgresql_conf { '/tmp/first-postgresql.conf:other':
value => 'bla',
}

postgresql_conf { '/tmp/first-postgresql.conf:port':
value => 5432,
}

postgresql_conf { '/tmp/second-postgresql.conf:other':
value => 'bla',
}

postgresql_conf { '/tmp/second-postgresql.conf:port':
value => 5433,
}

# TODO: make target optional
#postgresql_conf { 'port':
# value => 5434,
#}
45 changes: 45 additions & 0 deletions lib/puppet/provider/postgresql_conf/parsed.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# frozen_string_literal: true

require 'puppet/provider/parsedfile'

Puppet::Type.type(:postgresql_conf).provide(
:parsed,
parent: Puppet::Provider::ParsedFile,
default_target: '/etc/postgresql.conf',
filetype: :flat,
) do
desc 'Set key/values in postgresql.conf.'

text_line :comment, match: %r{^\s*#}
text_line :blank, match: %r{^\s*$}

record_line :parsed,
fields: ['key', 'value', 'comment'],
optional: ['comment'],
match: %r{^\s*([\w.]+)\s*=?\s*(.*?)(?:\s*#\s*(.*))?\s*$},
to_line: proc { |h|
# simple string and numeric values don't need to be enclosed in quotes
val = if h[:value].is_a?(Numeric)
h[:value].to_s
elsif h[:value].is_a?(Array)
# multiple listen_addresses specified as a string containing a comma-speparated list
h[:value].join(', ')
else
h[:value]
end
dontneedquote = val.match(%r{^(\d+.?\d+|\w+)$})
dontneedequal = h[:key].match(%r{^(include|include_if_exists)$}i)

str = h[:key].downcase # normalize case
str += dontneedequal ? ' ' : ' = '
str += "'" unless dontneedquote && !dontneedequal
str += val
str += "'" unless dontneedquote && !dontneedequal
str += " # #{h[:comment]}" unless h[:comment].nil? || h[:comment] == :absent
str
},
post_parse: proc { |h|
h[:key].downcase! # normalize case
h[:value].gsub!(%r{(^'|'$)}, '') # strip out quotes
}
end
167 changes: 0 additions & 167 deletions lib/puppet/provider/postgresql_conf/ruby.rb
View file
Open in desktop

This file was deleted.

40 changes: 18 additions & 22 deletions lib/puppet/type/postgresql_conf.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,40 +2,36 @@

Puppet::Type.newtype(:postgresql_conf) do
@doc = 'This type allows puppet to manage postgresql.conf parameters.'

ensurable

newparam(:name) do
desc 'A unique title for the resource.'
newvalues(%r{^[\w.]+$})
def self.title_patterns
[
[ /^(.+\.conf):([\w.]+)$/m, [ [:target], [:key] ] ],
# TODO: make target optional
#[ /^([\w.]+)$/m, [ [:key] ] ],
Comment on lines +11 to +12
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't have time to figure out how title_patterns works with matching, but it probably isn't hard.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that really depends on the point of view :D

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It comes down to understanding https://github.com/puppetlabs/puppet/blob/945beb6ef0200b1bc7879b62131282a61fc9a935/lib/puppet/resource.rb#L625-L663 and finding the right combination.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, couldn't help myself. I think the commented code should work. The reason it's commented is that I didn't know if it would really work. But this part suggests the patterns are correct:

require 'test/unit'

TITLE_PATTERNS = [ 
  [ /^(.+\.conf):([\w.]+)$/m, [ [:target], [:key] ] ],
  [ /^([\w.]+)$/m, [ [:key] ] ],
]

def match(title)
  h = {}
  TITLE_PATTERNS.each do |regexp, symbols_and_lambdas|
    captures = regexp.match(title.to_s)
    if captures
      symbols_and_lambdas.zip(captures[1..-1]).each do |symbol_and_lambda,capture|
        symbol, proc = symbol_and_lambda
        # Many types pass "identity" as the proc; we might as well give
        # them a shortcut to delivering that without the extra cost.
        # 
        # Especially because the global type defines title_patterns and
        # uses the identity patterns.
        # 
        # This was worth about 8MB of memory allocation saved in my
        # testing, so is worth the complexity for the API.
        if proc then
          h[symbol] = proc.call(capture)
        else
          h[symbol] = capture
        end
      end
      return h 
    end
  end
end

class MatcherTest < Test::Unit::TestCase
  def test_with_conf
    assert_equal({target: '/postgresql.conf', key: 'foo'}, match('/postgresql.conf:foo'))
  end

  def test_without_conf
    assert_equal({key: 'foo'}, match('foo'))
  end
end
$ ruby title_patterns.rb 
Loaded suite title_patterns
Started
Finished in 0.000252056 seconds.
--------------------------------------------------------------------------------------
2 tests, 2 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
--------------------------------------------------------------------------------------
7934.74 tests/s, 7934.74 assertions/s

]
end

newparam(:key) do
desc 'The Postgresql parameter to manage.'

newparam(:key, namevar: true) do
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept key instead of name to force me to implement ParsedFile agnostic of the name parameter. At this point I think it could revert back to name.

desc 'The postgresql parameter name to manage.'
isrequired
newvalues(%r{^[\w.]+$})
end

newproperty(:value) do
desc 'The value to set for this parameter.'
newvalues(%r{^\S(.*\S)?$})
end

munge do |value|
if value.to_i.to_s == value
value.to_i
elsif value.to_f.to_s == value
value.to_f
newparam(:target, namevar: true) do
desc 'The path to postgresql.conf'
defaultto do
if @resource.class.defaultprovider.ancestors.include?(Puppet::Provider::ParsedFile)
@resource.class.defaultprovider.default_target
else
value
nil
end
end
end

newproperty(:comment) do
desc 'The comment to set for this parameter.'
newvalues(%r{^[\w\W]+$})
end

newparam(:target) do
desc 'The path to the postgresql config file'
newvalues(%r{^/\S+[a-z0-9(/)-]*\w+.conf$})
end
end
Loading