diff --git a/CHANGELOG.md b/CHANGELOG.md index be546595..3bc3289c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,13 @@ First release of the `oddly.elasticstack` collection, forked from - Certificate expiry warnings and tag-driven renewal (`--tags certificates`, `--tags renew_es_cert`, `--tags renew_ca`). - Logstash standalone certificate mode for independent deployments. +- Logstash role writes the unencrypted PKCS#8 PEM key directly to + `.key` (the `-pkcs8.key` filename is gone). Standalone mode now + uses `community.crypto.openssl_privatekey` (PKCS#8 native) instead of + `openssl genrsa` + a separate `openssl pkcs8` conversion step. The new + `logstash_tls_copy_certs` variable lets external mode reference cert files + in place instead of copying them — useful with certmonger or cert-manager + where the renewal tool rotates the files and restarts Logstash out of band. - Beats Filebeat `filestream` input type for 9.x (replacing deprecated `log`). - Logstash `elastic_agent` input plugin support. - Elasticsearch `cluster_settings` for runtime cluster configuration via API. diff --git a/docs/guide/tls.md b/docs/guide/tls.md index fd0f08f3..7688359e 100644 --- a/docs/guide/tls.md +++ b/docs/guide/tls.md @@ -13,7 +13,7 @@ graph TD CA -->|"P12 keystore
per node"| ES_T["ES Transport :9300
Node-to-node encryption"] CA -->|"P12 keystore
per node"| ES_H["ES HTTP :9200
Client-to-node encryption"] CA -->|"P12 keystore"| KB["Kibana
ES connection + optional HTTPS"] - CA -->|"PEM cert + PKCS8 key
+ P12 keystore"| LS["Logstash
Beats input + ES output"] + CA -->|"PEM cert + key
+ P12 keystore"| LS["Logstash
Beats input + ES output"] CA -->|"PEM cert + key
per host"| BT["Beats
Logstash/ES output"] ES_T -.->|"mutual TLS"| ES_T @@ -41,7 +41,7 @@ The collection uses different formats depending on what each service expects nat | Elasticsearch (HTTP) | PKCS12 | `-http.p12` | | Kibana | PKCS12 | `-kibana.p12` | | Logstash (ES output) | PKCS12 | `keystore.pfx` | -| Logstash (Beats input) | PEM | `.crt` + `-pkcs8.key` | +| Logstash (Beats input) | PEM | `-server.crt` + `.key` | | Beats | PEM | `-beats.crt` + `-beats.key` | When using external certificates, both PEM (`.crt`, `.pem`) and PKCS12 (`.p12`, `.pfx`) are accepted. Format is auto-detected by probing the file content with `openssl`, not from the file extension. @@ -177,6 +177,36 @@ You don't always need to set every variable. The roles apply sensible defaults: By default (`*_tls_remote_src: false`), certificate files are on the Ansible controller and get copied to each managed node. Set `*_tls_remote_src: true` when files are already on the managed nodes — provisioned by certbot, cloud-init, Vault agent, or a configuration management tool. +### Logstash with certmonger / cert-manager (hands-off rotation) + +For Logstash specifically, certificate copies can get out of sync with automatic renewals. When certmonger or cert-manager rotates the cert, the copy under `/etc/logstash/certs/` becomes stale until the next Ansible run. + +Set `logstash_tls_copy_certs: false` to skip the copy. The pipeline config then references the original paths directly, and Logstash picks up the new cert on the next restart — which the renewal tool can trigger itself. + +```yaml title="group_vars/all.yml" +logstash_cert_source: external +logstash_tls_copy_certs: false +logstash_tls_certificate_file: /etc/pki/logstash/server.crt +logstash_tls_key_file: /etc/pki/logstash/server.key +logstash_tls_ca_file: /etc/pki/logstash/ca.crt +``` + +Logstash runs as the `logstash` user and must be able to read the key. For certmonger, request the cert with the right ownership and restart hook: + +```bash +getcert request \ + -f /etc/pki/logstash/server.crt -k /etc/pki/logstash/server.key \ + -o root:logstash -m 0640 \ + -O root:logstash -M 0644 \ + -C 'systemctl try-reload-or-restart logstash' \ + -c local -I logstash +``` + +The `-o root:logstash -m 0640` ensures Logstash can read the key; certmonger preserves this ownership across every renewal. The `-C` post-save hook replaces the Ansible re-run. Verified on Debian with certmonger 0.79 and tested on both certmonger and openssl-generated keys, which both produce PKCS#8 PEM directly. + +!!! note + This path does not generate `keystore.pfx`. If you need the Elasticsearch output's P12 keystore (`logstash_output_elasticsearch: true` with security), either leave `logstash_tls_copy_certs: true` or set `logstash_output_elasticsearch: false` and manage the ES output yourself. + ## Mode 3: Inline PEM content from a secrets manager When certificates come from HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or any system that provides certificate content rather than files, use the `*_content` variables. They take precedence over file paths. @@ -335,6 +365,7 @@ beats_cert_source: elasticsearch_ca # or external | | `logstash_input_beats_ssl` | inherited | TLS on Beats input | | | `logstash_tls_certificate_file` | `""` | Certificate path (external) | | | `logstash_tls_remote_src` | `false` | Certs on managed node | +| | `logstash_tls_copy_certs` | `true` | Copy external certs into `logstash_certs_dir`; set `false` for hands-off rotation | | | `logstash_cert_force_regenerate` | `false` | Force cert regen | | **Beats** | `beats_cert_source` | `elasticsearch_ca` | `elasticsearch_ca` or `external` | | | `beats_security` | `false` | Enable TLS (opt-in) | diff --git a/docs/reference/logstash.md b/docs/reference/logstash.md index 6f02e946..4bce36c8 100644 --- a/docs/reference/logstash.md +++ b/docs/reference/logstash.md @@ -404,6 +404,7 @@ logstash_cert_force_regenerate: false # logstash_tls_key_file: "/path/to/server.key" # logstash_tls_ca_file: "/path/to/ca.crt" # logstash_tls_remote_src: false +# logstash_tls_copy_certs: true # set to false for hands-off rotation (certmonger, cert-manager) ``` `logstash_cert_source` @@ -414,7 +415,7 @@ logstash_cert_force_regenerate: false - **`external`** — uses certificate files you provide via `logstash_tls_certificate_file`, `logstash_tls_key_file`, and optionally `logstash_tls_ca_file`. The role copies them into place but does NOT create the ES user/role (assumes you manage that separately). `logstash_certs_dir` -: Directory on the Logstash host where TLS certificates, keys, and the CA bundle are stored. The role creates this directory and writes the PEM certificate, PKCS8 key, P12 keystore, and CA certificate here. +: Directory on the Logstash host where TLS certificates, keys, and the CA bundle are stored. The role creates this directory and writes the PEM certificate, the unencrypted PKCS#8 PEM key, the P12 keystore for the Elasticsearch output, and the CA certificate here. `logstash_tls_key_passphrase` : Passphrase used when generating the P12 keystore for the Elasticsearch output plugin. Also used as the `ssl_keystore_password` in the output config. @@ -432,11 +433,14 @@ logstash_cert_force_regenerate: false : Force TLS certificate regeneration on the next run, even if current certificates are still valid. Useful after a CA rotation or if you suspect a key compromise. The role resets this to `false` internally after regeneration. `logstash_tls_certificate_file` / `logstash_tls_key_file` / `logstash_tls_ca_file` -: Paths to externally-managed certificate files. Only used when `logstash_cert_source: external`. The role copies these into `logstash_certs_dir`. +: Paths to externally-managed certificate files. Only used when `logstash_cert_source: external`. The role copies these into `logstash_certs_dir` by default; set `logstash_tls_copy_certs: false` to reference them in place instead. `logstash_tls_remote_src` : When `true`, the external certificate files are already on the remote host and are copied locally (no upload from the Ansible controller). Defaults to `false`. +`logstash_tls_copy_certs` +: Whether to copy the external `logstash_tls_*_file` paths into `logstash_certs_dir`. Defaults to `true`, which preserves the existing copy-and-rename flow. Set to `false` for a hands-off setup where the pipeline config references the original paths directly — useful for certmonger, cert-manager, or any tool that rotates the key files out-of-band. The `logstash` user must be able to read the key; typical on-disk permissions are `root:logstash 0640`. + ### Dead Letter Queue ```yaml @@ -610,12 +614,18 @@ The standard pipeline uses three config files in `/etc/logstash/conf.d/main/`: Logstash loads `.conf` files alphabetically, so the numbering ensures correct execution order. When you set `logstash_custom_pipeline`, the role writes a single `pipeline.conf` and removes the three numbered files. Switching back from custom to standard mode removes `pipeline.conf`. -### PKCS8 key requirement +### Key format + +The Beats and Elastic Agent inputs need an unencrypted PKCS#8 PEM key (`-----BEGIN PRIVATE KEY-----`) per the upstream plugin contract. The Elasticsearch output uses a P12 keystore. The role produces both from the same certificate: + +- **`.key`** — unencrypted PKCS#8 PEM, read by the Beats / Elastic Agent inputs +- **`keystore.pfx`** — P12 keystore (cert + key), read by the Elasticsearch output plugin -Logstash input plugins (Beats, Elastic Agent) require an unencrypted PKCS8 key, while the Elasticsearch output plugin uses a P12 keystore. The role generates both formats from the same certificate: +Where the key comes from depends on the mode: -- **P12 cert** — copied as `keystore.pfx` for the ES output plugin -- **PEM cert** — extracted from a ZIP, with the encrypted key converted to unencrypted PKCS8 via `openssl pkcs8 -topk8 -nocrypt` +- **`elasticsearch_ca`** — `elasticsearch-certutil --pem` emits an encrypted PKCS#1 key inside a zip. The role unpacks it and decrypts to PKCS#8 PEM in one `openssl pkcs8 -topk8 -nocrypt` step, writing directly to `.key`. +- **`standalone`** — generated with `community.crypto.openssl_privatekey`, which emits PKCS#8 PEM natively. No post-processing. +- **`external`** — the file you supply at `logstash_tls_key_file` is used as-is. Most modern key generators (certmonger, `openssl genpkey`, `openssl genrsa` on OpenSSL 3.0+) emit PKCS#8 by default. If you have a legacy PKCS#1 key, convert it once with `openssl pkcs8 -topk8 -nocrypt`. ### ES 9.x vs 8.x SSL syntax @@ -626,8 +636,8 @@ The Logstash input and output configuration templates use different SSL paramete ``` # Input (Beats / Elastic Agent) ssl_enabled => true - ssl_certificate => "/etc/logstash/certs/..." - ssl_key => "/etc/logstash/certs/...-pkcs8.key" + ssl_certificate => "/etc/logstash/certs/-server.crt" + ssl_key => "/etc/logstash/certs/.key" ssl_client_authentication => required ssl_certificate_authorities => ["/etc/logstash/certs/ca.crt"] @@ -643,8 +653,8 @@ The Logstash input and output configuration templates use different SSL paramete ``` # Input (Beats / Elastic Agent) ssl => true - ssl_certificate => "/etc/logstash/certs/..." - ssl_key => "/etc/logstash/certs/...-pkcs8.key" + ssl_certificate => "/etc/logstash/certs/-server.crt" + ssl_key => "/etc/logstash/certs/.key" ssl_verify_mode => force_peer ssl_certificate_authorities => ["/etc/logstash/certs/ca.crt"] @@ -655,7 +665,7 @@ The Logstash input and output configuration templates use different SSL paramete cacert => "/etc/logstash/certs/ca.crt" ``` -The template switches automatically based on `elasticstack_release | int >= 9`. +The template switches automatically based on `elasticstack_release | int >= 9`. With `logstash_cert_source: external` and `logstash_tls_copy_certs: false`, the `ssl_certificate`, `ssl_key`, and `ssl_certificate_authorities` values point at the paths you supplied rather than `logstash_certs_dir`. ### Event enrichment (ident stamping) diff --git a/molecule/logstash_ssl/converge.yml b/molecule/logstash_ssl/converge.yml index 924345fb..c7c3475c 100644 --- a/molecule/logstash_ssl/converge.yml +++ b/molecule/logstash_ssl/converge.yml @@ -18,7 +18,7 @@ # External certificate configuration logstash_cert_source: external logstash_tls_certificate_file: /tmp/test-certs/server.crt - logstash_tls_key_file: /tmp/test-certs/server-pkcs8.key + logstash_tls_key_file: /tmp/test-certs/server.key logstash_tls_ca_file: /tmp/test-certs/ca.crt logstash_tls_remote_src: true logstash_extra_outputs: | diff --git a/molecule/logstash_ssl/verify.yml b/molecule/logstash_ssl/verify.yml index 64d74e53..2c9eff06 100644 --- a/molecule/logstash_ssl/verify.yml +++ b/molecule/logstash_ssl/verify.yml @@ -24,11 +24,11 @@ register: server_cert failed_when: not server_cert.stat.exists - - name: Check PKCS8 key exists + - name: Check server key exists ansible.builtin.stat: - path: "/etc/logstash/certs/{{ inventory_hostname }}-pkcs8.key" - register: pkcs8_key - failed_when: not pkcs8_key.stat.exists + path: "/etc/logstash/certs/{{ inventory_hostname }}.key" + register: server_key + failed_when: not server_key.stat.exists - name: Check CA certificate exists ansible.builtin.stat: @@ -40,10 +40,24 @@ ansible.builtin.assert: that: - server_cert.stat.mode == '0640' - - pkcs8_key.stat.mode == '0640' + - server_key.stat.mode == '0640' - ca_cert.stat.mode == '0640' fail_msg: "Certificate file permissions are not correct (expected 0640)" + - name: Verify server key is unencrypted PKCS#8 PEM (not PKCS#1, not encrypted) + ansible.builtin.command: head -1 /etc/logstash/certs/{{ inventory_hostname }}.key + register: _server_key_header + changed_when: false + + - name: Assert PKCS#8 header + ansible.builtin.assert: + that: + - _server_key_header.stdout == '-----BEGIN PRIVATE KEY-----' + fail_msg: >- + Expected unencrypted PKCS#8 PEM (-----BEGIN PRIVATE KEY-----), + got {{ _server_key_header.stdout }}. The beats/elastic_agent input + requires PKCS#8 per upstream Logstash docs. + # --- Input verification (elastic_agent with SSL) --- - name: Read input configuration diff --git a/molecule/logstash_standalone_certs/converge.yml b/molecule/logstash_standalone_certs/converge.yml index c00121d5..00892855 100644 --- a/molecule/logstash_standalone_certs/converge.yml +++ b/molecule/logstash_standalone_certs/converge.yml @@ -20,7 +20,7 @@ # External certificate configuration logstash_cert_source: external logstash_tls_certificate_file: /tmp/test-certs/server.crt - logstash_tls_key_file: /tmp/test-certs/server-pkcs8.key + logstash_tls_key_file: /tmp/test-certs/server.key logstash_tls_ca_file: /tmp/test-certs/ca.crt logstash_tls_remote_src: true logstash_extra_outputs: | diff --git a/molecule/logstash_standalone_certs/verify.yml b/molecule/logstash_standalone_certs/verify.yml index 5085a9f9..12bc4276 100644 --- a/molecule/logstash_standalone_certs/verify.yml +++ b/molecule/logstash_standalone_certs/verify.yml @@ -37,7 +37,7 @@ - name: Check server key was copied ansible.builtin.stat: - path: "/etc/logstash/certs/{{ ansible_facts.hostname }}-pkcs8.key" + path: "/etc/logstash/certs/{{ ansible_facts.hostname }}.key" register: server_key - name: Assert server key exists @@ -46,6 +46,19 @@ - server_key.stat.exists fail_msg: "Server key not found in /etc/logstash/certs/" + - name: Verify server key is unencrypted PKCS#8 PEM + ansible.builtin.command: head -1 /etc/logstash/certs/{{ ansible_facts.hostname }}.key + register: _server_key_header + changed_when: false + + - name: Assert PKCS#8 header + ansible.builtin.assert: + that: + - _server_key_header.stdout == '-----BEGIN PRIVATE KEY-----' + fail_msg: >- + Expected unencrypted PKCS#8 PEM (-----BEGIN PRIVATE KEY-----), + got {{ _server_key_header.stdout }}. + - name: Check CA certificate was copied ansible.builtin.stat: path: /etc/logstash/certs/ca.crt diff --git a/molecule/shared/generate_test_certs.yml b/molecule/shared/generate_test_certs.yml index 98cde05a..d3c08c78 100644 --- a/molecule/shared/generate_test_certs.yml +++ b/molecule/shared/generate_test_certs.yml @@ -1,6 +1,6 @@ --- # Generate a self-signed CA and server certificate for testing TLS. -# Outputs to /tmp/test-certs/: ca.crt, server.crt, server-pkcs8.key +# Outputs to /tmp/test-certs/: ca.crt, server.crt, server.key (PKCS#8 PEM) - name: Install openssl and cryptography for certificate generation ansible.builtin.package: name: @@ -18,6 +18,7 @@ community.crypto.openssl_privatekey: path: /tmp/test-certs/ca.key size: 2048 + format: pkcs8 - name: Generate CA CSR community.crypto.openssl_csr: @@ -41,10 +42,12 @@ provider: selfsigned selfsigned_not_after: "+365d" -- name: Generate server private key +- name: Generate server private key (PKCS#8 PEM, unencrypted) community.crypto.openssl_privatekey: path: /tmp/test-certs/server.key size: 2048 + format: pkcs8 + mode: "0640" - name: Generate server CSR community.crypto.openssl_csr: @@ -64,16 +67,3 @@ ownca_privatekey_path: /tmp/test-certs/ca.key provider: ownca ownca_not_after: "+365d" - -- name: Create unencrypted PKCS8 key - ansible.builtin.command: >- - openssl pkcs8 -topk8 -nocrypt - -in /tmp/test-certs/server.key - -out /tmp/test-certs/server-pkcs8.key - args: - creates: /tmp/test-certs/server-pkcs8.key - -- name: Set key permissions - ansible.builtin.file: - path: /tmp/test-certs/server-pkcs8.key - mode: "0640" diff --git a/roles/logstash/defaults/main.yml b/roles/logstash/defaults/main.yml index b6a2c253..04752b09 100644 --- a/roles/logstash/defaults/main.yml +++ b/roles/logstash/defaults/main.yml @@ -200,6 +200,17 @@ logstash_cert_source: elasticsearch_ca # logstash_tls_ca_file: "/path/to/ca.crt" # logstash_tls_remote_src: false # Set true if cert files are on remote host +# @var logstash_tls_copy_certs:description: > +# Copy externally-managed certificate files into `logstash_certs_dir` (default). +# Set to `false` to reference the `logstash_tls_*_file` paths directly from the +# pipeline config — Logstash reads them in place. Use this for certmonger, +# cert-manager, or any renewal system where the cert files update out-of-band +# and you don't want to re-run Ansible to pick up rotations. Logstash must have +# read access to the files (typically `root:logstash 0640` and a +# world-traversable parent directory). +# @end +logstash_tls_copy_certs: true + # @var logstash_certs_dir:description: Directory on the Logstash host where TLS certificates are stored logstash_certs_dir: /etc/logstash/certs diff --git a/roles/logstash/tasks/logstash-security.yml b/roles/logstash/tasks/logstash-security.yml index f7071b76..9de855cf 100644 --- a/roles/logstash/tasks/logstash-security.yml +++ b/roles/logstash/tasks/logstash-security.yml @@ -13,47 +13,111 @@ - logstash_tls_key_file is defined fail_msg: "External cert mode requires logstash_tls_certificate_file and logstash_tls_key_file" - - name: logstash-security | Create certificate directory - ansible.builtin.file: - state: directory - path: "{{ logstash_certs_dir }}" - owner: root - group: logstash - mode: "0750" + - name: logstash-security | Copy external certificate files + when: logstash_tls_copy_certs | bool + block: + - name: logstash-security | Create certificate directory + ansible.builtin.file: + state: directory + path: "{{ logstash_certs_dir }}" + owner: root + group: logstash + mode: "0750" - - name: logstash-security | Copy external certificate - ansible.builtin.copy: - src: "{{ logstash_tls_certificate_file }}" - dest: "{{ logstash_certs_dir }}/{{ inventory_hostname }}-server.crt" - owner: root - group: logstash - mode: "0640" - remote_src: "{{ logstash_tls_remote_src | default(false) }}" - notify: - - Restart Logstash + - name: logstash-security | Copy external certificate + ansible.builtin.copy: + src: "{{ logstash_tls_certificate_file }}" + dest: "{{ logstash_certs_dir }}/{{ inventory_hostname }}-server.crt" + owner: root + group: logstash + mode: "0640" + remote_src: "{{ logstash_tls_remote_src | default(false) }}" + notify: + - Restart Logstash - - name: logstash-security | Copy external key - ansible.builtin.copy: - src: "{{ logstash_tls_key_file }}" - dest: "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" - owner: root - group: logstash - mode: "0640" - remote_src: "{{ logstash_tls_remote_src | default(false) }}" - notify: - - Restart Logstash + - name: logstash-security | Copy external key + ansible.builtin.copy: + src: "{{ logstash_tls_key_file }}" + dest: "{{ logstash_certs_dir }}/{{ inventory_hostname }}.key" + owner: root + group: logstash + mode: "0640" + remote_src: "{{ logstash_tls_remote_src | default(false) }}" + notify: + - Restart Logstash - - name: logstash-security | Copy external CA certificate - ansible.builtin.copy: - src: "{{ logstash_tls_ca_file }}" - dest: "{{ logstash_certs_dir }}/ca.crt" - owner: root - group: logstash - mode: "0640" - remote_src: "{{ logstash_tls_remote_src | default(false) }}" - when: logstash_tls_ca_file is defined - notify: - - Restart Logstash + - name: logstash-security | Copy external CA certificate + ansible.builtin.copy: + src: "{{ logstash_tls_ca_file }}" + dest: "{{ logstash_certs_dir }}/ca.crt" + owner: root + group: logstash + mode: "0640" + remote_src: "{{ logstash_tls_remote_src | default(false) }}" + when: logstash_tls_ca_file is defined + notify: + - Restart Logstash + + - name: logstash-security | Check external key readability (hands-off mode) + when: not (logstash_tls_copy_certs | bool) + block: + - name: logstash-security | Stat external certificate in place + ansible.builtin.stat: + path: "{{ logstash_tls_certificate_file }}" + register: _logstash_external_cert_stat + + - name: logstash-security | Stat external key in place + ansible.builtin.stat: + path: "{{ logstash_tls_key_file }}" + register: _logstash_external_key_stat + + - name: logstash-security | Determine external CA path used by pipeline + ansible.builtin.set_fact: + _logstash_external_ca_path: "{{ logstash_tls_ca_file | default(logstash_certs_dir ~ '/ca.crt') }}" + + - name: logstash-security | Stat external CA in place + ansible.builtin.stat: + path: "{{ _logstash_external_ca_path }}" + register: _logstash_external_ca_stat + + - name: logstash-security | Validate hands-off external TLS files are readable + ansible.builtin.assert: + that: + - _logstash_external_cert_stat.stat.exists | default(false) | bool + - _logstash_external_cert_stat.stat.readable | default(false) | bool + - _logstash_external_key_stat.stat.exists | default(false) | bool + - _logstash_external_key_stat.stat.readable | default(false) | bool + - _logstash_external_ca_stat.stat.exists | default(false) | bool + - _logstash_external_ca_stat.stat.readable | default(false) | bool + fail_msg: >- + With logstash_tls_copy_certs: false, the TLS certificate, key, and + CA files referenced by the Logstash pipeline must already exist and + be readable on the managed node. + + - name: logstash-security | Warn if external key is not readable by logstash group + ansible.builtin.debug: + msg: >- + Warning: {{ logstash_tls_key_file }} does not appear readable by the + logstash user (found: exists={{ _logstash_external_key_stat.stat.exists }}, + gr_name={{ _logstash_external_key_stat.stat.gr_name | default('?') }}, + mode={{ _logstash_external_key_stat.stat.mode | default('?') }}). + Logstash will fail to start if it cannot read the key. For certmonger, + use `getcert request -o root:logstash -m 0640`. Set + logstash_tls_copy_certs: true to let the role copy with correct perms. + vars: + _ls_key_mode: "{{ _logstash_external_key_stat.stat.mode | default('0000') }}" + _ls_key_owner: "{{ _logstash_external_key_stat.stat.pw_name | default('') }}" + _ls_key_group: "{{ _logstash_external_key_stat.stat.gr_name | default('') }}" + _ls_key_owner_read: "{{ _ls_key_mode[1] in ['4', '5', '6', '7'] }}" + _ls_key_group_read: "{{ _ls_key_mode[2] in ['4', '5', '6', '7'] }}" + _ls_key_world_read: "{{ _ls_key_mode[3] in ['4', '5', '6', '7'] }}" + when: + - _logstash_external_key_stat.stat.exists | default(false) | bool + - not ( + (_ls_key_owner == 'logstash' and (_ls_key_owner_read | bool)) + or (_ls_key_group == 'logstash' and (_ls_key_group_read | bool)) + or (_ls_key_world_read | bool) + ) - name: logstash-security | Handle Elasticsearch CA certificates when: _logstash_cert_mode == 'elasticsearch_ca' @@ -216,34 +280,27 @@ tags: - certificates - - name: logstash-security | Copy encrypted key - ansible.builtin.copy: - src: "{{ logstash_certs_dir }}/{{ ansible_facts.hostname }}/{{ ansible_facts.hostname }}.key" - dest: "{{ logstash_certs_dir }}/{{ inventory_hostname }}.key" - owner: root - group: root - mode: "0640" - remote_src: true - tags: - - certificates - - - name: logstash-security | Create unencrypted PKCS8 key for Logstash - ansible.builtin.command: > - openssl pkcs8 - -in {{ logstash_certs_dir }}/{{ inventory_hostname }}.key - -topk8 - -passin pass:{{ logstash_tls_key_passphrase }} - -out {{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key - -nocrypt - args: - creates: "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" + - name: logstash-security | Decrypt key to unencrypted PKCS#8 PEM + ansible.builtin.command: + argv: + - openssl + - pkcs8 + - -in + - "{{ logstash_certs_dir }}/{{ ansible_facts.hostname }}/{{ ansible_facts.hostname }}.key" + - -topk8 + - -nocrypt + - -passin + - "pass:{{ logstash_tls_key_passphrase }}" + - -out + - "{{ logstash_certs_dir }}/{{ inventory_hostname }}.key" + changed_when: true no_log: "{{ elasticstack_no_log }}" tags: - certificates - - name: logstash-security | Set permissions on PKCS8 key + - name: logstash-security | Set permissions on Logstash key ansible.builtin.file: - path: "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" + path: "{{ logstash_certs_dir }}/{{ inventory_hostname }}.key" owner: root group: logstash mode: "0640" @@ -302,10 +359,15 @@ - certificates - name: logstash-security | Generate server key - ansible.builtin.command: - cmd: >- - openssl genrsa -out {{ logstash_certs_dir }}/{{ inventory_hostname }}.key 2048 - creates: "{{ logstash_certs_dir }}/{{ inventory_hostname }}.key" + community.crypto.openssl_privatekey: + path: "{{ logstash_certs_dir }}/{{ inventory_hostname }}.key" + size: 2048 + type: RSA + format: pkcs8 + format_mismatch: convert + mode: "0640" + owner: root + group: logstash tags: - certificates @@ -331,20 +393,6 @@ tags: - certificates - - name: logstash-security | Create PKCS8 key for Logstash - ansible.builtin.command: > - openssl pkcs8 - -topk8 - -inform PEM - -outform PEM - -in {{ logstash_certs_dir }}/{{ inventory_hostname }}.key - -out {{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key - -nocrypt - args: - creates: "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" - tags: - - certificates - - name: logstash-security | Set permissions on standalone certificates ansible.builtin.file: path: "{{ item }}" @@ -353,7 +401,6 @@ mode: "0640" loop: - "{{ logstash_certs_dir }}/{{ inventory_hostname }}-server.crt" - - "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" - "{{ logstash_certs_dir }}/standalone-ca.crt" - "{{ logstash_certs_dir }}/standalone-ca.key" tags: diff --git a/roles/logstash/tasks/main.yml b/roles/logstash/tasks/main.yml index fee636aa..b381528a 100644 --- a/roles/logstash/tasks/main.yml +++ b/roles/logstash/tasks/main.yml @@ -173,6 +173,25 @@ - renew_ca - renew_logstash_cert +- name: Compute pipeline cert paths + vars: + _hands_off: >- + {{ (logstash_cert_source | default('elasticsearch_ca')) == 'external' + and not (logstash_tls_copy_certs | default(true) | bool) }} + ansible.builtin.set_fact: + _logstash_input_cert_path: >- + {{ logstash_tls_certificate_file if _hands_off + else (logstash_certs_dir ~ '/' ~ inventory_hostname ~ '-server.crt') }} + _logstash_input_key_path: >- + {{ logstash_tls_key_file if _hands_off + else (logstash_certs_dir ~ '/' ~ inventory_hostname ~ '.key') }} + _logstash_ca_cert_path: >- + {{ logstash_tls_ca_file if (_hands_off and logstash_tls_ca_file is defined) + else (logstash_certs_dir ~ '/ca.crt') }} + tags: + - configuration + - logstash_configuration + - name: Configure Logstash ansible.builtin.template: src: logstash.yml.j2 diff --git a/roles/logstash/templates/10-input.conf.j2 b/roles/logstash/templates/10-input.conf.j2 index 1e0ce114..ba086745 100644 --- a/roles/logstash/templates/10-input.conf.j2 +++ b/roles/logstash/templates/10-input.conf.j2 @@ -7,16 +7,16 @@ input { {% if logstash_input_beats_ssl | default(false) | bool %} {% if elasticstack_release | int >= 9 %} ssl_enabled => true - ssl_certificate => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-server.crt" - ssl_key => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" + ssl_certificate => "{{ _logstash_input_cert_path }}" + ssl_key => "{{ _logstash_input_key_path }}" ssl_client_authentication => {{ logstash_input_beats_client_auth | default('required') }} - ssl_certificate_authorities => ["{{ logstash_certs_dir }}/ca.crt"] + ssl_certificate_authorities => ["{{ _logstash_ca_cert_path }}"] {% else %} ssl => true - ssl_certificate => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-server.crt" - ssl_key => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" + ssl_certificate => "{{ _logstash_input_cert_path }}" + ssl_key => "{{ _logstash_input_key_path }}" ssl_verify_mode => force_peer - ssl_certificate_authorities => ["{{ logstash_certs_dir }}/ca.crt"] + ssl_certificate_authorities => ["{{ _logstash_ca_cert_path }}"] {% endif %} {% endif %} {% if logstash_input_beats_timeout is defined %} @@ -30,16 +30,16 @@ input { {% if logstash_input_elastic_agent_ssl | default(true) | bool %} {% if elasticstack_release | int >= 9 %} ssl_enabled => true - ssl_certificate => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-server.crt" - ssl_key => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" + ssl_certificate => "{{ _logstash_input_cert_path }}" + ssl_key => "{{ _logstash_input_key_path }}" ssl_client_authentication => required - ssl_certificate_authorities => ["{{ logstash_certs_dir }}/ca.crt"] + ssl_certificate_authorities => ["{{ _logstash_ca_cert_path }}"] {% else %} ssl => true - ssl_certificate => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-server.crt" - ssl_key => "{{ logstash_certs_dir }}/{{ inventory_hostname }}-pkcs8.key" + ssl_certificate => "{{ _logstash_input_cert_path }}" + ssl_key => "{{ _logstash_input_key_path }}" ssl_verify_mode => force_peer - ssl_certificate_authorities => ["{{ logstash_certs_dir }}/ca.crt"] + ssl_certificate_authorities => ["{{ _logstash_ca_cert_path }}"] {% endif %} {% endif %} }