TLS certificate removal blocked by NXDOMAIN

[DavidePrincipi]

When a DNS record that corresponds to a name in an obtained certificate is removed, certificate renewal continuously and silently fails. The name cannot be removed from TLS certificates page.

Steps to reproduce

[this procedure is not verified]

• Obtain three certificates with the following unique set of names:
  1. one.example.com
  2. one.example.com, two.example.com
  3. one.example.com, two.example.com, three.example.com
     Be aware of LE rate limits, specifically https://letsencrypt.org/docs/rate-limits/#new-certificates-per-exact-set-of-identifiers
• Create three distinct HTTP routes, based on host name with LE cert: one.example.com, two.example.com, three.example.com
• Remove two.example.com record from DNS
• After DNS has been propagated and two.example.com resolution returns NXDOMAIN, try to remove it from the TLS certificates page

Expected behavior

1. The name is unlisted from TLS certificates page
2. The certificate is not renewed any more

Actual behavior

Inspect logs with these commands:

journalctl --grep acmeCA
logcli query -q --no-labels --limit=1000 --since=7d '{module_id="traefik1"} |= "acmeCA" | json | line_format "{{.MESSAGE}}"'

When certificate is soon to expire, a message like this appears in the logs:

ERR Error renewing certificate from LE: {two.example.com []} error="error: one or more domains had a problem:
[two.example.com] invalid authorization: acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for two.example.com - check that a DNS record exists for this domain; DNS problem: NXDOMAIN looking up AAAA for two.example.com - check that a DNS record exists for this domain
" acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=acmeServer.acme

Or (with SANs):

ERR Error renewing certificate from LE: {one.example.com [two.example.com three.example.com]} error="error: one or more domains had a problem:\n[two.example.com] invalid authorization: acme: error: 400 :: urn:ietf:params:acme:error:dns :: DNS problem: NXDOMAIN looking up A for two.example.com - check that a DNS record exists for this domain; DNS problem: NXDOMAIN looking up AAAA for two.example.com - check that a DNS record exists for this domain\n" acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=acmeServer.acme

Name cannot be removed from TLS certificates page because the validation procedure fails with a timeout after 30 seconds.

Components

• Traefik 3.2.0
• Core 3.9.0

See also

• https://community.nethserver.org/t/ssl-certificate-error-err-cert-date-invalid/25970
• https://community.nethserver.org/t/all-certificates-are-expired-or-expiring-nothing-renewing/25972
• https://community.nethserver.org/t/latest-issue-with-letsencrypt/25927 (here the certificate was never obtained)

Discussion https://mattermost.nethesis.it/nethesis/pl/tq5zx5hj6pbrimn3suoxbji5xc

[Neustradamus]

To follow this ticket.

[DavidePrincipi]

In Testing

See parent issue #7544 (comment)