[SDBM-2104] Updates Postgres config handling around dbname when autodiscovery used #21884

eric-weaver · 2025-11-14T21:10:07Z

What does this PR do?

Currently dbname is not required, but will default to "postgres" if not provided in the config. The goal of this PR is to fix a config bug where we'll still try to connect to the "postgres" db if using database_autodiscovery with "postgres" in the exclusion list.

Currently the config below will result in still attempting to connect and query from the "postgres" database which might not exist or be in use on some providers

database_autodiscovery:
  exclude: ["postgres"]

After the change this config will now result in a config validation error telling the user to either set dbname or database_autodiscovery.global_view_db

Currently the config below will result in still attempting to connect and query from the "postgres" database which might not exist or be in use on some providers

database_autodiscovery:
  global_view_db: mydb
  exclude: ["postgres"]

After the change dbname will become mydb instead of "postgres

Motivation

This was identified as a bug via a customer support ticket

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2025-11-14T21:14:46Z

postgres/datadog_checks/postgres/config.py

+                if re.search(exclude_pattern, config.dbname, re.IGNORECASE):
+                    # Check if user explicitly set dbname
+                    if 'dbname' in instance:
+                        validation_result.add_error(
+                            f'The configured dbname "{config.dbname}" matches the autodiscovery '


Do not reject configs that intentionally exclude the global view DB

The new validation treats any autodiscovery configuration where dbname matches an exclude pattern as an error. However the exclude list is only used to filter which databases are collected from autodiscovery; it does not dictate which database the agent connects to for the global view. Many existing setups connect to postgres for discovery but exclude it so no metrics are gathered from that database. With this change, those previously valid configurations now fail validation even though connection succeeds and metrics were intentionally suppressed. Consider limiting this error to the case where dbname was auto‑defaulted or when the database truly cannot be connected to, rather than rejecting explicit dbname values that happen to be excluded.

Useful? React with 👍 / 👎.

codecov · 2025-11-14T21:24:43Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.31%. Comparing base (52b2493) to head (17a44c4).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

eric-weaver · 2025-11-14T21:45:03Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2025-11-14T21:49:07Z

postgres/datadog_checks/postgres/config.py

+    if config.database_autodiscovery.enabled and config.dbname and 'dbname' not in instance:
+        import re
+
+        for exclude_pattern in config.database_autodiscovery.exclude:
+            try:
+                if re.search(exclude_pattern, config.dbname, re.IGNORECASE):


Handle null autodiscovery exclude list before iterating

The new validation assumes config.database_autodiscovery.exclude is iterable, but the model defines this field as optional. If a user explicitly sets database_autodiscovery.exclude: null to remove default exclusions, the merge logic leaves exclude as None and this loop will raise TypeError: 'NoneType' object is not iterable, preventing the check from starting. Guard against None (e.g., iterate over config.database_autodiscovery.exclude or ()) before scanning for regex matches.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2025-11-14T21:50:10Z

postgres/datadog_checks/postgres/config.py

+    # Validate dbname is not excluded when using autodiscovery
+    # Only validate when dbname was NOT explicitly set by the user (auto-defaulted)
+    # If user explicitly set dbname, they may intentionally want to connect to an excluded database
+    # for global operations while excluding it from per-database metric collection
+    if config.database_autodiscovery.enabled and config.dbname and 'dbname' not in instance:
+        import re
+
+        for exclude_pattern in config.database_autodiscovery.exclude:
+            try:
+                if re.search(exclude_pattern, config.dbname, re.IGNORECASE):
+                    # Auto-defaulted dbname conflicts - suggest setting global_view_db


Align autodiscovery exclude validation with runtime matching

The new validation rejects configurations when the auto‑defaulted dbname case‑insensitively matches an exclude pattern (re.search(..., re.IGNORECASE)). However the autodiscovery code that actually filters databases (Filter in datadog_checks_base/utils/discovery/filter.py) compiles exclude patterns without any flags, so matching is case‑sensitive. This discrepancy means a config such as global_view_db: "POSTGRES" with exclude: ["postgres"] will now fail validation even though the runtime logic would treat those values as different databases. Either remove the IGNORECASE flag here or make the runtime filter case‑insensitive so both paths behave consistently.

Useful? React with 👍 / 👎.

eric-weaver · 2025-11-17T15:47:00Z

@codex review

chatgpt-codex-connector · 2025-11-17T15:51:45Z

Codex Review: Didn't find any major issues. Hooray!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…iscovery used (DataDog#21884) * Updates config handling around dbname when autodiscovery used * Add changelog * Update logic to properly handle expected cases * Fix regex and additional cases b9e94e3

…iscovery used (#21884) * Updates config handling around dbname when autodiscovery used * Add changelog * Update logic to properly handle expected cases * Fix regex and additional cases

* [MOPU-112] Remove "silenced" options with values from monitor definitions. * Fix Python upgrade tests to be independent of repository Python version (#21888) * Microsoft Copilot: Introduction of assets for first version of integration (reports only) (#21436) * [DBM][Postgres] Debug log if extension is not in loader map (#21891) * Added pg_stat_statement to PG_EXTENSION_LOADER_QUERY * Updated query * Updated code based on discussion with Engineering * [DOCS-12665] Fix typo in metric name (#21890) * [SDBM-2104] Updates Postgres config handling around dbname when autodiscovery used (#21884) * Updates config handling around dbname when autodiscovery used * Add changelog * Update logic to properly handle expected cases * Fix regex and additional cases * Create shared schemas collector for DBM integrations (#21720) * Create shared schemas collector for DBM integrations * WIP * WIP * Changelog * Warning * Remove unused * Lint * AI Fixes * Fix * Feedback * Feedback * Lint * Refactor health * Fix * WIP * Revert "WIP" This reverts commit aa5b86b. * Comment * [DBMON-5907] Update dbms process signatures (#21892) * Update process signatures * Revert accidental single quote addition * Release `datadog_checks_base` (#21903) * [Release] Bumped datadog_checks_base version to 37.24.0 * [Release] Update metadata * Fix Unicode handling in YAML config parsing Update the pull request reference for a fix in YAML configuration parsing. * box ga (#21907) * Bump base version for Postgres, MySQL, SQLServer (#21906) * Bump base version for Postgres, MySQL, SQLServer * Changelog * Lint * [Lustre] Fix device discovery for older versions of Lustre (<2.15.5) (#21901) * Add test for non-yaml output of lctl dl * Implement fallback to non-yaml version of lctl dl * Add changelog * Format * Assert for exact match of devices in test * Unify device list fixtures for readability * Improve code based on review * Remove redundant devices definition * Format * Minor syntax improvement * Add Process Signatures to Windows Integrations (#21909) * adding process signatures * adjusting signatures * adding another signature * adjusting signatures * Add owner field to agent-integrations manifest files (#21912) Adds 'owner': 'agent-integrations' field to 3 integration manifest.json files: - barracuda_secure_edge - cloudgen_firewall - zscaler_private_access This provides clear ownership tracking for these security and networking integrations as part of the initiative to add owner fields to all integration manifest.json files. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Claude <noreply@anthropic.com> * IDP-800: Add owner field to saas-integrations manifest files (#21913) * Add owner field to saas-integrations manifest files Adds 'owner': 'saas-integrations' field to 7 integration manifest.json files: - cofense_triage - crowdstrike_fdr - have_i_been_pwned - kandji - obsidian_security - push_security - zerofox_cloud_platform This provides clear ownership tracking for these SaaS security integrations as part of the initiative to add owner fields to all integration manifest.json files. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Add owner field to obsidian_security and zerofox_cloud_platform These two integrations were missed in the previous commit due to different indentation formatting in their manifest.json files. --------- Co-authored-by: Claude <noreply@anthropic.com> * [AI-6169] DDS: Forescout Integration v1.0.0 (#21617) * Add forescout integration * Update test samples * Update changelog file * Update pipeline and samples * Add test sample results * Add images and manifest file * Logo update in dashboards * Address review comments * update dashboard panels name * Address review comments * Update logo images in dashboards * Resolve validate-dashboard CI check --------- Co-authored-by: akaila-crest <abhi.kaila@crestdata.ai> Co-authored-by: Thibault Krebs <150821137+thibaultkrebs@users.noreply.github.com> --------- Co-authored-by: Kyle Neale <kyle.neale@datadoghq.com> Co-authored-by: Brian Williams <bskinny@users.noreply.github.com> Co-authored-by: Aldrick Castro <aldrick.castro@datadoghq.com> Co-authored-by: Eva Parish <eva.parish@datadoghq.com> Co-authored-by: Eric Weaver <eweaver755@gmail.com> Co-authored-by: Seth Samuel <seth.samuel@datadoghq.com> Co-authored-by: apiazza-dd <120594144+apiazza-dd@users.noreply.github.com> Co-authored-by: dkirov-dd <166512750+dkirov-dd@users.noreply.github.com> Co-authored-by: Mohammad Rafi <36865458+mrafi97@users.noreply.github.com> Co-authored-by: davidfeng-datadog <david.feng@datadoghq.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tirthraj Chaudhari <105789910+tirthrajchaudhari-crest@users.noreply.github.com> Co-authored-by: akaila-crest <abhi.kaila@crestdata.ai> Co-authored-by: Thibault Krebs <150821137+thibaultkrebs@users.noreply.github.com>

Updates config handling around dbname when autodiscovery used

6dce30d

eric-weaver requested review from a team as code owners November 14, 2025 21:10

temporal-github-worker-1 bot added agent/review-requested ecosystems/review-requested product/review-requested labels Nov 14, 2025

datadog-agent-integrations-bot bot added integration/postgres team/agent-integrations team/database-monitoring-agent labels Nov 14, 2025

chatgpt-codex-connector bot reviewed Nov 14, 2025

View reviewed changes

Add changelog

3efd320

Update logic to properly handle expected cases

dfcdbcf

chatgpt-codex-connector bot reviewed Nov 14, 2025

View reviewed changes

Fix regex and additional cases

17a44c4

sethsamuel approved these changes Nov 17, 2025

View reviewed changes

eric-weaver added this pull request to the merge queue Nov 17, 2025

Merged via the queue into master with commit b9e94e3 Nov 17, 2025
100 of 102 checks passed

eric-weaver deleted the eric.weaver/SDBM-2104 branch November 17, 2025 20:09

eric-weaver mentioned this pull request Nov 18, 2025

Statement metrics collection respect global_view_db #21838

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SDBM-2104] Updates Postgres config handling around dbname when autodiscovery used #21884

[SDBM-2104] Updates Postgres config handling around dbname when autodiscovery used #21884

eric-weaver commented Nov 14, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Nov 14, 2025

Uh oh!

codecov bot commented Nov 14, 2025 •

edited

Loading

Uh oh!

eric-weaver commented Nov 14, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Nov 14, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Nov 14, 2025

Uh oh!

eric-weaver commented Nov 17, 2025

Uh oh!

chatgpt-codex-connector bot commented Nov 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[SDBM-2104] Updates Postgres config handling around dbname when autodiscovery used #21884

[SDBM-2104] Updates Postgres config handling around dbname when autodiscovery used #21884

Conversation

eric-weaver commented Nov 14, 2025

What does this PR do?

Motivation

Review checklist (to be filled by reviewers)

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

eric-weaver commented Nov 14, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

eric-weaver commented Nov 17, 2025

Uh oh!

chatgpt-codex-connector bot commented Nov 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Nov 14, 2025 •

edited

Loading