Added audit log categories for cluster and index settings changes#6113
Added audit log categories for cluster and index settings changes#6113Rishav9852Kumar wants to merge 9 commits intoopensearch-project:mainfrom
Conversation
Signed-off-by: rishavaz <rishavaz@amazon.com>
Signed-off-by: Rishav9852Kumar <rishavkumaraug20005212@gmail.com>
Signed-off-by: Rishav9852Kumar <rishavkumaraug20005212@gmail.com>
Signed-off-by: Rishav9852Kumar <rishavkumaraug20005212@gmail.com>
Signed-off-by: Rishav9852Kumar <rishavkumaraug20005212@gmail.com>
PR Reviewer Guide 🔍(Review updated until commit b025aec)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to b025aec
Previous suggestionsSuggestions up to commit 63168c2
|
PR Code Analyzer ❗AI-powered 'Code-Diff-Analyzer' found issues on commit b025aec.
The table above displays the top 10 most important findings. Pull Requests Author(s): Please update your Pull Request according to the report above. Repository Maintainer(s): You can Thanks. |
|
Persistent review updated to latest commit b025aec |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6113 +/- ##
==========================================
+ Coverage 74.80% 74.88% +0.08%
==========================================
Files 447 447
Lines 28463 28575 +112
Branches 4328 4342 +14
==========================================
+ Hits 21291 21399 +108
- Misses 5176 5182 +6
+ Partials 1996 1994 -2
🚀 New features to boost your workflow:
|
Description
Added new audit log categories
CLUSTER_SETTINGS_CHANGEDandINDEX_SETTINGS_CHANGEDto track changes to cluster and index settings.
Both categories are disabled by default for transport (matching the existing behavior of AUTHENTICATED and GRANTED_PRIVILEGES). Users can enable them by removing them from disabled_transport_categories in their audit config.
Each setting change is logged with an operation field:
Each setting change is logged with an scope field:
PUT /_cluster/settings with persistent)
PUT /_cluster/settings with transient)
Category: New feature
Why these changes are required?
Currently, the audit log feature has no category to track changes to cluster or index settings. Operators have no way to audit what settings were changed, when, by whom, or what the previous values were. The existing INDEX_EVENT
category only logs the request body without old values or sensitive setting redaction.
What is the old behavior before changes and new behavior after changes?
Example: CLUSTER_SETTINGS_CHANGED
{ "audit_category": "CLUSTER_SETTINGS_CHANGED", "audit_request_effective_user": "admin", "audit_transport_action": "cluster:admin/settings/update", "audit_settings_changes": [ { "setting": "cluster.max_shards_per_node", "old_value": null, "new_value": "2000", "operation": "set", "scope": "persistent" } ] }Example: INDEX_SETTINGS_CHANGED
{ "audit_category": "INDEX_SETTINGS_CHANGED", "audit_request_effective_user": "admin", "audit_transport_action": "indices:admin/settings/update", "audit_trace_indices": ["my-index-*"], "audit_trace_resolved_indices": ["my-index-001", "my-index-002"], "audit_settings_changes": [ { "setting": "index.number_of_replicas", "old_value": "1", "new_value": "2", "operation": "set", "scope": "index" } ] }Issues Resolved
Resolves #5320
Testing
Unit Tests
Added SettingsChangeAuditTest.java with 27 tests covering all new functionality. Updated existing tests in DisabledCategoriesTest, AuditCategoryTest, AuditConfigSerializeTest, and AuditConfigFilterTest for the new categories and default-disabled behavior.
Integ Test
Manual Testing
Tested on OpenSearch 3.7.0-SNAPSHOT with security plugin 3.7.0.0-SNAPSHOT, single node cluster.
Test 1: Set persistent cluster setting
Request:
Audit log:
{ "audit_category": "CLUSTER_SETTINGS_CHANGED", "audit_request_effective_user": "admin", "audit_transport_action": "cluster:admin/settings/update", "audit_transport_request_type": "ClusterUpdateSettingsRequest", "audit_request_origin": "REST", "audit_request_layer": "TRANSPORT", "@timestamp": "2026-04-27T19:17:37.811+00:00", "audit_format_version": 4, "audit_settings_changes": [ { "setting": "cluster.max_shards_per_node", "old_value": null, "new_value": "2000", "operation": "set", "scope": "persistent" } ] }✅ Persistent setting captured with correct old/new values, operation, and scope.
Test 2: Set transient cluster setting
Request:
Audit log:
{ "audit_category": "CLUSTER_SETTINGS_CHANGED", "audit_transport_action": "cluster:admin/settings/update", "audit_settings_changes": [ { "setting": "cluster.routing.allocation.enable", "old_value": null, "new_value": "primaries", "operation": "set", "scope": "transient" } ] }✅ Transient scope correctly identified.
Test 3: Reset setting to default (null → removed)
Request:
Audit log:
{ "audit_category": "CLUSTER_SETTINGS_CHANGED", "audit_settings_changes": [ { "setting": "cluster.max_shards_per_node", "old_value": "2000", "new_value": null, "operation": "removed", "scope": "persistent" } ] }✅ Old value captured, operation correctly set to removed.
Test 4: Index setting change
Request:
Audit log:
{ "audit_category": "INDEX_SETTINGS_CHANGED", "audit_transport_action": "indices:admin/settings/update", "audit_trace_indices": ["test-audit-index"], "audit_trace_resolved_indices": ["test-audit-index"], "audit_settings_changes": [ { "setting": "index.number_of_replicas", "old_value": "1", "new_value": "2", "operation": "set", "scope": "index" } ] }✅ Index name captured in audit_trace_indices and audit_trace_resolved_indices.
Test 5: Multiple settings in one call (persistent + transient)
Request:
Audit log:
{ "audit_category": "CLUSTER_SETTINGS_CHANGED", "audit_settings_changes": [ { "setting": "cluster.max_shards_per_node", "old_value": null, "new_value": "3000", "operation": "set", "scope": "persistent" }, { "setting": "cluster.routing.rebalance.enable", "old_value": null, "new_value": "none", "operation": "set", "scope": "persistent" }, { "setting": "cluster.routing.allocation.enable", "old_value": "primaries", "new_value": "all", "operation": "set", "scope": "transient" } ] }✅ Single audit entry with all 3 changes across both scopes.
Test 6: Sensitive setting redaction
Request:
Audit log (audit_settings_changes):
[ { "setting": "cluster.max_shards_per_node", "old_value": "3000", "new_value": "5000", "operation": "set", "scope": "persistent" }, { "setting": "plugins.security.auth_token_provider", "old_value": null, "new_value": "***REDACTED***", "operation": "set", "scope": "persistent" }, { "setting": "plugins.security.ssl.http.keystore_password", "old_value": null, "new_value": "***REDACTED***", "operation": "set", "scope": "persistent" }, { "setting": "plugins.security.ssl.transport.keystore_password", "old_value": null, "new_value": "***REDACTED***", "operation": "set", "scope": "persistent" } ]✅ Settings with password, secret, or token in the name are redacted. Non-sensitive settings show actual values.
Test 7: Wildcard index pattern with resolved indices
Request:
Audit log:
{ "audit_category": "INDEX_SETTINGS_CHANGED", "audit_trace_indices": ["test-wild-*"], "audit_trace_resolved_indices": ["test-wild-001", "test-wild-002"], "audit_settings_changes": [ { "setting": "index.number_of_replicas", "old_value": "1", "new_value": "0", "operation": "set", "scope": "index" } ] }✅ Wildcard pattern preserved in audit_trace_indices, concrete indices resolved in audit_trace_resolved_indices.
Test 8: Disable category via audit.yml
Added to disabled_transport_categories in audit.yml:
Applied via securityadmin.sh -t audit, then changed cluster settings — no new audit entries created.
✅ Categories can be independently disabled via existing config mechanism.
Summary
Check List
N/A
-project/opensearch-api-specification/blob/main/DEVELOPER_GUIDE.md) — N/A
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.