From 8aafc31db107ef42902eb4c4849304996e1adc2c Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Fri, 15 Aug 2025 11:27:08 -0400 Subject: [PATCH 01/13] Update for 9.1 Stack Release --- rfcs/text/0049-entity-fields.md | 52 ++++++++++++++++++++++++--------- 1 file changed, 39 insertions(+), 13 deletions(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 9ce7ef4a5d..9c0bc22426 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -2,6 +2,7 @@ - Stage: **1 (draft)** - Date: 2025-04-23 +- Updated: 2025-08-15 An entity represents a discrete, identifiable component within an IT environment that can be described by a set of attributes and maintains its identity over time. Entities can be physical (like hosts or devices), logical (like containers or processes), or abstract (like applications or services). @@ -20,31 +21,55 @@ This approach would allow ECS to accommodate new types of entities without requi | Field | Type | Description | |-------|------|-------------| +| entity.definition_id | keyword | Used Elastic solutions (e.g., Security, Observability) to denote the ID of the entity definition which is used to extract entity details from ingested logs, events, intelligence, and other data types. Use of this value is reserved, and ECS producers, including data ingestion pipelines, must not populate this field| +| entity.definition_id | keyword | Used by Elastic solutions (e.g., Security, Observability) to denote the version of the entity definition which is used to extract entity details from ingested logs, events, intelligence, and other data types. Use of this value is reserved, and ECS producers, including data ingestion pipelines, must not populate this field| +| entity.schema_version | keyword | Denotes the version of the entity schema,as published in Elastic Security documentation, to which this entity information conforms. Usually conforms to the Elastic Stack version. +| entity.last_seen_timestamp | date | Indicates the date/time when this entity was last "seen," usually based upon the last event/log that is initiated by this entity. Note: this field may contain the same timestamp as is included in custom field `entity.lifecycle.Last_activity`. | entity.id | keyword | A unique identifier for the entity. When multiple identifiers exist, this should be the most stable and commonly used identifier that: 1) persists across the entity's lifecycle, 2) ensures uniqueness within its scope, 3) is commonly used for queries and correlation, and 4) is readily available in most observations (logs/events). For entities with dedicated field sets (e.g., host, user), this value should match the corresponding *.id field. Alternative identifiers (e.g., ARNs values in AWS, URLs) can be preserved in entity.raw. | | entity.source | keyword | The module or integration that provided this entity data (similar to event.module). | -| entity.type | keyword | A standardized high-level classification of the entity. This provides a normalized way to group similar entities across different providers or systems. Example values: `bucket`, `database`, `container`, `function`, `queue`, `host`, `user`, etc. | -| entity.sub_type | keyword | The specific type designation for the entity as defined by its provider or system. This field provides more granular classification than entity.type. Examples: `aws_s3_bucket`, `gcp_cloud_storage_bucket`, `azure_blob_container` would all map to type `bucket`. | +| entity.type | keyword | A standardized high-level classification of the entity. This provides a normalized way to group similar entities across different providers or systems. Example values: `bucket`, `database`, `container`, `function`, `queue`, `host`, `user`, `application`, `session`, etc. | +| entity.sub_type | keyword | The specific type designation for the entity as defined by its provider or system. This field provides more granular classification than entity.type. Examples: `aws_s3_bucket`, `gcp_cloud_storage_bucket`, `azure_blob_container` would all map to type `bucket`. `hardware` , `virtual` , `container` , `node` , `cloud_instance` would all map to entity type `host'.| | entity.name | keyword, text | The name of the entity. The keyword field enables exact matches for filtering and aggregations, while the text field enables full-text search. For entities with dedicated field sets (e.g., `host`), this field should mirrors the corresponding *.name value. | +| entity.display_name | keyword, text | An optional field used when a pretty name is desired for entity-centric operations. This field should not be used for correlation with `*.name` fields for entities with dedicated field sets (e.g., `host`). | | entity.reference | keyword | A URI, URL, or other direct reference to access or locate the entity in its source system. This could be an API endpoint, web console URL, or other addressable location. Format may vary by entity type and source system. | -| entity.attributes.* | object | Normalized entity attributes using capitalized field names (e.g., `entity.attributes.StorageClass`, `entity.attributes.MfaEnabled`). Use this field set when you need specific data types, advanced search capabilities, or normalized values across different providers/sources. The capitalization pattern indicates these are entity-specific fields that won't be enumerated in the ECS schema. | -| entity.raw.* | flattened | Original, unmodified fields from the source system stored in a flattened format that maintains basic searchability. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities. Supports existence queries, exact value matches, and simple aggregations. | +| entity.attributes.* | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Examples include: `entity.attributes.Storage_class`, `entity.attributes.Mfa_enabled` , `entity.attributes.Privileged` , `entity.attributes.Granted_permissions` , `entity.attributes.Known_redirect` , `entity.attributes.Asset` , `entity.attrbitues.Os_current` , `entity.attibutes.Os_patch_current` , `entity.attributes.Oauth_consent_restriction`). Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern for Examples indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | +| entity.lifecycle.* | object | A set of temporal characteristics of the entity. Usually date field data type. Examples include: `entity.lifecycle.First_seen`, `entity.lifecycle.Last_activity` , `entity.lifecycle.Issued_at` , `entity.lifecycle.Last_password_change` ,etc. ). Use this field set when you need to track temporal characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | +| entity.behavior.* | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Behaviors are usually captured in event logs under fields such as `event.action` and other fields, but this field set captures "attributified" behavior indicators, using semantics like "this behavior was seen one or more times during this time period." Sytems using this field set may need to force a "reset" of these behavioral indicators at the end of their current period. Usually boolean field data type. Examples include: `entity.behavior.Used_usb_device`, `entity.behavior.Brute_force_victim` , `entity.behavior.New_country_login` ,etc. ). Use this field set when you need to capture and track ephemeral characterstics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | +| entity.raw.* | object | Original, unmodified fields from the source system stored in a flattened format that maintains basic searchability. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities. Supports existence queries, exact value matches, and simple aggregations. | +| entity.metrics.* | object | Field set for any fields containing numeric entity metrics. These use dymanic field data type mapping. Any fields under entity.metrics.* will automatically get the appropriate numeric mapping without needing explicit definition. -When representing entities that correspond to existing ECS field sets (e.g., hosts, users, services, containers), the relevant ECS field set should be used to capture detailed metadata about that entity. For example: +When representing entities that have existing ECS entiy field sets (e.g., hosts, users, services, containers), the relevant ECS field set should continue to be used to capture a copy of the detailed metadata about that entity. For example: -- Host entities should utilize the `host.*` field set to capture detailed host information -- User entities should leverage the `user.*` field set for user-specific attributes -- Cloud resource entities should use the `cloud.*` field set for cloud provider details -- Service entities should employ the `service.*` field set for service-specific metadata +- Host entities should also utilize the `host.*` field set to capture detailed host information +- User entities should also leverage the `user.*` field set for user-specific attributes +- Cloud resource entities should also use the `cloud.*` field set for cloud provider details +- Service entities should also employ the `service.*` field set for service-specific metadata -This approach ensures backward compatibility, maintains existing ECS patterns, and preserves the rich metadata capabilities of established field sets while allowing the entity fields to provide a consistent way to identify and categorize all entity types. The entity fields serve as a complementary layer that enables unified entity representation, particularly for entity types that don't have dedicated field sets. +This approach ensures backward compatibility, maintains existing ECS patterns, and preserves the rich metadata capabilities of established field sets while allowing the entity fields to provide a consistent way to identify and categorize *all* entity types. The entity fields serve as a complementary layer that enables unified entity representation, particularly for entity types that don't have dedicated field sets. -## Usage +## Field Re-use -The entity field set enables us to normalize entity data in such a way where we can easily query key attributes in a standardized way regardless of the type and source of the entity. This will be how we'll normalize all entity data in the upcoming inventory experience that we're planning for the security solution. +### ECS Producers + +For ECS producers, such as Beats, Elastic Agent integrations, ingest pipelines, and other methods for shipping data to Elastic, the `entity.*` fields are expected to be nested as follows: +- If the entity type is one of host, user, service, cloud, orchestrator), then the entity fields should be nested under the respecitve root field set, for example `host.entity.*` , `user.entity.*`, etc. +- If the entity type is not one of the above, then that `entity.*` fields should be nested under a new root-level object, called `generic`, as `generic.entity.*` + +Special note for use with entity fields that use `target` fields. +If the entity type includes a provision for `target.*` fields, such as `user.target.*`, then the `entity.*` fields should be nested under the `*.target.*` field set, for example, `user.target.entity.*`. + +### ECS Consumers or Data Stores + +For ECS consumers, such as the Elastic Security Solution entity store indices, the `entity.*` fields should be used directly at the root of the events. + +## User Benefits This approach will enable security analysts to view all the entities discovered inside of their environment, whether from logs or other data sources. The entity field set will then begin powering all parts of our security solution experience like alerts, where we can now represent more entities beyond just users and hosts. -Essentially, this field set gives us a standard way to represent any entity's metadata, regardless of its type or source, and provides customers with the same ability to standardize that information across their environments. +Essentially, this field set gives us a standard way to represent any entity's metadata, regardless of its type or source, and provides users with the same ability to standardize that information across their environments. + +The entity field set enables users to normalize entity data in such a way where key attributes can be queried in a standardized way regardless of the type and source of the entity metadata. Elastic solutions normalize all entity data using this entity.* field set. + ## Source data @@ -77,3 +102,4 @@ TO DO - Stage 0: - Stage 1: +- Stage 1 Update: From 4f8b4f2813f2b6cb1af59a845c6ffa5488ad3047 Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Thu, 28 Aug 2025 14:13:37 -0400 Subject: [PATCH 02/13] Update 0049-entity-fields.md Add example for entity.attributes.Managed --- rfcs/text/0049-entity-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 9c0bc22426..e41f6b3dbb 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -32,7 +32,7 @@ This approach would allow ECS to accommodate new types of entities without requi | entity.name | keyword, text | The name of the entity. The keyword field enables exact matches for filtering and aggregations, while the text field enables full-text search. For entities with dedicated field sets (e.g., `host`), this field should mirrors the corresponding *.name value. | | entity.display_name | keyword, text | An optional field used when a pretty name is desired for entity-centric operations. This field should not be used for correlation with `*.name` fields for entities with dedicated field sets (e.g., `host`). | | entity.reference | keyword | A URI, URL, or other direct reference to access or locate the entity in its source system. This could be an API endpoint, web console URL, or other addressable location. Format may vary by entity type and source system. | -| entity.attributes.* | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Examples include: `entity.attributes.Storage_class`, `entity.attributes.Mfa_enabled` , `entity.attributes.Privileged` , `entity.attributes.Granted_permissions` , `entity.attributes.Known_redirect` , `entity.attributes.Asset` , `entity.attrbitues.Os_current` , `entity.attibutes.Os_patch_current` , `entity.attributes.Oauth_consent_restriction`). Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern for Examples indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | +| entity.attributes.* | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Examples include: `entity.attributes.Storage_class`, `entity.attributes.Mfa_enabled` , `entity.attributes.Privileged` , `entity.attributes.Granted_permissions` , `entity.attributes.Known_redirect` , `entity.attributes.Asset` , `entity.attributes.Managed` ,`entity.attrbitues.Os_current` , `entity.attibutes.Os_patch_current` , `entity.attributes.Oauth_consent_restriction`). Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern for Examples indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.lifecycle.* | object | A set of temporal characteristics of the entity. Usually date field data type. Examples include: `entity.lifecycle.First_seen`, `entity.lifecycle.Last_activity` , `entity.lifecycle.Issued_at` , `entity.lifecycle.Last_password_change` ,etc. ). Use this field set when you need to track temporal characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.behavior.* | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Behaviors are usually captured in event logs under fields such as `event.action` and other fields, but this field set captures "attributified" behavior indicators, using semantics like "this behavior was seen one or more times during this time period." Sytems using this field set may need to force a "reset" of these behavioral indicators at the end of their current period. Usually boolean field data type. Examples include: `entity.behavior.Used_usb_device`, `entity.behavior.Brute_force_victim` , `entity.behavior.New_country_login` ,etc. ). Use this field set when you need to capture and track ephemeral characterstics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.raw.* | object | Original, unmodified fields from the source system stored in a flattened format that maintains basic searchability. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities. Supports existence queries, exact value matches, and simple aggregations. | From f0571efb00f0f9acf82279840c1bd9afee064408 Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Thu, 4 Sep 2025 07:51:05 -0400 Subject: [PATCH 03/13] Update rfcs/text/0049-entity-fields.md typo Co-authored-by: Uri Weisman <68195305+uri-weisman@users.noreply.github.com> --- rfcs/text/0049-entity-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index e41f6b3dbb..303789662d 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -38,7 +38,7 @@ This approach would allow ECS to accommodate new types of entities without requi | entity.raw.* | object | Original, unmodified fields from the source system stored in a flattened format that maintains basic searchability. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities. Supports existence queries, exact value matches, and simple aggregations. | | entity.metrics.* | object | Field set for any fields containing numeric entity metrics. These use dymanic field data type mapping. Any fields under entity.metrics.* will automatically get the appropriate numeric mapping without needing explicit definition. -When representing entities that have existing ECS entiy field sets (e.g., hosts, users, services, containers), the relevant ECS field set should continue to be used to capture a copy of the detailed metadata about that entity. For example: +When representing entities that have existing ECS entity field sets (e.g., hosts, users, services, containers), the relevant ECS field set should continue to be used to capture a copy of the detailed metadata about that entity. For example: - Host entities should also utilize the `host.*` field set to capture detailed host information - User entities should also leverage the `user.*` field set for user-specific attributes From 531d844f0a78032950f8d26aab8e75408f021c69 Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Thu, 4 Sep 2025 07:52:24 -0400 Subject: [PATCH 04/13] Update rfcs/text/0049-entity-fields.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit fix typos Co-authored-by: Rômulo Farias --- rfcs/text/0049-entity-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 303789662d..5cb8d7f33c 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -32,7 +32,7 @@ This approach would allow ECS to accommodate new types of entities without requi | entity.name | keyword, text | The name of the entity. The keyword field enables exact matches for filtering and aggregations, while the text field enables full-text search. For entities with dedicated field sets (e.g., `host`), this field should mirrors the corresponding *.name value. | | entity.display_name | keyword, text | An optional field used when a pretty name is desired for entity-centric operations. This field should not be used for correlation with `*.name` fields for entities with dedicated field sets (e.g., `host`). | | entity.reference | keyword | A URI, URL, or other direct reference to access or locate the entity in its source system. This could be an API endpoint, web console URL, or other addressable location. Format may vary by entity type and source system. | -| entity.attributes.* | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Examples include: `entity.attributes.Storage_class`, `entity.attributes.Mfa_enabled` , `entity.attributes.Privileged` , `entity.attributes.Granted_permissions` , `entity.attributes.Known_redirect` , `entity.attributes.Asset` , `entity.attributes.Managed` ,`entity.attrbitues.Os_current` , `entity.attibutes.Os_patch_current` , `entity.attributes.Oauth_consent_restriction`). Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern for Examples indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | +| entity.attributes | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Examples include: `entity.attributes.Storage_class`, `entity.attributes.Mfa_enabled` , `entity.attributes.Privileged` , `entity.attributes.Granted_permissions` , `entity.attributes.Known_redirect` , `entity.attributes.Asset` , `entity.attributes.Managed` ,`entity.attributes.Os_current` , `entity.attributes.Os_patch_current` , `entity.attributes.Oauth_consent_restriction`). Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern for Examples indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.lifecycle.* | object | A set of temporal characteristics of the entity. Usually date field data type. Examples include: `entity.lifecycle.First_seen`, `entity.lifecycle.Last_activity` , `entity.lifecycle.Issued_at` , `entity.lifecycle.Last_password_change` ,etc. ). Use this field set when you need to track temporal characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.behavior.* | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Behaviors are usually captured in event logs under fields such as `event.action` and other fields, but this field set captures "attributified" behavior indicators, using semantics like "this behavior was seen one or more times during this time period." Sytems using this field set may need to force a "reset" of these behavioral indicators at the end of their current period. Usually boolean field data type. Examples include: `entity.behavior.Used_usb_device`, `entity.behavior.Brute_force_victim` , `entity.behavior.New_country_login` ,etc. ). Use this field set when you need to capture and track ephemeral characterstics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.raw.* | object | Original, unmodified fields from the source system stored in a flattened format that maintains basic searchability. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities. Supports existence queries, exact value matches, and simple aggregations. | From 4b079eca981442048c42c18f113bc10141e0559a Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Tue, 9 Sep 2025 07:35:38 -0400 Subject: [PATCH 05/13] Update rfcs/text/0049-entity-fields.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit clarification. Co-authored-by: Rômulo Farias --- rfcs/text/0049-entity-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 5cb8d7f33c..22d8490c2e 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -28,7 +28,7 @@ This approach would allow ECS to accommodate new types of entities without requi | entity.id | keyword | A unique identifier for the entity. When multiple identifiers exist, this should be the most stable and commonly used identifier that: 1) persists across the entity's lifecycle, 2) ensures uniqueness within its scope, 3) is commonly used for queries and correlation, and 4) is readily available in most observations (logs/events). For entities with dedicated field sets (e.g., host, user), this value should match the corresponding *.id field. Alternative identifiers (e.g., ARNs values in AWS, URLs) can be preserved in entity.raw. | | entity.source | keyword | The module or integration that provided this entity data (similar to event.module). | | entity.type | keyword | A standardized high-level classification of the entity. This provides a normalized way to group similar entities across different providers or systems. Example values: `bucket`, `database`, `container`, `function`, `queue`, `host`, `user`, `application`, `session`, etc. | -| entity.sub_type | keyword | The specific type designation for the entity as defined by its provider or system. This field provides more granular classification than entity.type. Examples: `aws_s3_bucket`, `gcp_cloud_storage_bucket`, `azure_blob_container` would all map to type `bucket`. `hardware` , `virtual` , `container` , `node` , `cloud_instance` would all map to entity type `host'.| +| entity.sub_type | keyword | The specific type designation for the entity as defined by its provider or system. This field provides more granular classification than entity.type. Examples: `aws_s3_bucket`, `gcp_cloud_storage_bucket`, `azure_blob_container` would all map to entity type `bucket`. `hardware` , `virtual` , `container` , `node` , `cloud_instance` would all map to entity type `host'.| | entity.name | keyword, text | The name of the entity. The keyword field enables exact matches for filtering and aggregations, while the text field enables full-text search. For entities with dedicated field sets (e.g., `host`), this field should mirrors the corresponding *.name value. | | entity.display_name | keyword, text | An optional field used when a pretty name is desired for entity-centric operations. This field should not be used for correlation with `*.name` fields for entities with dedicated field sets (e.g., `host`). | | entity.reference | keyword | A URI, URL, or other direct reference to access or locate the entity in its source system. This could be an API endpoint, web console URL, or other addressable location. Format may vary by entity type and source system. | From 1121d888cd1619220b529879a9f0fc5558d110e6 Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Tue, 9 Sep 2025 07:43:57 -0400 Subject: [PATCH 06/13] Update rfcs/text/0049-entity-fields.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit two typos. Co-authored-by: Rômulo Farias --- rfcs/text/0049-entity-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 22d8490c2e..7bc676908e 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -34,7 +34,7 @@ This approach would allow ECS to accommodate new types of entities without requi | entity.reference | keyword | A URI, URL, or other direct reference to access or locate the entity in its source system. This could be an API endpoint, web console URL, or other addressable location. Format may vary by entity type and source system. | | entity.attributes | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Examples include: `entity.attributes.Storage_class`, `entity.attributes.Mfa_enabled` , `entity.attributes.Privileged` , `entity.attributes.Granted_permissions` , `entity.attributes.Known_redirect` , `entity.attributes.Asset` , `entity.attributes.Managed` ,`entity.attributes.Os_current` , `entity.attributes.Os_patch_current` , `entity.attributes.Oauth_consent_restriction`). Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern for Examples indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.lifecycle.* | object | A set of temporal characteristics of the entity. Usually date field data type. Examples include: `entity.lifecycle.First_seen`, `entity.lifecycle.Last_activity` , `entity.lifecycle.Issued_at` , `entity.lifecycle.Last_password_change` ,etc. ). Use this field set when you need to track temporal characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | -| entity.behavior.* | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Behaviors are usually captured in event logs under fields such as `event.action` and other fields, but this field set captures "attributified" behavior indicators, using semantics like "this behavior was seen one or more times during this time period." Sytems using this field set may need to force a "reset" of these behavioral indicators at the end of their current period. Usually boolean field data type. Examples include: `entity.behavior.Used_usb_device`, `entity.behavior.Brute_force_victim` , `entity.behavior.New_country_login` ,etc. ). Use this field set when you need to capture and track ephemeral characterstics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | +| entity.behavior.* | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Behaviors are usually captured in event logs under fields such as `event.action` and other fields, but this field set captures "attributified" behavior indicators, using semantics like "this behavior was seen one or more times during this time period." Systems using this field set may need to force a "reset" of these behavioral indicators at the end of their current period. Usually boolean field data type. Examples include: `entity.behavior.Used_usb_device`, `entity.behavior.Brute_force_victim` , `entity.behavior.New_country_login` ,etc. ). Use this field set when you need to capture and track ephemeral characteristics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | | entity.raw.* | object | Original, unmodified fields from the source system stored in a flattened format that maintains basic searchability. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities. Supports existence queries, exact value matches, and simple aggregations. | | entity.metrics.* | object | Field set for any fields containing numeric entity metrics. These use dymanic field data type mapping. Any fields under entity.metrics.* will automatically get the appropriate numeric mapping without needing explicit definition. From 9f3f0b68096bd1d07fe25fc9f6c7b0ce80ab73e2 Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Tue, 9 Sep 2025 07:48:03 -0400 Subject: [PATCH 07/13] Update rfcs/text/0049-entity-fields.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit extraneous ")" Co-authored-by: Rômulo Farias --- rfcs/text/0049-entity-fields.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 7bc676908e..98fd751912 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -52,7 +52,7 @@ This approach ensures backward compatibility, maintains existing ECS patterns, a ### ECS Producers For ECS producers, such as Beats, Elastic Agent integrations, ingest pipelines, and other methods for shipping data to Elastic, the `entity.*` fields are expected to be nested as follows: -- If the entity type is one of host, user, service, cloud, orchestrator), then the entity fields should be nested under the respecitve root field set, for example `host.entity.*` , `user.entity.*`, etc. +- If the entity type is one of host, user, service, cloud, orchestrator, then the entity fields should be nested under the respecitve root field set, for example `host.entity.*` , `user.entity.*`, etc. - If the entity type is not one of the above, then that `entity.*` fields should be nested under a new root-level object, called `generic`, as `generic.entity.*` Special note for use with entity fields that use `target` fields. From e304bd4b967674215f2cf2a8347c7025981a901d Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Wed, 24 Sep 2025 14:51:05 -0400 Subject: [PATCH 08/13] Update 0049-entity-fields.md --- rfcs/text/0049-entity-fields.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 98fd751912..48b2294e85 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -21,8 +21,6 @@ This approach would allow ECS to accommodate new types of entities without requi | Field | Type | Description | |-------|------|-------------| -| entity.definition_id | keyword | Used Elastic solutions (e.g., Security, Observability) to denote the ID of the entity definition which is used to extract entity details from ingested logs, events, intelligence, and other data types. Use of this value is reserved, and ECS producers, including data ingestion pipelines, must not populate this field| -| entity.definition_id | keyword | Used by Elastic solutions (e.g., Security, Observability) to denote the version of the entity definition which is used to extract entity details from ingested logs, events, intelligence, and other data types. Use of this value is reserved, and ECS producers, including data ingestion pipelines, must not populate this field| | entity.schema_version | keyword | Denotes the version of the entity schema,as published in Elastic Security documentation, to which this entity information conforms. Usually conforms to the Elastic Stack version. | entity.last_seen_timestamp | date | Indicates the date/time when this entity was last "seen," usually based upon the last event/log that is initiated by this entity. Note: this field may contain the same timestamp as is included in custom field `entity.lifecycle.Last_activity`. | entity.id | keyword | A unique identifier for the entity. When multiple identifiers exist, this should be the most stable and commonly used identifier that: 1) persists across the entity's lifecycle, 2) ensures uniqueness within its scope, 3) is commonly used for queries and correlation, and 4) is readily available in most observations (logs/events). For entities with dedicated field sets (e.g., host, user), this value should match the corresponding *.id field. Alternative identifiers (e.g., ARNs values in AWS, URLs) can be preserved in entity.raw. | From 148ee1fc0e60ff3ff541b214a79255fedc31923b Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Wed, 24 Sep 2025 14:55:07 -0400 Subject: [PATCH 09/13] Update 0049-entity-fields.md timestamp field note removed --- rfcs/text/0049-entity-fields.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 48b2294e85..d2600ec82f 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -21,8 +21,7 @@ This approach would allow ECS to accommodate new types of entities without requi | Field | Type | Description | |-------|------|-------------| -| entity.schema_version | keyword | Denotes the version of the entity schema,as published in Elastic Security documentation, to which this entity information conforms. Usually conforms to the Elastic Stack version. -| entity.last_seen_timestamp | date | Indicates the date/time when this entity was last "seen," usually based upon the last event/log that is initiated by this entity. Note: this field may contain the same timestamp as is included in custom field `entity.lifecycle.Last_activity`. +| entity.last_seen_timestamp | date | Indicates the date/time when this entity was last "seen," usually based upon the last event/log that is initiated by this entity. | entity.id | keyword | A unique identifier for the entity. When multiple identifiers exist, this should be the most stable and commonly used identifier that: 1) persists across the entity's lifecycle, 2) ensures uniqueness within its scope, 3) is commonly used for queries and correlation, and 4) is readily available in most observations (logs/events). For entities with dedicated field sets (e.g., host, user), this value should match the corresponding *.id field. Alternative identifiers (e.g., ARNs values in AWS, URLs) can be preserved in entity.raw. | | entity.source | keyword | The module or integration that provided this entity data (similar to event.module). | | entity.type | keyword | A standardized high-level classification of the entity. This provides a normalized way to group similar entities across different providers or systems. Example values: `bucket`, `database`, `container`, `function`, `queue`, `host`, `user`, `application`, `session`, etc. | From 306e76672c8d11f4974ba49fa846be1090075e59 Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Wed, 24 Sep 2025 15:09:12 -0400 Subject: [PATCH 10/13] Update 0049-entity-fields.md updated definitions --- rfcs/text/0049-entity-fields.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index d2600ec82f..3b1b0625f2 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -29,10 +29,10 @@ This approach would allow ECS to accommodate new types of entities without requi | entity.name | keyword, text | The name of the entity. The keyword field enables exact matches for filtering and aggregations, while the text field enables full-text search. For entities with dedicated field sets (e.g., `host`), this field should mirrors the corresponding *.name value. | | entity.display_name | keyword, text | An optional field used when a pretty name is desired for entity-centric operations. This field should not be used for correlation with `*.name` fields for entities with dedicated field sets (e.g., `host`). | | entity.reference | keyword | A URI, URL, or other direct reference to access or locate the entity in its source system. This could be an API endpoint, web console URL, or other addressable location. Format may vary by entity type and source system. | -| entity.attributes | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Examples include: `entity.attributes.Storage_class`, `entity.attributes.Mfa_enabled` , `entity.attributes.Privileged` , `entity.attributes.Granted_permissions` , `entity.attributes.Known_redirect` , `entity.attributes.Asset` , `entity.attributes.Managed` ,`entity.attributes.Os_current` , `entity.attributes.Os_patch_current` , `entity.attributes.Oauth_consent_restriction`). Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern for Examples indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | -| entity.lifecycle.* | object | A set of temporal characteristics of the entity. Usually date field data type. Examples include: `entity.lifecycle.First_seen`, `entity.lifecycle.Last_activity` , `entity.lifecycle.Issued_at` , `entity.lifecycle.Last_password_change` ,etc. ). Use this field set when you need to track temporal characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | -| entity.behavior.* | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Behaviors are usually captured in event logs under fields such as `event.action` and other fields, but this field set captures "attributified" behavior indicators, using semantics like "this behavior was seen one or more times during this time period." Systems using this field set may need to force a "reset" of these behavioral indicators at the end of their current period. Usually boolean field data type. Examples include: `entity.behavior.Used_usb_device`, `entity.behavior.Brute_force_victim` , `entity.behavior.New_country_login` ,etc. ). Use this field set when you need to capture and track ephemeral characteristics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types. Note the initial capitalization pattern indicates that any such fields are custom entity-specific fields that won't be enumerated in the ECS schema, and won't collide with any fields that may be defined by ECS in the future. | -| entity.raw.* | object | Original, unmodified fields from the source system stored in a flattened format that maintains basic searchability. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities. Supports existence queries, exact value matches, and simple aggregations. | +| entity.attributes | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types.| +| entity.lifecycle | object | A set of temporal characteristics of the entity. Usually date field data type. Use this field set when you need to track temporal characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. | +| entity.behavior | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Usually boolean field data type. Use this field set when you need to capture and track ephemeral characteristics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types.| +| entity.raw | object | Original, unmodified fields from the source system. Usually flattened field data type. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities.| | entity.metrics.* | object | Field set for any fields containing numeric entity metrics. These use dymanic field data type mapping. Any fields under entity.metrics.* will automatically get the appropriate numeric mapping without needing explicit definition. When representing entities that have existing ECS entity field sets (e.g., hosts, users, services, containers), the relevant ECS field set should continue to be used to capture a copy of the detailed metadata about that entity. For example: From b879c6642b0c2cd4c457de7cfe79c82f4681f39a Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Wed, 24 Sep 2025 15:17:41 -0400 Subject: [PATCH 11/13] Update 0049-entity-fields.md --- rfcs/text/0049-entity-fields.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 3b1b0625f2..3b02227f27 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -50,15 +50,11 @@ This approach ensures backward compatibility, maintains existing ECS patterns, a For ECS producers, such as Beats, Elastic Agent integrations, ingest pipelines, and other methods for shipping data to Elastic, the `entity.*` fields are expected to be nested as follows: - If the entity type is one of host, user, service, cloud, orchestrator, then the entity fields should be nested under the respecitve root field set, for example `host.entity.*` , `user.entity.*`, etc. -- If the entity type is not one of the above, then that `entity.*` fields should be nested under a new root-level object, called `generic`, as `generic.entity.*` +- If the entity type is not one of the above, then that `entity.*` fields should be nested under a new root-level object. Special note for use with entity fields that use `target` fields. If the entity type includes a provision for `target.*` fields, such as `user.target.*`, then the `entity.*` fields should be nested under the `*.target.*` field set, for example, `user.target.entity.*`. -### ECS Consumers or Data Stores - -For ECS consumers, such as the Elastic Security Solution entity store indices, the `entity.*` fields should be used directly at the root of the events. - ## User Benefits This approach will enable security analysts to view all the entities discovered inside of their environment, whether from logs or other data sources. The entity field set will then begin powering all parts of our security solution experience like alerts, where we can now represent more entities beyond just users and hosts. From 83233423acb292758f3c58ae46d5e549375b2719 Mon Sep 17 00:00:00 2001 From: Mike Paquette Date: Wed, 24 Sep 2025 15:28:00 -0400 Subject: [PATCH 12/13] Update 0049-entity-fields.md added allowed values for entity.type --- rfcs/text/0049-entity-fields.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 3b02227f27..39afe60c4d 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -57,11 +57,11 @@ If the entity type includes a provision for `target.*` fields, such as `user.tar ## User Benefits -This approach will enable security analysts to view all the entities discovered inside of their environment, whether from logs or other data sources. The entity field set will then begin powering all parts of our security solution experience like alerts, where we can now represent more entities beyond just users and hosts. +This approach will enable analysts to view all entities discovered inside of their environment, whether from logs or other data sources. -Essentially, this field set gives us a standard way to represent any entity's metadata, regardless of its type or source, and provides users with the same ability to standardize that information across their environments. +Essentially, this field set provides a standard way to represent any entity's metadata, regardless of its type or source, and provides analysts with the same ability to standardize that information across their environments. -The entity field set enables users to normalize entity data in such a way where key attributes can be queried in a standardized way regardless of the type and source of the entity metadata. Elastic solutions normalize all entity data using this entity.* field set. +The entity field set enables users to normalize entity data in such a way where key attributes can be queried in a standardized way regardless of the type and source of the entity metadata. ## Source data @@ -76,7 +76,7 @@ TO DO ### Entity Type Governance -The `entity.type` field needs a controlled vocabulary to maintain consistency and interoperability. However, an overly restrictive list might limit the field set's utility for emerging technologies and use cases. +The `entity.type` field will use an allowed value list as part of a controlled vocabulary to maintain consistency and interoperability. Allowed values include: `host`, `user`, `service`. **Potential solution:** Establish a governance process for `entity.type` values, including an initial set of well-defined types and a mechanism for proposing and reviewing new types. Document a clear taxonomy with examples to guide users in selecting appropriate types. From d5159f128544e6b2a603303ec091cc67de5e3c15 Mon Sep 17 00:00:00 2001 From: Michael Wolf Date: Wed, 24 Sep 2025 13:39:08 -0700 Subject: [PATCH 13/13] Update entity fields RFC with new date and corrections --- rfcs/text/0049-entity-fields.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/rfcs/text/0049-entity-fields.md b/rfcs/text/0049-entity-fields.md index 39afe60c4d..5514d78931 100644 --- a/rfcs/text/0049-entity-fields.md +++ b/rfcs/text/0049-entity-fields.md @@ -2,7 +2,7 @@ - Stage: **1 (draft)** - Date: 2025-04-23 -- Updated: 2025-08-15 +- Updated: 2025-09-24 An entity represents a discrete, identifiable component within an IT environment that can be described by a set of attributes and maintains its identity over time. Entities can be physical (like hosts or devices), logical (like containers or processes), or abstract (like applications or services). @@ -29,11 +29,11 @@ This approach would allow ECS to accommodate new types of entities without requi | entity.name | keyword, text | The name of the entity. The keyword field enables exact matches for filtering and aggregations, while the text field enables full-text search. For entities with dedicated field sets (e.g., `host`), this field should mirrors the corresponding *.name value. | | entity.display_name | keyword, text | An optional field used when a pretty name is desired for entity-centric operations. This field should not be used for correlation with `*.name` fields for entities with dedicated field sets (e.g., `host`). | | entity.reference | keyword | A URI, URL, or other direct reference to access or locate the entity in its source system. This could be an API endpoint, web console URL, or other addressable location. Format may vary by entity type and source system. | -| entity.attributes | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Use this field set when you need to track static or semi-static characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types.| -| entity.lifecycle | object | A set of temporal characteristics of the entity. Usually date field data type. Use this field set when you need to track temporal characterstics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. | +| entity.attributes | object | A set of static or semi-static attributes of the entity. Usually boolean or keyword field data types. Use this field set when you need to track static or semi-static characteristics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types.| +| entity.lifecycle | object | A set of temporal characteristics of the entity. Usually date field data type. Use this field set when you need to track temporal characteristics of an entity for advanced searching and correlation of normalized values across different providers/sources and entity types. | | entity.behavior | object | A set of ephemeral characteristics of the entity, derived from observed behaviors during a specific time period. Usually boolean field data type. Use this field set when you need to capture and track ephemeral characteristics of an entity for advanced searching, correlation of normalized values across different providers/sources and entity types.| | entity.raw | object | Original, unmodified fields from the source system. Usually flattened field data type. While `entity.attributes` should be used for normalized fields requiring advanced queries, this field preserves all source metadata with basic search capabilities.| -| entity.metrics.* | object | Field set for any fields containing numeric entity metrics. These use dymanic field data type mapping. Any fields under entity.metrics.* will automatically get the appropriate numeric mapping without needing explicit definition. +| entity.metrics | object | Field set for any fields containing numeric entity metrics. These use dynamic field data type mapping. When representing entities that have existing ECS entity field sets (e.g., hosts, users, services, containers), the relevant ECS field set should continue to be used to capture a copy of the detailed metadata about that entity. For example: @@ -49,7 +49,7 @@ This approach ensures backward compatibility, maintains existing ECS patterns, a ### ECS Producers For ECS producers, such as Beats, Elastic Agent integrations, ingest pipelines, and other methods for shipping data to Elastic, the `entity.*` fields are expected to be nested as follows: -- If the entity type is one of host, user, service, cloud, orchestrator, then the entity fields should be nested under the respecitve root field set, for example `host.entity.*` , `user.entity.*`, etc. +- If the entity type is one of host, user, service, cloud, orchestrator, then the entity fields should be nested under the respective root field set, for example `host.entity.*` , `user.entity.*`, etc. - If the entity type is not one of the above, then that `entity.*` fields should be nested under a new root-level object. Special note for use with entity fields that use `target` fields.