From 6e083e4d0b5666ae49ea8dff84cdbf93e85d6c9e Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Wed, 18 Dec 2024 10:57:59 +0200 Subject: [PATCH 01/19] related content 2 --- .../alter-callout-policy-command.md | 35 +++++++++++-------- .../alter-database-sharding-policy-command.md | 11 +++++- ...base-streaming-ingestion-policy-command.md | 17 +++++++-- .../alter-merge-callout-policy-command.md | 35 +++++++++++-------- ...-merge-database-sharding-policy-command.md | 11 +++++- ...ter-merge-table-sharding-policy-command.md | 11 +++++- ...able-streaming-ingestion-policy-command.md | 13 ++++++- .../alter-table-sharding-policy-command.md | 11 +++++- ...able-streaming-ingestion-policy-command.md | 13 ++++++- .../delete-callout-policy-command.md | 29 +++++++++------ ...delete-database-sharding-policy-command.md | 14 +++++++- ...base-streaming-ingestion-policy-command.md | 15 +++++++- .../delete-table-sharding-policy-command.md | 14 +++++++- ...able-streaming-ingestion-policy-command.md | 16 +++++++-- .../kusto/management/sharding-policy.md | 20 ++++++++--- .../show-cluster-callout-policy-command.md | 31 ++++++++++------ .../show-database-sharding-policy-command.md | 14 +++++++- .../show-table-sharding-policy-command.md | 14 +++++++- ...able-streaming-ingestion-policy-command.md | 17 +++++++-- .../management/streaming-ingestion-policy.md | 21 ++++++++--- 20 files changed, 287 insertions(+), 75 deletions(-) diff --git a/data-explorer/kusto/management/alter-callout-policy-command.md b/data-explorer/kusto/management/alter-callout-policy-command.md index 895fd1ce22..8cc623e66d 100644 --- a/data-explorer/kusto/management/alter-callout-policy-command.md +++ b/data-explorer/kusto/management/alter-callout-policy-command.md @@ -3,7 +3,7 @@ title: .alter cluster policy callout command description: Learn how to use the `.alter cluster policy callout` command to change the cluster's callout policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter cluster policy callout command @@ -24,19 +24,19 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co ## Parameters -| Name | Type | Required | Description | -|----------------------------------|--------|----------|----------------------------------------------------------------------------------------------------------| -| *SerializedArrayOfPolicyObjects* | `string` | :heavy_check_mark: | A serialized array of JSON policy objects. See [callout policy](callout-policy.md) for policy properties. | +| Name | Type | Required | Description | +|--|--|--|--| +| *SerializedArrayOfPolicyObjects* | `string` | :heavy_check_mark: | A serialized array of JSON policy objects. See [callout policy](callout-policy.md) for policy properties. | ## Returns -| Name | Type | Description | -|---------------|--------|-----------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster callout policy, this value is **CalloutPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For cluster callout policy, this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy, this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy, this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster callout policy, this value is **CalloutPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For cluster callout policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy, this value is an empty string. | ## Examples @@ -59,6 +59,13 @@ Define permitted callouts for the cluster callout policy. **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|---------------|------------|-------------------------------------------------------------------------------------------------------|---------------|------------| -| CalloutPolicy | | [{"CalloutType":"sql","CalloutUriRegex":"sqlname\\\\.database\\\\.azure\\\\.com/?$","CanCall": true}] | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| CalloutPolicy | | [{"CalloutType":"sql","CalloutUriRegex":"sqlname\\\\.database\\\\.azure\\\\.com/?$","CanCall": true}] | | | + +# Related content + +* [Callout policy](callout-policy.md) +* [.show cluster policy callout command](show-cluster-callout-policy-command.md) +* [.alter-merge cluster policy callout command](alter-merge-callout-policy-command.md) +* [.delete cluster policy callout command](delete-callout-policy-command.md) diff --git a/data-explorer/kusto/management/alter-database-sharding-policy-command.md b/data-explorer/kusto/management/alter-database-sharding-policy-command.md index 06435d7790..cb9d7de583 100644 --- a/data-explorer/kusto/management/alter-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-database-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter database policy sharding command description: Learn how to use the `.alter database policy sharding` command to change the database's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter database policy sharding command @@ -49,3 +49,12 @@ The following command returns the updated extents sharding policy for the databa } ``` ```` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show database policy sharding command](show-database-sharding-policy-command.md) +* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) +* [.delete database policy sharding command](delete-database-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md index a43107c0f4..919ffab786 100644 --- a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .alter database policy streamingingestion command description: Learn how to use the `.alter database policy streamingingestion` command to change the database streaming ingestion policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter database policy streamingingestion command @@ -14,7 +14,7 @@ Changes the database streaming ingestion policy. The [streaming policy](../manag Streaming ingestion is best suited for low latency scenarios where the ingestion time is under 10 seconds for varying data volume. It can optimize processing for multiple tables across one or more databases, especially when the tables receive only a few records per second but the ingestion volume is thousands of records per second. -However, when the amount of data grows beyond 4 Gb per hour per table, it's recommended to switch to classic bulk ingestion instead of streaming ingestion. +However, when the amount of data grows beyond 4 GB per hour per table, the recommendation is to switch to classic bulk ingestion instead of streaming ingestion. To learn how to implement streaming ingestion, see [streaming ingestion](/azure/data-explorer/ingest-data-streaming). Streaming ingestion must be enabled at the cluster level before it can be used at the database level. @@ -48,3 +48,16 @@ The following command enables streaming ingestion and determines the suggestion .alter database MyDatabase policy streamingingestion '{"IsEnabled": true, "HintAllocatedRate": 2.1}' ``` + +## Related content + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command] +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) +(alter-merge-table-streaming-ingestion-policy-command.md) +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) diff --git a/data-explorer/kusto/management/alter-merge-callout-policy-command.md b/data-explorer/kusto/management/alter-merge-callout-policy-command.md index 8c7da8881d..b660c02e93 100644 --- a/data-explorer/kusto/management/alter-merge-callout-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-callout-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge cluster policy callout command description: Learn how to use the `.alter-merge cluster policy callout` command to turn on or turn off a cluster's callout policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter-merge cluster policy callout command @@ -24,19 +24,19 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co ## Parameters -| Name | Type | Required | Description | -|----------------------------------|--------|----------|----------------------------------------------------------------------------------------------------------| -| *SerializedArrayOfPolicyObjects* | `string` | :heavy_check_mark: | A serialized array of JSON policy objects. See [callout policy](callout-policy.md) for policy properties. | +| Name | Type | Required | Description | +|--|--|--|--| +| *SerializedArrayOfPolicyObjects* | `string` | :heavy_check_mark: | A serialized array of JSON policy objects. See [callout policy](callout-policy.md) for policy properties. | ## Returns -| Name | Type | Description | -|---------------|--------|-----------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster callout policy, this value is **CalloutPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For cluster callout policy, this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy, this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy, this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster callout policy, this value is **CalloutPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For cluster callout policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy, this value is an empty string. | ### Examples @@ -55,12 +55,19 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|---------------|------------|------------------------------------------------------------------------------------------------------|---------------|------------| -| CalloutPolicy | | [{"CalloutType":"sql","CalloutUriRegex":"sqlname\\\\.database\\\\.azure\\\\.com/?$","CanCall":true}] | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| CalloutPolicy | | [{"CalloutType":"sql","CalloutUriRegex":"sqlname\\\\.database\\\\.azure\\\\.com/?$","CanCall":true}] | | | ## Remarks If the callout policies currently defined for the cluster already contain one defined for the exact same combination of CalloutType and CalloutUriRegex provided in any of the policies passed to this command, that policy's CanCall property is set to whatever value is specified in this command. All other objects passed in the array are simply merged/added to the existing cluster policy. + +## Related content + +* [Callout policy](callout-policy.md) +* [.show cluster policy callout command](show-cluster-callout-policy-command.md) +* [.alter cluster policy callout command](alter-callout-policy-command.md) +* [.delete cluster policy callout command](delete-callout-policy-command.md) diff --git a/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md b/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md index 209cd8c5bf..ba851cf75c 100644 --- a/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge database policy sharding command description: Learn how to use the `.alter-merge database policy sharding` command to change the database sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter-merge database policy sharding command @@ -43,3 +43,12 @@ The following command changes a single property for the sharding policy for a da .alter-merge database MyDatabase policy sharding @'{ "MaxExtentSizeInMb": 1024}' ``` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show database policy sharding command](show-database-sharding-policy-command.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) +* [.delete database policy sharding command](delete-database-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md b/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md index 54ecdc3e1e..eb263fe3f3 100644 --- a/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge table policy sharding command description: Learn how to use the `.alter-merge table policy sharding` command to change the table's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter-merge table policy sharding command @@ -43,3 +43,12 @@ The following command changes a single property for the sharding policy for a ta .alter-merge table MyTable policy sharding @'{ "MaxExtentSizeInMb": 1024}' ``` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show table policy sharding command](show-table-sharding-policy-command.md) +* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) +* [.delete table policy sharding command](delete-table-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md index 5027f29e59..43df59d067 100644 --- a/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md @@ -14,7 +14,7 @@ Changes the table's streaming ingestion policy. Use the [streaming policy](../ma Streaming ingestion is best suited for low latency scenarios where the ingestion time is under 10 seconds for varying data volume. It can optimize processing for multiple tables across one or more databases, especially when the tables receive only a few records per second but the ingestion volume is thousands of records per second. -However, when the amount of data grows beyond 4 Gb per hour per table, it's recommended to switch to classic bulk ingestion instead of streaming ingestion. +However, when the amount of data grows beyond 4 GB per hour per table, it's recommended to switch to classic bulk ingestion instead of streaming ingestion. To learn how to implement streaming ingestion, see [streaming ingestion](/azure/data-explorer/ingest-data-streaming). @@ -54,3 +54,14 @@ The following command disables the streaming ingestion policy: .alter-merge table MyTable policy streamingingestion '{"IsEnabled": false}' ``` + +## Related content + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) +(alter-merge-table-streaming-ingestion-policy-command.md) diff --git a/data-explorer/kusto/management/alter-table-sharding-policy-command.md b/data-explorer/kusto/management/alter-table-sharding-policy-command.md index a21b59f727..db62a4a97d 100644 --- a/data-explorer/kusto/management/alter-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter table policy sharding command description: Learn how to use the `.alter table policy sharding` command to change the table sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter table policy sharding command @@ -49,3 +49,12 @@ The following command returns the updated extents sharding policy for the table. } ``` ```` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show table policy sharding command](show-table-sharding-policy-command.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) +* [.delete table policy sharding command](delete-table-sharding-policy-command.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md index b7a81495e4..43f92895f0 100644 --- a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .alter table policy streamingingestion command description: Learn how to use the `.alter table policy streamingingestion` command to change the table streaming policy ingestion. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .alter table policy streamingingestion command @@ -46,3 +46,14 @@ The following command turns on streaming ingestion and determines the suggested ```kusto .alter table Table1 policy streamingingestion '{"IsEnabled": true, "HintAllocatedRate": 2.1}' ``` + +## Related content + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) +(alter-merge-table-streaming-ingestion-policy-command.md) diff --git a/data-explorer/kusto/management/delete-callout-policy-command.md b/data-explorer/kusto/management/delete-callout-policy-command.md index 37b24d8e18..7323399f30 100644 --- a/data-explorer/kusto/management/delete-callout-policy-command.md +++ b/data-explorer/kusto/management/delete-callout-policy-command.md @@ -3,7 +3,7 @@ title: .delete cluster policy callout command description: Learn how to use the `.delete cluster policy callout` command to delete the callout policy defined for a cluster. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .delete cluster policy callout command @@ -24,13 +24,13 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co ## Returns -| Name | Type | Description | -|---------------|--------|-----------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster callout policy this value is **CalloutPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For cluster callout policy this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. This command sets this value to null. | -| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster callout policy, this value is **CalloutPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For cluster callout policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. This command sets this value to null. | +| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy, this value is an empty string. | ## Examples @@ -44,10 +44,17 @@ Delete cluster callout policy: **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|---------------|------------|------------------------------------------------------------------------------------------------------|---------------|------------| -| CalloutPolicy | | null | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| CalloutPolicy | | null | | | ## Remarks Notice that this command doesn't delete the immutable predefined callout policies. For more information, see [predefined callout policies](callout-policy.md#predefined-callout-policies). + +## Related content + +* [Callout policy](callout-policy.md) +* [.show cluster policy callout command](show-cluster-callout-policy-command.md) +* [.alter cluster policy callout command](alter-callout-policy-command.md) +* [.alter-merge cluster policy callout command](alter-merge-callout-policy-command.md) diff --git a/data-explorer/kusto/management/delete-database-sharding-policy-command.md b/data-explorer/kusto/management/delete-database-sharding-policy-command.md index c0465303d1..4e5afa83f1 100644 --- a/data-explorer/kusto/management/delete-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/delete-database-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .delete database policy sharding command description: Learn how to use the `.delete database policy sharding` command to delete the database sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .delete database policy sharding command @@ -37,3 +37,15 @@ The following example deleted the sharding policy for a database: ```kusto .delete database MyDatabase policy sharding ``` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show database policy sharding command](show-database-sharding-policy-command.md) +* [.show table policy sharding command](show-table-sharding-policy-command.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) +* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) +* [.delete table policy sharding command](delete-table-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md index af8d5ac8b2..713e250349 100644 --- a/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md @@ -12,7 +12,7 @@ monikerRange: "azure-data-explorer" Use this command to delete the database streaming ingestion policy. Use the [streaming policy](../management/streaming-ingestion-policy.md) to manage streaming ingestion for databases and tables. -Streaming ingestion is targeted for scenarios that require low latency, with an ingestion time of less than 10 seconds for varied volume data. It's used to optimize operational processing of many tables, in one or more databases, where the stream of data into each table is relatively small (a few records per second) but the overall data ingestion volume is high (thousands of records per second). +Streaming ingestion is targeted for scenarios that require low latency, with an ingestion time of less than 10 seconds for varied volume data. It's used to optimize operational processing of many tables across one or more databases. Each table receives a small stream of data (a few records per second), but the total data ingestion volume is high (thousands of records per second). Use the classic (bulk) ingestion instead of streaming ingestion when the amount of data grows to more than 4 Gb per hour per table. @@ -41,3 +41,16 @@ The following command deletes the streaming ingestion policy: ```kusto .delete database MyDatabase policy streamingingestion ``` + +## Related content + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command] +* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) +(alter-merge-table-streaming-ingestion-policy-command.md) +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/delete-table-sharding-policy-command.md b/data-explorer/kusto/management/delete-table-sharding-policy-command.md index 0aa3a39e12..4bcf350787 100644 --- a/data-explorer/kusto/management/delete-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/delete-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .delete table policy sharding command description: Learn how to use the `.delete table policy sharding` command to delete a table's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .delete table policy sharding command @@ -37,3 +37,15 @@ The following example deleted the sharding policy for a table: ```kusto .delete table MyTable policy sharding ``` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show database policy sharding command](show-database-sharding-policy-command.md) +* [.show table policy sharding command](show-table-sharding-policy-command.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) +* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) +* [.delete database policy sharding command](delete-database-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md index 89d19e8e1d..ebf1508a93 100644 --- a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .delete table policy streamingingestion command description: Learn how to use the `.delete table policy streamingingestion` command to delete a table's streaming ingestion policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .delete table policy streamingingestion command @@ -12,7 +12,7 @@ monikerRange: "azure-data-explorer" Delete the table streaming ingestion policy. Use the [streaming policy](../management/streaming-ingestion-policy.md) to manage streaming ingestion for databases and tables. -Streaming ingestion is targeted for scenarios that require low latency, with an ingestion time of less than 10 seconds for varied volume data. It's used to optimize operational processing of many tables, in one or more databases, where the stream of data into each table is relatively small (a few records per second) but the overall data ingestion volume is high (thousands of records per second). +Streaming ingestion is targeted for scenarios that require low latency, with an ingestion time of less than 10 seconds for varied volume data. It's used to optimize operational processing of many tables across one or more databases. Each table receives a small stream of data (a few records per second), but the total data ingestion volume is high (thousands of records per second). Use the classic (bulk) ingestion instead of streaming ingestion when the amount of data grows to more than 4 Gb per hour per table. @@ -41,3 +41,15 @@ The following command deletes the streaming ingestion policy: ```kusto .delete table MyTable policy streamingingestion ``` + +## Related content + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command] +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/sharding-policy.md b/data-explorer/kusto/management/sharding-policy.md index 00c11c5dbc..d330e03450 100644 --- a/data-explorer/kusto/management/sharding-policy.md +++ b/data-explorer/kusto/management/sharding-policy.md @@ -3,12 +3,12 @@ title: Data sharding policy description: Learn how to use the data sharding policy to define if and how extents in the database are created. ms.reviewer: orspodek ms.topic: reference -ms.date: 11/11/2024 +ms.date: 12/18/2024 monikerRange: "microsoft-fabric || azure-data-explorer" --- # Data sharding policy -> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] +> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] The sharding policy defines if and how [extents (data shards)](../management/extents-overview.md) in your cluster are created. You can only query data in an extent once it's created. @@ -27,12 +27,12 @@ The data sharding policy contains the following properties: * If you must limit the number of rows in extents created by merge operations, adjust the `RowCountUpperBoundForMerge` property in the entity's [extents merge policy](merge-policy.md). * **ShardEngineMaxExtentSizeInMb**: * Maximum allowed compressed data size (in megabytes) for an extent created by a merge or rebuild operation. - * Defaults to 8,192 (8GB). + * Defaults to 8,192 (8 GB). * **ShardEngineMaxOriginalSizeInMb**: * Maximum allowed original data size (in megabytes) for an extent created by a rebuild operation. * In effect **only for [rebuild](merge-policy.md) operations**. - * Defaults to 3,072 (3GB). + * Defaults to 3,072 (3 GB). > [!WARNING] > Consult with the support team before altering a data sharding policy. @@ -40,3 +40,15 @@ The data sharding policy contains the following properties: When a database is created, it contains the default data sharding policy. This policy is inherited by all tables created in the database (unless the policy is explicitly overridden at the table level). Use the [sharding policy management commands](show-table-sharding-policy-command.md) to manage data sharding policies for databases and tables. + +## Related content + +* [.show database policy sharding command](show-database-sharding-policy-command.md) +* [.show table policy sharding command](show-table-sharding-policy-command.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) +* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) +* [.delete database policy sharding command](delete-database-sharding-policy-command.md) +* [.delete table policy sharding command](delete-table-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/show-cluster-callout-policy-command.md b/data-explorer/kusto/management/show-cluster-callout-policy-command.md index 69df20c15f..67d8b0bad8 100644 --- a/data-explorer/kusto/management/show-cluster-callout-policy-command.md +++ b/data-explorer/kusto/management/show-cluster-callout-policy-command.md @@ -3,7 +3,7 @@ title: .show cluster policy callout command description: Learn how to use the `.show cluster policy callout` command to display a cluster's callout policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .show cluster policy callout command @@ -24,13 +24,13 @@ You must have at least [Cluster AllDatabasesMonitor](../access-control/role-base ## Returns -| Name | Type | Description | -|---------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster callout policy this value is **CalloutPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy this value is an array of strings, each of which corresponds to the name of system and user databases attached to the cluster. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy this value is *Cluster*. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster callout policy, this value is **CalloutPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For cluster callout policy, this value is an array of strings, each of which corresponds to the name of system and user databases attached to the cluster. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster callout policy, this value is *Cluster*. | ### Examples @@ -42,6 +42,15 @@ Display the cluster's callout policy. **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|---------------|------------|------------------------------------------------------------------------------------------------------|------------------------------------------------------------|------------| -| CalloutPolicy | | [] | ["$systemdb","KustoMonitoringPersistentDatabase","TestDB"] | Cluster | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| CalloutPolicy | | [] | ["$systemdb","KustoMonitoringPersistentDatabase","TestDB"] | Cluster | + +# Related content + +* [Callout policy](callout-policy.md) +* [.alter cluster policy callout command](alter-callout-policy-command.md) +* [.alter-merge cluster policy callout command](alter-merge-callout-policy-command.md) +* [.delete cluster policy callout command](delete-callout-policy-command.md) +* [External tables](../query/schema-entities/external-tables.md) +* [externaldata operator](../query/externaldata-operator.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/show-database-sharding-policy-command.md b/data-explorer/kusto/management/show-database-sharding-policy-command.md index 35a8f41a7e..e220bbdb91 100644 --- a/data-explorer/kusto/management/show-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/show-database-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .show database policy sharding command description: Learn how to use the `.show database policy sharding` command to show the database's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .show database policy sharding command @@ -41,3 +41,15 @@ The following example shows the table sharding policy: ```kusto .show database MyDatabase policy sharding ``` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show table policy sharding command](show-table-sharding-policy-command.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) +* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) +* [.delete database policy sharding command](delete-database-sharding-policy-command.md) +* [.delete table policy sharding command](delete-table-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/show-table-sharding-policy-command.md b/data-explorer/kusto/management/show-table-sharding-policy-command.md index 1552be8ee7..e3f66bf003 100644 --- a/data-explorer/kusto/management/show-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/show-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .show table policy sharding command description: Learn how to use the `.show table policy sharding` command to show the table's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 11/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer || microsoft-fabric" --- # .show table policy sharding command @@ -41,3 +41,15 @@ The following example shows the sharding policies for all tables: ```kusto .show table * policy sharding ``` + +## Related content + +* [Data sharding policy](sharding-policy.md) +* [.show database policy sharding command](show-database-sharding-policy-command.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) +* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) +* [.delete database policy sharding command](delete-database-sharding-policy-command.md) +* [.delete table policy sharding command](delete-table-sharding-policy-command.md) +* [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md index 0c8662fa1b..f875e4182f 100644 --- a/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .show table policy streamingingestion command description: Learn how to use the `.show table policy streamingingestion` command to display the table's streaming ingestion policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # .show table policy streamingingestion command @@ -12,7 +12,7 @@ monikerRange: "azure-data-explorer" Displays the table's streaming ingestion policy. Use the [streaming policy](../management/streaming-ingestion-policy.md) to manage streaming ingestion for databases and tables. -Streaming ingestion is targeted for scenarios that require low latency, with an ingestion time of less than 10 seconds for varied volume data. It's used to optimize operational processing of many tables, in one or more databases, where the stream of data into each table is relatively small (a few records per second) but the overall data ingestion volume is high (thousands of records per second). +Streaming ingestion is targeted for scenarios that require low latency, with an ingestion time of less than 10 seconds for varied volume data. It's used to optimize operational processing of many tables across one or more databases. Each table receives a small stream of data (a few records per second), but the total data ingestion volume is high (thousands of records per second). Use the classic (bulk) ingestion instead of streaming ingestion when the amount of data grows to more than 4 Gb per hour per table. @@ -41,3 +41,16 @@ The following command displays the streaming ingestion policy: ```kusto .show table MyTable policy streamingingestion ``` + +## Related content + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command] +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) diff --git a/data-explorer/kusto/management/streaming-ingestion-policy.md b/data-explorer/kusto/management/streaming-ingestion-policy.md index e8e647ee5b..72f4240473 100644 --- a/data-explorer/kusto/management/streaming-ingestion-policy.md +++ b/data-explorer/kusto/management/streaming-ingestion-policy.md @@ -3,7 +3,7 @@ title: Streaming ingestion policy description: Learn how to use the streaming ingestion policy to optimize operational processing of many tables where the stream of data is small. ms.reviewer: orspodek ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 monikerRange: "azure-data-explorer" --- # Streaming ingestion policy @@ -41,6 +41,19 @@ Defining this policy at the database level applies the same settings to all exis ## Set the data rate hint -The streaming ingestion policy can provide a hint about the hourly volume of data expected for the table. This hint will help the system adjust the amount of resources allocated for this table in support of streaming ingestion. -Set the hint if the rate of streaming data ingress into the table will exceed 1 Gb/hour. -If setting _HintAllocatedRate_ in the streaming ingestion policy for the database, set it by the table with the highest expected data rate. It isn't recommended to set the effective hint for a table to a value much higher than the expected peak hourly data rate. This setting may have an adverse effect on the query performance. \ No newline at end of file +The streaming ingestion policy can provide a hint about the hourly volume of data expected for the table. This hint helps the system adjust the amount of resources allocated for this table in support of streaming ingestion. +Set the hint if the rate of streaming data ingresses into the table exceeds 1 Gb/hour. +If setting _HintAllocatedRate_ in the streaming ingestion policy for the database, set it by the table with the highest expected data rate. It isn't recommended to set the effective hint for a table to a value much higher than the expected peak hourly data rate. This setting might have an adverse effect on the query performance. + +## Related content + +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command] +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) +(alter-merge-table-streaming-ingestion-policy-command.md) +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) From 0b6ce0b23673a2bb75c5de984639af491086eb76 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Wed, 18 Dec 2024 12:09:00 +0200 Subject: [PATCH 02/19] edit --- data-explorer/kusto/management/alter-callout-policy-command.md | 2 +- .../kusto/management/show-cluster-callout-policy-command.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/data-explorer/kusto/management/alter-callout-policy-command.md b/data-explorer/kusto/management/alter-callout-policy-command.md index 8cc623e66d..8abf1ff522 100644 --- a/data-explorer/kusto/management/alter-callout-policy-command.md +++ b/data-explorer/kusto/management/alter-callout-policy-command.md @@ -63,7 +63,7 @@ Define permitted callouts for the cluster callout policy. |--|--|--|--|--| | CalloutPolicy | | [{"CalloutType":"sql","CalloutUriRegex":"sqlname\\\\.database\\\\.azure\\\\.com/?$","CanCall": true}] | | | -# Related content +## Related content * [Callout policy](callout-policy.md) * [.show cluster policy callout command](show-cluster-callout-policy-command.md) diff --git a/data-explorer/kusto/management/show-cluster-callout-policy-command.md b/data-explorer/kusto/management/show-cluster-callout-policy-command.md index 67d8b0bad8..04ad01d019 100644 --- a/data-explorer/kusto/management/show-cluster-callout-policy-command.md +++ b/data-explorer/kusto/management/show-cluster-callout-policy-command.md @@ -46,7 +46,7 @@ Display the cluster's callout policy. |--|--|--|--|--| | CalloutPolicy | | [] | ["$systemdb","KustoMonitoringPersistentDatabase","TestDB"] | Cluster | -# Related content +## Related content * [Callout policy](callout-policy.md) * [.alter cluster policy callout command](alter-callout-policy-command.md) From 4fe9590cd94c2231ebf02826b6225828f450023a Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Thu, 19 Dec 2024 15:33:34 +0200 Subject: [PATCH 03/19] edit --- data-explorer/kusto/management/callout-policy.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/data-explorer/kusto/management/callout-policy.md b/data-explorer/kusto/management/callout-policy.md index 8031db40fc..ffdee6bc67 100644 --- a/data-explorer/kusto/management/callout-policy.md +++ b/data-explorer/kusto/management/callout-policy.md @@ -64,3 +64,12 @@ More predefined policies on your cluster may be observed with next query: ## Remarks If an external resource of a given type matches more than one policy defined for such type, and at least one of the matched policies has their CanCall property set to false, access to the resource is denied. + +## Related content + +* [.show cluster policy callout command](show-cluster-callout-policy-command.md) +* [.alter cluster policy callout command](alter-callout-policy-command.md) +* [.alter-merge cluster policy callout command](alter-merge-callout-policy-command.md) +* [.delete cluster policy callout command](delete-callout-policy-command.md) +* [External tables](../query/schema-entities/external-tables.md) +* [externaldata operator](../query/externaldata-operator.md) From 97af9c586516b3a562cc9c4f71ff71435be23bd6 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Thu, 19 Dec 2024 19:42:35 +0200 Subject: [PATCH 04/19] edits --- .../alter-database-streaming-ingestion-policy-command.md | 4 ---- ...er-merge-database-streaming-ingestion-policy-command.md | 7 +++++++ .../management/delete-database-sharding-policy-command.md | 3 --- .../delete-database-streaming-ingestion-policy-command.md | 3 --- .../management/delete-table-sharding-policy-command.md | 2 -- .../delete-table-streaming-ingestion-policy-command.md | 5 ++--- .../management/show-cluster-callout-policy-command.md | 2 -- .../management/show-database-sharding-policy-command.md | 3 --- .../kusto/management/show-table-sharding-policy-command.md | 5 +---- .../kusto/management/streaming-ingestion-policy.md | 3 +-- 10 files changed, 11 insertions(+), 26 deletions(-) diff --git a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md index 919ffab786..37e1f2c5b8 100644 --- a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md @@ -53,11 +53,7 @@ The following command enables streaming ingestion and determines the suggestion * [Streaming ingestion policy](streaming-ingestion-policy.md) * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) -* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) * [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) -* [.alter-merge table policy streamingingestion command] * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) -* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) -(alter-merge-table-streaming-ingestion-policy-command.md) * [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) diff --git a/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md index 4bfdf625f5..5e2053dfda 100644 --- a/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md @@ -54,3 +54,10 @@ The following command disables the streaming ingestion policy: .alter-merge database MyDatabase policy streamingingestion '{"IsEnabled": false}' ``` + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) diff --git a/data-explorer/kusto/management/delete-database-sharding-policy-command.md b/data-explorer/kusto/management/delete-database-sharding-policy-command.md index 4e5afa83f1..7e84fb7647 100644 --- a/data-explorer/kusto/management/delete-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/delete-database-sharding-policy-command.md @@ -42,10 +42,7 @@ The following example deleted the sharding policy for a database: * [Data sharding policy](sharding-policy.md) * [.show database policy sharding command](show-database-sharding-policy-command.md) -* [.show table policy sharding command](show-table-sharding-policy-command.md) * [.alter database policy sharding command](alter-database-sharding-policy-command.md) * [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) -* [.alter table policy sharding command](alter-table-sharding-policy-command.md) -* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) * [.delete table policy sharding command](delete-table-sharding-policy-command.md) * [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md index 713e250349..574a7f7ab5 100644 --- a/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md @@ -46,11 +46,8 @@ The following command deletes the streaming ingestion policy: * [Streaming ingestion policy](streaming-ingestion-policy.md) * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) -* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) * [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) * [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) -* [.alter-merge table policy streamingingestion command] * [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) -(alter-merge-table-streaming-ingestion-policy-command.md) * [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/delete-table-sharding-policy-command.md b/data-explorer/kusto/management/delete-table-sharding-policy-command.md index 4bcf350787..ed66a75ac5 100644 --- a/data-explorer/kusto/management/delete-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/delete-table-sharding-policy-command.md @@ -43,8 +43,6 @@ The following example deleted the sharding policy for a table: * [Data sharding policy](sharding-policy.md) * [.show database policy sharding command](show-database-sharding-policy-command.md) * [.show table policy sharding command](show-table-sharding-policy-command.md) -* [.alter database policy sharding command](alter-database-sharding-policy-command.md) -* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) * [.alter table policy sharding command](alter-table-sharding-policy-command.md) * [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) * [.delete database policy sharding command](delete-database-sharding-policy-command.md) diff --git a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md index ebf1508a93..a70adbbb35 100644 --- a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md @@ -47,9 +47,8 @@ The following command deletes the streaming ingestion policy: * [Streaming ingestion policy](streaming-ingestion-policy.md) * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) * [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) -* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) -* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.alter-merge table policy streamingingestion command] +* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) -* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) \ No newline at end of file +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/show-cluster-callout-policy-command.md b/data-explorer/kusto/management/show-cluster-callout-policy-command.md index 04ad01d019..0ce9d887b5 100644 --- a/data-explorer/kusto/management/show-cluster-callout-policy-command.md +++ b/data-explorer/kusto/management/show-cluster-callout-policy-command.md @@ -52,5 +52,3 @@ Display the cluster's callout policy. * [.alter cluster policy callout command](alter-callout-policy-command.md) * [.alter-merge cluster policy callout command](alter-merge-callout-policy-command.md) * [.delete cluster policy callout command](delete-callout-policy-command.md) -* [External tables](../query/schema-entities/external-tables.md) -* [externaldata operator](../query/externaldata-operator.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/show-database-sharding-policy-command.md b/data-explorer/kusto/management/show-database-sharding-policy-command.md index e220bbdb91..b0e0e25dbf 100644 --- a/data-explorer/kusto/management/show-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/show-database-sharding-policy-command.md @@ -48,8 +48,5 @@ The following example shows the table sharding policy: * [.show table policy sharding command](show-table-sharding-policy-command.md) * [.alter database policy sharding command](alter-database-sharding-policy-command.md) * [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) -* [.alter table policy sharding command](alter-table-sharding-policy-command.md) -* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) * [.delete database policy sharding command](delete-database-sharding-policy-command.md) -* [.delete table policy sharding command](delete-table-sharding-policy-command.md) * [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/show-table-sharding-policy-command.md b/data-explorer/kusto/management/show-table-sharding-policy-command.md index e3f66bf003..2b82bdcadc 100644 --- a/data-explorer/kusto/management/show-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/show-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .show table policy sharding command description: Learn how to use the `.show table policy sharding` command to show the table's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/19/2024 monikerRange: "azure-data-explorer || microsoft-fabric" --- # .show table policy sharding command @@ -46,10 +46,7 @@ The following example shows the sharding policies for all tables: * [Data sharding policy](sharding-policy.md) * [.show database policy sharding command](show-database-sharding-policy-command.md) -* [.alter database policy sharding command](alter-database-sharding-policy-command.md) -* [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) * [.alter table policy sharding command](alter-table-sharding-policy-command.md) * [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) -* [.delete database policy sharding command](delete-database-sharding-policy-command.md) * [.delete table policy sharding command](delete-table-sharding-policy-command.md) * [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/streaming-ingestion-policy.md b/data-explorer/kusto/management/streaming-ingestion-policy.md index 72f4240473..f2237a6c46 100644 --- a/data-explorer/kusto/management/streaming-ingestion-policy.md +++ b/data-explorer/kusto/management/streaming-ingestion-policy.md @@ -52,8 +52,7 @@ If setting _HintAllocatedRate_ in the streaming ingestion policy for the databas * [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) * [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) -* [.alter-merge table policy streamingingestion command] +* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) * [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) -(alter-merge-table-streaming-ingestion-policy-command.md) * [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) From 2a6d474c0d03b8c5584a37f6982abcd61c073cc4 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Thu, 19 Dec 2024 21:00:41 +0200 Subject: [PATCH 05/19] edits --- ...ter-merge-table-streaming-ingestion-policy-command.md | 1 - .../alter-table-streaming-ingestion-policy-command.md | 1 - ...delete-database-streaming-ingestion-policy-command.md | 3 +-- .../delete-table-streaming-ingestion-policy-command.md | 3 +-- .../show-database-streaming-ingestion-policy-command.md | 9 +++++++++ 5 files changed, 11 insertions(+), 6 deletions(-) diff --git a/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md index 43df59d067..d24e0e41af 100644 --- a/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md @@ -64,4 +64,3 @@ The following command disables the streaming ingestion policy: * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) * [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) -(alter-merge-table-streaming-ingestion-policy-command.md) diff --git a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md index 43f92895f0..eedb9bd3c1 100644 --- a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md @@ -56,4 +56,3 @@ The following command turns on streaming ingestion and determines the suggested * [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) * [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) -(alter-merge-table-streaming-ingestion-policy-command.md) diff --git a/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md index 574a7f7ab5..5441b8b31f 100644 --- a/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-database-streaming-ingestion-policy-command.md @@ -48,6 +48,5 @@ The following command deletes the streaming ingestion policy: * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) * [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) * [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) -* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) -* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) \ No newline at end of file +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) diff --git a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md index a70adbbb35..5692fc00d3 100644 --- a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md @@ -45,10 +45,9 @@ The following command deletes the streaming ingestion policy: ## Related content * [Streaming ingestion policy](streaming-ingestion-policy.md) -* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +(show-database-streaming-ingestion-policy-command.md) * [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) -* [.alter-merge table policy streamingingestion command] * [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) * [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/show-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/show-database-streaming-ingestion-policy-command.md index 0011be8b63..6540b97c70 100644 --- a/data-explorer/kusto/management/show-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/show-database-streaming-ingestion-policy-command.md @@ -41,3 +41,12 @@ The following command shows the streaming ingestion policy: ```kusto .show database MyDatabase policy streamingingestion ``` + +## Related content + +* [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) +* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) From e9288fbf556e4acd5846e0c7b329ae325283fefe Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Sun, 22 Dec 2024 11:07:03 +0200 Subject: [PATCH 06/19] edits --- .../management/alter-database-sharding-policy-command.md | 4 ++-- .../alter-database-streaming-ingestion-policy-command.md | 4 ++-- .../alter-merge-database-sharding-policy-command.md | 4 ++-- ...alter-merge-database-streaming-ingestion-policy-command.md | 4 ++-- .../management/alter-merge-table-sharding-policy-command.md | 4 ++-- .../alter-merge-table-streaming-ingestion-policy-command.md | 4 ++-- .../kusto/management/alter-table-sharding-policy-command.md | 4 ++-- .../alter-table-streaming-ingestion-policy-command.md | 4 ++-- data-explorer/kusto/management/callout-policy.md | 2 -- .../kusto/management/delete-table-sharding-policy-command.md | 3 +-- .../delete-table-streaming-ingestion-policy-command.md | 4 ++-- 11 files changed, 19 insertions(+), 22 deletions(-) diff --git a/data-explorer/kusto/management/alter-database-sharding-policy-command.md b/data-explorer/kusto/management/alter-database-sharding-policy-command.md index cb9d7de583..6d62b04f2b 100644 --- a/data-explorer/kusto/management/alter-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-database-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter database policy sharding command description: Learn how to use the `.alter database policy sharding` command to change the database's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter database policy sharding command @@ -53,8 +53,8 @@ The following command returns the updated extents sharding policy for the databa ## Related content * [Data sharding policy](sharding-policy.md) +* [.alter table policy sharding command](alter-table-sharding-policy-command.md) * [.show database policy sharding command](show-database-sharding-policy-command.md) * [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) -* [.alter table policy sharding command](alter-table-sharding-policy-command.md) * [.delete database policy sharding command](delete-database-sharding-policy-command.md) * [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md index 37e1f2c5b8..63ae307e31 100644 --- a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .alter database policy streamingingestion command description: Learn how to use the `.alter database policy streamingingestion` command to change the database streaming ingestion policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter database policy streamingingestion command @@ -52,8 +52,8 @@ The following command enables streaming ingestion and determines the suggestion ## Related content * [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) * [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) -* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) * [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) diff --git a/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md b/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md index ba851cf75c..d35d3538a2 100644 --- a/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-database-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge database policy sharding command description: Learn how to use the `.alter-merge database policy sharding` command to change the database sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter-merge database policy sharding command @@ -47,8 +47,8 @@ The following command changes a single property for the sharding policy for a da ## Related content * [Data sharding policy](sharding-policy.md) +* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) * [.show database policy sharding command](show-database-sharding-policy-command.md) * [.alter database policy sharding command](alter-database-sharding-policy-command.md) -* [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) * [.delete database policy sharding command](delete-database-sharding-policy-command.md) * [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md index 5e2053dfda..568a987f6d 100644 --- a/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-database-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge database policy streamingingestion command description: Learn how to use the `.alter-merge database policy streamingingestion` command to change the database streaming policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter-merge database policy streamingingestion command @@ -56,8 +56,8 @@ The following command disables the streaming ingestion policy: ``` * [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) * [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) -* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) * [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) diff --git a/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md b/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md index eb263fe3f3..bcfa2a6f66 100644 --- a/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge table policy sharding command description: Learn how to use the `.alter-merge table policy sharding` command to change the table's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter-merge table policy sharding command @@ -47,8 +47,8 @@ The following command changes a single property for the sharding policy for a ta ## Related content * [Data sharding policy](sharding-policy.md) -* [.show table policy sharding command](show-table-sharding-policy-command.md) * [.alter-merge database policy sharding command](alter-merge-database-sharding-policy-command.md) +* [.show table policy sharding command](show-table-sharding-policy-command.md) * [.alter table policy sharding command](alter-table-sharding-policy-command.md) * [.delete table policy sharding command](delete-table-sharding-policy-command.md) * [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md index d24e0e41af..fa68d3ef16 100644 --- a/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-table-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge table policy streamingingestion command description: Learn how to use the `.alter-merge table policy streamingingestion` command to change the table's streaming ingestion policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter-merge table policy streamingingestion command @@ -58,9 +58,9 @@ The following command disables the streaming ingestion policy: ## Related content * [Streaming ingestion policy](streaming-ingestion-policy.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) * [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) -* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) * [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) diff --git a/data-explorer/kusto/management/alter-table-sharding-policy-command.md b/data-explorer/kusto/management/alter-table-sharding-policy-command.md index db62a4a97d..6b591c3731 100644 --- a/data-explorer/kusto/management/alter-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/alter-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .alter table policy sharding command description: Learn how to use the `.alter table policy sharding` command to change the table sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter table policy sharding command @@ -53,8 +53,8 @@ The following command returns the updated extents sharding policy for the table. ## Related content * [Data sharding policy](sharding-policy.md) +* [.alter database policy sharding command](alter-database-sharding-policy-command.md) * [.show table policy sharding command](show-table-sharding-policy-command.md) * [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) * [.delete table policy sharding command](delete-table-sharding-policy-command.md) -* [.alter database policy sharding command](alter-database-sharding-policy-command.md) * [Extents (data shards)](extents-overview.md) diff --git a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md index eedb9bd3c1..f494d7d816 100644 --- a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .alter table policy streamingingestion command description: Learn how to use the `.alter table policy streamingingestion` command to change the table streaming policy ingestion. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .alter table policy streamingingestion command @@ -50,8 +50,8 @@ The following command turns on streaming ingestion and determines the suggested ## Related content * [Streaming ingestion policy](streaming-ingestion-policy.md) -* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) * [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) * [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) * [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) diff --git a/data-explorer/kusto/management/callout-policy.md b/data-explorer/kusto/management/callout-policy.md index ffdee6bc67..203328c607 100644 --- a/data-explorer/kusto/management/callout-policy.md +++ b/data-explorer/kusto/management/callout-policy.md @@ -71,5 +71,3 @@ If an external resource of a given type matches more than one policy defined for * [.alter cluster policy callout command](alter-callout-policy-command.md) * [.alter-merge cluster policy callout command](alter-merge-callout-policy-command.md) * [.delete cluster policy callout command](delete-callout-policy-command.md) -* [External tables](../query/schema-entities/external-tables.md) -* [externaldata operator](../query/externaldata-operator.md) diff --git a/data-explorer/kusto/management/delete-table-sharding-policy-command.md b/data-explorer/kusto/management/delete-table-sharding-policy-command.md index ed66a75ac5..89c5f501ed 100644 --- a/data-explorer/kusto/management/delete-table-sharding-policy-command.md +++ b/data-explorer/kusto/management/delete-table-sharding-policy-command.md @@ -3,7 +3,7 @@ title: .delete table policy sharding command description: Learn how to use the `.delete table policy sharding` command to delete a table's sharding policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .delete table policy sharding command @@ -41,7 +41,6 @@ The following example deleted the sharding policy for a table: ## Related content * [Data sharding policy](sharding-policy.md) -* [.show database policy sharding command](show-database-sharding-policy-command.md) * [.show table policy sharding command](show-table-sharding-policy-command.md) * [.alter table policy sharding command](alter-table-sharding-policy-command.md) * [.alter-merge table policy sharding command](alter-merge-table-sharding-policy-command.md) diff --git a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md index 5692fc00d3..9f68fb52c4 100644 --- a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md @@ -3,7 +3,7 @@ title: .delete table policy streamingingestion command description: Learn how to use the `.delete table policy streamingingestion` command to delete a table's streaming ingestion policy. ms.reviewer: yonil ms.topic: reference -ms.date: 12/18/2024 +ms.date: 12/22/2024 monikerRange: "azure-data-explorer" --- # .delete table policy streamingingestion command @@ -50,4 +50,4 @@ The following command deletes the streaming ingestion policy: * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) -* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) \ No newline at end of file +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) From d8e5d838581771779f1f9c9e064357c410d17be0 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Sun, 22 Dec 2024 13:26:58 +0200 Subject: [PATCH 07/19] edits --- .../delete-table-streaming-ingestion-policy-command.md | 3 +-- .../show-table-streaming-ingestion-policy-command.md | 6 +----- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md index 9f68fb52c4..c1afb7edbb 100644 --- a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md @@ -45,8 +45,7 @@ The following command deletes the streaming ingestion policy: ## Related content * [Streaming ingestion policy](streaming-ingestion-policy.md) -(show-database-streaming-ingestion-policy-command.md) -* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md)* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) diff --git a/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md index f875e4182f..401bc40a78 100644 --- a/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/show-table-streaming-ingestion-policy-command.md @@ -46,11 +46,7 @@ The following command displays the streaming ingestion policy: * [Streaming ingestion policy](streaming-ingestion-policy.md) * [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) -* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) -* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) -* [.alter-merge table policy streamingingestion command] -* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) -* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) * [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) +* [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) * [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) From 8bb52f93e8c0a83f73c19028941f6a2594ecd0f1 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Sun, 22 Dec 2024 13:36:21 +0200 Subject: [PATCH 08/19] edits --- .../delete-table-streaming-ingestion-policy-command.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md index c1afb7edbb..d4992c7f13 100644 --- a/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/delete-table-streaming-ingestion-policy-command.md @@ -45,7 +45,8 @@ The following command deletes the streaming ingestion policy: ## Related content * [Streaming ingestion policy](streaming-ingestion-policy.md) -* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md)* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) +* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) +* [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) * [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) * [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) From e5b484e71857bd8ff48f4fb876dbb2d11ef09664 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Sun, 22 Dec 2024 14:04:39 +0200 Subject: [PATCH 09/19] edits --- .../alter-table-streaming-ingestion-policy-command.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md index f494d7d816..a0f10934f1 100644 --- a/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-table-streaming-ingestion-policy-command.md @@ -51,8 +51,8 @@ The following command turns on streaming ingestion and determines the suggested * [Streaming ingestion policy](streaming-ingestion-policy.md) * [.show table policy streamingingestion command](show-table-streaming-ingestion-policy-command.md) -* [.show database policy streamingingestion command](show-database-streaming-ingestion-policy-command.md) -* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) -* [.alter-merge database policy streamingingestion command](alter-merge-database-streaming-ingestion-policy-command.md) -* [.delete database policy streamingingestion command](delete-database-streaming-ingestion-policy-command.md) +* [.alter table policy streamingingestion command](alter-table-streaming-ingestion-policy-command.md) +* [.alter-merge table policy streamingingestion command](alter-merge-table-streaming-ingestion-policy-command.md) * [.delete table policy streamingingestion command](delete-table-streaming-ingestion-policy-command.md) +* [.alter database policy streamingingestion command](alter-database-streaming-ingestion-policy-command.md) +* [Streaming ingestion and schema changes](data-ingestion/streaming-ingestion-schema-changes.md) From fea4e994a4c2b5d41d16de8fa3966f4f1f2a213d Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Mon, 23 Dec 2024 18:35:54 +0200 Subject: [PATCH 10/19] ai --- data-explorer/kusto/management/callout-policy.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/callout-policy.md b/data-explorer/kusto/management/callout-policy.md index 203328c607..e4c417ea38 100644 --- a/data-explorer/kusto/management/callout-policy.md +++ b/data-explorer/kusto/management/callout-policy.md @@ -3,7 +3,7 @@ title: Callout policy description: Learn how to update a cluster's callout policy to manage authorized domains for external calls. ms.reviewer: orspodek ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/23/2024 monikerRange: "azure-data-explorer" --- # Callout policy @@ -38,6 +38,7 @@ Callout policies are managed at cluster-level and are classified into the follow | sandbox_artifacts | Controls sandboxed plugins ([python](../query/python-plugin.md) and [R](../query/r-plugin.md)). | | external_data | Controls access to external data through [external tables](../query/schema-entities/external-tables.md) or [externaldata](../query/externaldata-operator.md) operator. | | webapi | Controls access to http endpoints. | +| ai_embed_text | Controls the [ai_embed_text plugin)](../query/ai-embed-text-plugin.md). | ## Predefined callout policies From 554da3bcfa5afeb9856689298ebcb0bd309633d8 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Wed, 1 Jan 2025 17:26:22 +0200 Subject: [PATCH 11/19] related content 3 workgroup, edits --- ...r-policy-request-classification-command.md | 62 +++++++++++-------- ...r-policy-request-classification-command.md | 54 +++++++++------- .../alter-merge-workload-group-command.md | 50 ++++++++------- .../create-or-alter-workload-group-command.md | 32 ++++++---- ...r-policy-request-classification-command.md | 53 +++++++++------- .../management/drop-workload-group-command.md | 11 +++- .../management/query-consistency-policy.md | 15 +++-- .../request-classification-policy.md | 55 ++++++++-------- .../kusto/management/request-limits-policy.md | 30 +++++---- .../management/request-queuing-policy.md | 7 ++- .../management/request-rate-limit-policy.md | 37 ++++++----- .../request-rate-limits-enforcement-policy.md | 16 ++--- ...r-policy-request-classification-command.md | 50 ++++++++------- .../management/show-workload-group-command.md | 14 ++++- 14 files changed, 287 insertions(+), 199 deletions(-) diff --git a/data-explorer/kusto/management/alter-cluster-policy-request-classification-command.md b/data-explorer/kusto/management/alter-cluster-policy-request-classification-command.md index ab3ae9b0ed..44b4d67a08 100644 --- a/data-explorer/kusto/management/alter-cluster-policy-request-classification-command.md +++ b/data-explorer/kusto/management/alter-cluster-policy-request-classification-command.md @@ -3,7 +3,7 @@ title: .alter cluster policy request_classification command description: Learn how to use the `.alter cluster policy request_classification` command to alter the request classification policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .alter cluster policy request_classification command @@ -13,7 +13,7 @@ ms.date: 08/11/2024 Alters the cluster's request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -Alters the eventhouse's request classification policy. +Alters the Eventhouse's request classification policy. ::: moniker-end ## Permissions @@ -33,10 +33,10 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co ## Parameters -| Name | Type | Required | Description | -|------------------------------|--------|----------|--------------------------------------------------------------------------------------------------------------------------------| -| *SerializedPartialPolicy* | `string` | :heavy_check_mark: | Define a serialized JSON policy. For policy properties, see [request classification policy](request-classification-policy.md). | -| *ClassificationFunctionBody* | `string` | :heavy_check_mark: | The body of the function to use for classifying requests. | +| Name | Type | Required | Description | +|--|--|--|--| +| *SerializedPartialPolicy* | `string` | :heavy_check_mark: | Define a serialized JSON policy. For policy properties, see [request classification policy](request-classification-policy.md). | +| *ClassificationFunctionBody* | `string` | :heavy_check_mark: | The body of the function to use for classifying requests. | ## Returns @@ -44,28 +44,28 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co The command returns one row showing the details of the cluster request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -The command returns one row showing the details of the eventhouse request classification policy. +The command returns one row showing the details of the Eventhouse request classification policy. ::: moniker-end Following is the schema of the output returned: :::moniker range="azure-data-explorer" -| Name | Type | Description | -|---------------|--------|--------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster request classification policy this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For cluster request classification policy this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For cluster request classification policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy, this value is an empty string. | ::: moniker-end :::moniker range="microsoft-fabric" -| Name | Type | Description | -|---------------|--------|--------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For eventhouse request classification policy this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For eventhouse request classification policy this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For eventhouse request classification policy this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For eventhouse request classification policy this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For Eventhouse request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For Eventhouse request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For Eventhouse request classification policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For Eventhouse request classification policy, this value is an empty string. | ::: moniker-end ## Examples @@ -87,9 +87,9 @@ Set a policy which, based on the evaluation of multiple conditions, assigns the **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|------------------------------------|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|------------| -| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_database", "request_description", "current_application", "current_principal", "request_type"], "IsEnabled": true, "ClassificationFunction": "case(current_principal_is_member_of('aadgroup=somesecuritygroup@contoso.com'), \"First workload group\", request_properties.current_database == \"MyDatabase\" and request_properties.current_principal has 'aadapp=', \"Second workload group\", request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\", \"Third workload group\", request_properties.current_application == \"KustoQueryRunner\", \"Fourth workload group\", request_properties.request_description == \"this is a test\", \"Fifth workload group\", hourofday(now()) between (17 .. 23), \"Sixth workload group\", \"default\")"} | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_database", "request_description", "current_application", "current_principal", "request_type"], "IsEnabled": true, "ClassificationFunction": "case(current_principal_is_member_of('aadgroup=somesecuritygroup@contoso.com'), \"First workload group\", request_properties.current_database == \"MyDatabase\" and request_properties.current_principal has 'aadapp=', \"Second workload group\", request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\", \"Third workload group\", request_properties.current_application == \"KustoQueryRunner\", \"Fourth workload group\", request_properties.request_description == \"this is a test\", \"Fifth workload group\", hourofday(now()) between (17 .. 23), \"Sixth workload group\", \"default\")"} | | | ### Set a policy with a single workload group @@ -104,6 +104,14 @@ Set a policy which, based on the evaluation of a composed condition, assigns the **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|------------------------------------|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|------------| -| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": true, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": true, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | | | + +## Related content + +* [Workload groups](workload-groups.md) +* [Request classification policy](request-classification-policy.md) +* [`.alter-merge cluster request classification policy`](alter-merge-cluster-policy-request-classification-command.md) +* [`.delete cluster request classification policy`](delete-cluster-policy-request-classification-command.md) +* [`.show cluster request classification policy`](show-cluster-policy-request-classification-command.md) diff --git a/data-explorer/kusto/management/alter-merge-cluster-policy-request-classification-command.md b/data-explorer/kusto/management/alter-merge-cluster-policy-request-classification-command.md index 9834fb579f..79026664ee 100644 --- a/data-explorer/kusto/management/alter-merge-cluster-policy-request-classification-command.md +++ b/data-explorer/kusto/management/alter-merge-cluster-policy-request-classification-command.md @@ -3,7 +3,7 @@ title: .alter-merge cluster policy request_classification command description: Learn how to use the `alter-merge cluster policy request_classification` command to enable or disable the cluster's request classification policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .alter-merge cluster policy request_classification command @@ -13,7 +13,7 @@ ms.date: 08/11/2024 Enables or disables the cluster's request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -Enables or disables the eventhouse's request classification policy. +Enables or disables the Eventhouse's request classification policy. ::: moniker-end ## Permissions @@ -37,28 +37,28 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co The command returns one row showing the details of the cluster request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -The command returns one row showing the details of the eventhouse request classification policy. +The command returns one row showing the details of the Eventhouse request classification policy. ::: moniker-end Following is the schema of the output returned: :::moniker range="azure-data-explorer" -| Name | Type | Description | -|---------------|--------|--------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster request classification policy this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For cluster request classification policy this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For cluster request classification policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy, this value is an empty string. | ::: moniker-end :::moniker range="microsoft-fabric" -| Name | Type | Description | -|---------------|--------|--------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For eventhouse request classification policy this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For eventhouse request classification policy this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For eventhouse request classification policy this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For eventhouse request classification policy this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For Eventhouse request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For Eventhouse request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For Eventhouse request classification policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For Eventhouse request classification policy, this value is an empty string. | ::: moniker-end ## Examples @@ -78,9 +78,9 @@ Enable request classification policy for the eventhouse: **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|------------------------------------|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|------------| -| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": true, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": true, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | | | ### Disable the policy @@ -97,6 +97,14 @@ Disable request classification policy for the eventhouse: **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|------------------------------------|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|------------| -| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": false, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": false, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | | | + +## Related content + +* [Workload groups](workload-groups.md) +* [Request classification policy](request-classification-policy.md) +* [`.alter cluster request classification policy`](alter-cluster-policy-request-classification-command.md) +* [`.delete cluster request classification policy`](delete-cluster-policy-request-classification-command.md) +* [`.show cluster request classification policy`](show-cluster-policy-request-classification-command.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/alter-merge-workload-group-command.md b/data-explorer/kusto/management/alter-merge-workload-group-command.md index 51cfc546ee..f45427e060 100644 --- a/data-explorer/kusto/management/alter-merge-workload-group-command.md +++ b/data-explorer/kusto/management/alter-merge-workload-group-command.md @@ -3,7 +3,7 @@ title: .alter-merge workload_group command description: Learn how to use the ".alter-merge workload_group" command to alter a workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .alter-merge workload_group command @@ -29,17 +29,18 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co ## Parameters -| Name | Type | Required | Description | -|----------------------------------|--------|----------|-------------------------------------------------------------------------------------------| -| *WorkloadGroupName* | `string` | :heavy_check_mark: | Name of the workload group. Can be specified with bracket notation ['WorkLoadGroupName']. | -| *SerializedPolicyObject* | `string` | :heavy_check_mark: | JSON representation of the policy. `*` | +| Name | Type | Required | Description | +|--|--|--|--| +| *WorkloadGroupName* | `string` | :heavy_check_mark: | Name of the workload group. Can be specified with bracket notation ['WorkLoadGroupName']. | +| *SerializedPolicyObject* | `string` | :heavy_check_mark: | JSON representation of the policy. `*` | `*` The following policies apply to workload groups: * [request classification](request-classification-policy.md) * [request limits](request-limits-policy.md) * [request rate limit](request-rate-limit-policy.md) -* [request rate limits enforcement](request-rate-limits-enforcement-policy.md). +* [request rate limits enforcement](request-rate-limits-enforcement-policy.md) +* [specify query consistency mode](query-consistency-policy.md) ## Returns @@ -76,9 +77,9 @@ Alter specific limits in the request limits policy of the `default` workload gro **Output** -| WorkloadGroupName | WorkloadGroup | -|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Database"}} | +| WorkloadGroupName | WorkloadGroup | +|--|--| +| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Database"}} | ### Alter the request rate limit policies @@ -102,9 +103,9 @@ Alter the request rate limit policies of the `default` workload group, while kee **Output** -| WorkloadGroupName | WorkloadGroup | -|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Database"}} | +| WorkloadGroupName | WorkloadGroup | +|--|--| +| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Database"}} | ### Alter the request queuing policy @@ -121,9 +122,9 @@ Turn on request queuing for the `default` workload group, while keeping its requ **Output** -| WorkloadGroupName | WorkloadGroup | -|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestQueuingPolicy":{"IsEnabled":true},"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":\n"QueryHead","CommandsEnforcementLevel":"Database"}} | +| WorkloadGroupName | WorkloadGroup | +|--|--| +| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestQueuingPolicy":{"IsEnabled":true},"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":\n"QueryHead","CommandsEnforcementLevel":"Database"}} | ### Alter the request rate limits enforcement policy @@ -141,9 +142,9 @@ Turn on request rate limits enforcement policy for the `default` workload group, **Output** -| WorkloadGroupName | WorkloadGroup | -|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestQueuingPolicy":{"IsEnabled":true},"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Cluster"}} | +| WorkloadGroupName | WorkloadGroup | +|--|--| +| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestQueuingPolicy":{"IsEnabled":true},"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Cluster"}} | ### Alter the query consistency policy @@ -163,6 +164,13 @@ Specify the applicable option for the query consistency model. **Output** -| WorkloadGroupName | WorkloadGroup | -|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestQueuingPolicy":{"IsEnabled":true},"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Cluster"},"QueryConsistencyPolicy":{"QueryConsistency":{"IsRelaxable":true,"Value":"Weak"}}} | +| WorkloadGroupName | WorkloadGroup | +|--|--| +| default | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":false,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":8589699072},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":false,"Value":"00:01:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}}],"RequestQueuingPolicy":{"IsEnabled":true},"RequestRateLimitsEnforcementPolicy":{"QueriesEnforcementLevel":"QueryHead","CommandsEnforcementLevel":"Cluster"},"QueryConsistencyPolicy":{"QueryConsistency":{"IsRelaxable":true,"Value":"Weak"}}} | + +## Related content + +* [Workload groups](workload-groups.md) +* [.drop workload_group](drop-workload-group-command.md) +* [.show workload_group](show-workload-group-command.md) +* [.create-or-alter workload_group](create-or-alter-workload-group-command.md) diff --git a/data-explorer/kusto/management/create-or-alter-workload-group-command.md b/data-explorer/kusto/management/create-or-alter-workload-group-command.md index 847ac7e45f..da0ef5003b 100644 --- a/data-explorer/kusto/management/create-or-alter-workload-group-command.md +++ b/data-explorer/kusto/management/create-or-alter-workload-group-command.md @@ -3,13 +3,13 @@ title: .create-or-alter workload_group command description: Learn how to use the `create-or-alter workload_group` command to create a new workload group or alter an existing workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .create-or-alter workload_group command > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] -Create a new workload group, or alters an existing workload group. +Creates a new workload group, or alters an existing workload group. ## Permissions @@ -28,17 +28,18 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co ## Parameters -| Name | Type | Required | Description | -|----------------------------------|--------|----------|-------------------------------------------------------------------------------------------| -| *WorkloadGroupName* | `string` | :heavy_check_mark: | Name of the workload group. Can be specified with bracket notation ['WorkLoadGroupName']. | -| *SerializedPolicyObject* | `string` | :heavy_check_mark: | JSON representation of the policy. `*` | +| Name | Type | Required | Description | +|--|--|--|--| +| *WorkloadGroupName* | `string` | :heavy_check_mark: | Name of the workload group. Can be specified with bracket notation ['WorkLoadGroupName']. | +| *SerializedPolicyObject* | `string` | :heavy_check_mark: | JSON representation of the policy. `*` | `*` The following policies apply to workload groups: * [request classification](request-classification-policy.md) * [request limits](request-limits-policy.md) * [request rate limit](request-rate-limit-policy.md) -* [request rate limits enforcement](request-rate-limits-enforcement-policy.md). +* [request rate limits enforcement](request-rate-limits-enforcement-policy.md) +* [specify query consistency mode](query-consistency-policy.md) ## Returns @@ -103,9 +104,9 @@ Create **MyWorkloadGroup** workload group with a full definition of its request **Output** -| WorkloadGroupName | WorkloadGroup | -|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| MyWorkloadGroup | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":true,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":false,"Value":6442450944},"MaxMemoryPerIterator":{"IsRelaxable":false,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":true,"Value":"00:04:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":20}}]} | +| WorkloadGroupName | WorkloadGroup | +|--|--| +| MyWorkloadGroup | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":true,"Value":"HotCache"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":false,"Value":6442450944},"MaxMemoryPerIterator":{"IsRelaxable":false,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":true,"Value":"00:04:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":20}}]} | ### Define request limits policy and request rate limits policies @@ -175,6 +176,13 @@ Create **My Workload Group** workload group with a full definition of its reques **Output** -| WorkloadGroupName | WorkloadGroup | -|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| WorkloadGroupName | WorkloadGroup | +|--|--| | My Workload Group | {"RequestLimitsPolicy":{"DataScope":{"IsRelaxable":true,"Value":"All"},"MaxMemoryPerQueryPerNode":{"IsRelaxable":true,"Value":6442450944},"MaxMemoryPerIterator":{"IsRelaxable":true,"Value":5368709120},"MaxFanoutThreadsPercentage":{"IsRelaxable":true,"Value":100},"MaxFanoutNodesPercentage":{"IsRelaxable":true,"Value":100},"MaxResultRecords":{"IsRelaxable":true,"Value":500000},"MaxResultBytes":{"IsRelaxable":true,"Value":67108864},"MaxExecutionTime":{"IsRelaxable":true,"Value":"00:04:00"}},"RequestRateLimitPolicies":[{"IsEnabled":true,"Scope":"WorkloadGroup","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":100}},{"IsEnabled":true,"Scope":"Principal","LimitKind":"ConcurrentRequests","Properties":{"MaxConcurrentRequests":25}}]} | + +## Related content + +* [Workload groups](workload-groups.md) +* [.show workload_group](show-workload-group-command.md) +* [.alter-merge workload_group](alter-merge-workload-group-command.md) +* [.drop workload_group](drop-workload-group-command.md) diff --git a/data-explorer/kusto/management/delete-cluster-policy-request-classification-command.md b/data-explorer/kusto/management/delete-cluster-policy-request-classification-command.md index 8153cecba4..32b6153d50 100644 --- a/data-explorer/kusto/management/delete-cluster-policy-request-classification-command.md +++ b/data-explorer/kusto/management/delete-cluster-policy-request-classification-command.md @@ -2,7 +2,7 @@ title: .delete cluster policy request_classification command description: Learn how to use the `.delete cluster policy request_classification` command to delete the request classification policy. ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .delete cluster policy request_classification command @@ -12,7 +12,7 @@ ms.date: 08/11/2024 Delete the cluster's request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -Delete the eventhouse's request classification policy. +Delete the Eventhouse's request classification policy. ::: moniker-end ## Permissions @@ -36,28 +36,28 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co The command returns one row showing the details of the cluster request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -The command returns one row showing the details of the eventhouse request classification policy. +The command returns one row showing the details of the Eventhouse request classification policy. ::: moniker-end Following is the schema of the output returned: :::moniker range="azure-data-explorer" -| Name | Type | Description | -|---------------|--------|--------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster request classification policy this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. This command sets this property to null. | -| ChildEntities | `string` | Child entities for which this policy is set. For cluster request classification policy this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. This command sets this property to null. | +| ChildEntities | `string` | Child entities for which this policy is set. For cluster request classification policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy, this value is an empty string. | ::: moniker-end :::moniker range="microsoft-fabric" -| Name | Type | Description | -|---------------|--------|--------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For eventhouse request classification policy this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For eventhouse request classification policy this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. This command sets this property to null. | -| ChildEntities | `string` | Child entities for which this policy is set. For eventhouse request classification policy this value is an empty string. | -| EntityType | `string` | Type of entity for which this policy is set. For eventhouse request classification policy this value is an empty string. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For Eventhouse request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For Eventhouse request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. This command sets this property to null. | +| ChildEntities | `string` | Child entities for which this policy is set. For Eventhouse request classification policy, this value is an empty string. | +| EntityType | `string` | Type of entity for which this policy is set. For Eventhouse request classification policy, this value is an empty string. | ::: moniker-end ## Examples @@ -68,16 +68,23 @@ Following is the schema of the output returned: Delete cluster request classification policy: ::: moniker-end :::moniker range="microsoft-fabric" -### Delete eventhouse request classification policy +### Delete Eventhouse request classification policy -Delete eventhouse request classification policy: +Delete Eventhouse request classification policy: ::: moniker-end - `.delete` `cluster` `policy` `request_classification` **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|------------------------------------|------------|--------|---------------|------------| -| ClusterRequestClassificationPolicy | | null | | | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| ClusterRequestClassificationPolicy | | null | | | + +## Related content + +* [Workload groups](workload-groups.md) +* [Request classification policy](request-classification-policy.md) +* [`.alter cluster request classification policy`](alter-cluster-policy-request-classification-command.md) +* [`.alter-merge cluster request classification policy`](alter-merge-cluster-policy-request-classification-command.md) +* [`.show cluster request classification policy`](show-cluster-policy-request-classification-command.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/drop-workload-group-command.md b/data-explorer/kusto/management/drop-workload-group-command.md index 6307d2a663..97703232c4 100644 --- a/data-explorer/kusto/management/drop-workload-group-command.md +++ b/data-explorer/kusto/management/drop-workload-group-command.md @@ -3,7 +3,7 @@ title: .drop workload_group command description: Learn how to use the `.drop workload_group` command to drop a workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .drop workload_group command @@ -38,7 +38,7 @@ You must have [Cluster AllDatabasesAdmin](../access-control/role-based-access-co The command returns all workload groups in the cluster, after the drop, which is the output of the [`.show workload_group`](show-workload-group-command.md#show-workload_group-command) command. ::: moniker-end :::moniker range="microsoft-fabric" -The command returns all workload groups in the eventhouse, after the drop, which is the output of the [`.show workload_group`](show-workload-group-command.md#show-workload_group-command) command. +The command returns all workload groups in the Eventhouse, after the drop, which is the output of the [`.show workload_group`](show-workload-group-command.md#show-workload_group-command) command. ::: moniker-end Following is the schema of the output returned: @@ -83,3 +83,10 @@ Drop `My Workload Group` workload group: ## Remarks The `internal` and `default` workload groups can't be dropped. An attempt to drop either of them aborts the command with an error indicating the reason. + +## Related content + +* [Workload groups](workload-groups.md) +* [.alter-merge workload_group](alter-merge-workload-group-command.md) +* [.show workload_group](show-workload-group-command.md) +* [.create-or-alter workload_group](create-or-alter-workload-group-command.md) diff --git a/data-explorer/kusto/management/query-consistency-policy.md b/data-explorer/kusto/management/query-consistency-policy.md index 4b8e6232db..50f6def385 100644 --- a/data-explorer/kusto/management/query-consistency-policy.md +++ b/data-explorer/kusto/management/query-consistency-policy.md @@ -3,7 +3,7 @@ title: Query consistency policy description: Learn how to use the query consistency policy to control the consistency mode of queries. ms.reviewer: yonil ms.topic: reference -ms.date: 11/20/2024 +ms.date: 01/01/2025 --- # Query consistency policy @@ -20,10 +20,10 @@ Each option consists of: The following limits are configurable: -| Name | Type | Description | Supported values | Default value | Matching client request property | -|------------------------|----------------------|--------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------|---------------|----------------------------------| -| QueryConsistency | `QueryConsistency` | The [consistency mode](../concepts/query-consistency.md) to use. | `Strong`, `Weak`, or `WeakAffinitizedByQuery`, `WeakAffinitizedByDatabase` | `Strong` | `queryconsistency` | -| CachedResultsMaxAge | `timespan` | The maximum age of [cached query results](../query/query-results-cache.md) that can be returned. | A non-negative `timespan` | `null` | `query_results_cache_max_age` | +| Name | Type | Description | Supported values | Default value | Matching client request property | +|--|--|--|--|--|--| +| QueryConsistency | `QueryConsistency` | The [consistency mode](../concepts/query-consistency.md) to use. | `Strong`, `Weak`, or `WeakAffinitizedByQuery`, `WeakAffinitizedByDatabase` | `Strong` | `queryconsistency` | +| CachedResultsMaxAge | `timespan` | The maximum age of [cached query results](../query/query-results-cache.md) that can be returned. | A non-negative `timespan` | `null` | `query_results_cache_max_age` | > [!NOTE] > The default value applies in the following cases: @@ -55,5 +55,10 @@ You can monitor the latency of the metadata snapshot age on nodes serving as wea ## Related content +* [Workload groups](workload-groups.md) +* [Query consistency](../concepts/query-consistency.md) * [.show workload_group command](show-workload-group-command.md) +* [.alter-merge workload_group](alter-merge-workload-group-command.md) * [.create-or-alter workload_group command](create-or-alter-workload-group-command.md) +* [.drop workload_group](drop-workload-group-command.md) + diff --git a/data-explorer/kusto/management/request-classification-policy.md b/data-explorer/kusto/management/request-classification-policy.md index fcd1371e92..f721ee731b 100644 --- a/data-explorer/kusto/management/request-classification-policy.md +++ b/data-explorer/kusto/management/request-classification-policy.md @@ -3,7 +3,7 @@ title: Request classification policy description: Learn how to use the request classification policy to assign incoming requests to a workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # Request classification policy @@ -13,7 +13,7 @@ ms.date: 08/11/2024 The classification process assigns incoming requests to a workload group, based on the characteristics of the requests. Tailor the classification logic by writing a user-defined function, as part of a cluster-level request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -The classification process assigns incoming requests to a workload group, based on the characteristics of the requests. Tailor the classification logic by writing a user-defined function, as part of a eventhouse-level request classification policy. +The classification process assigns incoming requests to a workload group, based on the characteristics of the requests. Tailor the classification logic by writing a user-defined function, as part of an Eventhouse-level request classification policy. ::: moniker-end In the absence of an enabled request classification policy, all requests are classified into the `default` workload group. @@ -34,20 +34,20 @@ The user-defined function has the following characteristics and behaviors: * If `IsEnabled` is set to `true` in the policy, the user-defined function is evaluated for every new request. * The user-defined function gives workload group context for the request for the full lifetime of the request. * The request is given the `default` workload group context in the following situations: - * The user-defined function returns an empty string, `default`, or the name of non-existent workload group. + * The user-defined function returns an empty string, `default`, or the name of nonexistent workload group. * The function fails for any reason. * Only one user-defined function can be designated at any given time. :::moniker range="azure-data-explorer" > [!IMPORTANT] -> The request classification function will be evaluated for each request that runs on the cluster. -> It is recommended to keep it as lightweight as possible, and not include heavy computations in it. +> The request classification function is evaluated for each request that runs on the cluster. +> It's recommended to keep it as lightweight as possible, and not include heavy computations in it. > For example, avoid having to evaluate many regular expressions as part of its execution. ::: moniker-end :::moniker range="microsoft-fabric" > [!IMPORTANT] -> The request classification function will be evaluated for each request that runs on the eventhouse. -> It is recommended to keep it as lightweight as possible, and not include heavy computations in it. +> The request classification function is evaluated for each request that runs on the Eventhouse. +> It's recommended to keep it as lightweight as possible, and not include heavy computations in it. > For example, avoid having to evaluate many regular expressions as part of its execution. ::: moniker-end @@ -55,9 +55,9 @@ The user-defined function has the following characteristics and behaviors: A classification function: -* Must return a single scalar value of type `string`, that is the name of the workload group to assign the request to. +* Must return a single scalar value of type `string`. That is the name of the workload group to assign the request to. * Must not reference any other entity (database, table, or function). - * Specifically - it may not use the following functions and operators: + * Specifically - it might not use the following functions and operators: * `cluster()` * `database()` * `table()` @@ -65,18 +65,18 @@ A classification function: * `externaldata` * Has access to a special `dynamic` symbol, a property-bag named `request_properties`, with the following properties: - | Name | Type | Description | Examples | - |-----------------------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------| - | `current_database` | `string` | The name of the request database. | `"MyDatabase"` | - | `current_application` | `string` | The name of the application that sent the request. | `"Kusto.Explorer"`, `"KusWeb"` | - | `current_principal` | `string` | The fully qualified name of the principal identity that sent the request. | `"aaduser=1793eb1f-4a18-418c-be4c-728e310c86d3;83af1c0e-8c6d-4f09-b249-c67a2e8fda65"` | - | `query_consistency` | `string` | For queries: the consistency of the query - `strongconsistency` or `weakconsistency`. This property can be set by the caller as part of the request's [request properties](../api/rest/request-properties.md): The client request property to set is: `queryconsistency`. | `"strongconsistency"`, `"weakconsistency"` | - | `request_description` | `string` | Custom text that the author of the request can include. The text can be set by the caller as part of the request's [Client request properties](../api/rest/request-properties.md): The client request property to set is: `request_description`. | `"Some custom description"`; automatically populated for dashboards: `"dashboard:{dashboard_id};version:{version};sourceId:{source_id};sourceType:{tile/parameter}"` | - | `request_text` | `string` | The obfuscated text of the request. Obfuscated string literals included in the query text are replaced by multiple of star (`*`) characters. **Note:** only the leading 65,536 characters of the request text are evaluated. | `".show version"` | - | `request_type` | `string` | The type of the request - `Command` or `Query`. | `"Command"`, `"Query"` | +| | Name | Type | Description | Examples | +|--|--|--|--|--| +| | `current_database` | `string` | The name of the request database. | `"MyDatabase"` | +| | `current_application` | `string` | The name of the application that sent the request. | `"Kusto.Explorer"`, `"KusWeb"` | +| | `current_principal` | `string` | The fully qualified name of the principal identity that sent the request. | `"aaduser=1793eb1f-4a18-418c-be4c-728e310c86d3;83af1c0e-8c6d-4f09-b249-c67a2e8fda65"` | +| | `query_consistency` | `string` | For queries: the consistency of the query - `strongconsistency` or `weakconsistency`. This property is set by the caller as part of the request's [request properties](../api/rest/request-properties.md): The client request property to set is: `queryconsistency`. | `"strongconsistency"`, `"weakconsistency"` | +| | `request_description` | `string` | Custom text that the author of the request can include. The text is set by the caller as part of the request's [Client request properties](../api/rest/request-properties.md): The client request property to set is: `request_description`. | `"Some custom description"`; automatically populated for dashboards: `"dashboard:{dashboard_id};version:{version};sourceId:{source_id};sourceType:{tile/parameter}"` | +| | `request_text` | `string` | The obfuscated text of the request. Obfuscated string literals included in the query text are replaced by multiple of star (`*`) characters. **Note:** only the leading 65,536 characters of the request text are evaluated. | `".show version"` | +| | `request_type` | `string` | The type of the request - `Command` or `Query`. | `"Command"`, `"Query"` | > [!NOTE] -> Setting the aforementioned request options using `set` statements does not allow classification by them. The options need to be set in the client request properties object. +> Setting the request options using `set` statements doesn't allow classification by them. The options need to be set in the client request properties object. ### Examples @@ -114,12 +114,17 @@ Use the following management commands to manage a cluster's request classificati | [`.show cluster request classification policy`](show-cluster-policy-request-classification-command.md) | Shows the cluster's request classification policy | ::: moniker-end :::moniker range="microsoft-fabric" -Use the following management commands to manage an eventhouse's request classification policy. +Use the following management commands to manage an Eventhouse's request classification policy. | Command | Description | |--|--| -| [`.alter cluster request classification policy`](alter-cluster-policy-request-classification-command.md) | Alters eventhouse's request classification policy | -| [`.alter-merge cluster request classification policy`](alter-merge-cluster-policy-request-classification-command.md) | Enables or disables a eventhouse's request classification policy | -| [`.delete cluster request classification policy`](delete-cluster-policy-request-classification-command.md) | Deletes the eventhouse's request classification policy | -| [`.show cluster request classification policy`](show-cluster-policy-request-classification-command.md) | Shows the cluster's request classification policy | -::: moniker-end \ No newline at end of file +| [`.alter cluster request classification policy`](alter-cluster-policy-request-classification-command.md) | Alters Eventhouse's request classification policy | +| [`.alter-merge cluster request classification policy`](alter-merge-cluster-policy-request-classification-command.md) | Enables or disables an Eventhouse's request classification policy | +| [`.delete cluster request classification policy`](delete-cluster-policy-request-classification-command.md) | Deletes the Eventhouse's request classification policy | +| [`.show cluster request classification policy`](show-cluster-policy-request-classification-command.md) | Shows the Eventhouse's request classification policy | +::: moniker-end + +## Related content + +* [Workload groups](workload-groups.md) +* [Request properties](../api/rest/request-properties.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/request-limits-policy.md b/data-explorer/kusto/management/request-limits-policy.md index 878d8ad5e1..28874aa2d1 100644 --- a/data-explorer/kusto/management/request-limits-policy.md +++ b/data-explorer/kusto/management/request-limits-policy.md @@ -3,7 +3,7 @@ title: Request limits policy description: Learn how to use the request limits policy to limit the resources used by the request during its execution. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # Request limits policy @@ -28,9 +28,9 @@ The following limits are configurable: | MaxMemoryPerIterator | `long` | The maximum amount of memory (in bytes) a [query operator](../concepts/query-limits.md#limit-on-memory-consumed-by-query-operators-e_runaway_query) can allocate. | [`1`, *50% of a single node's total RAM*] | `maxmemoryconsumptionperiterator` | | MaxFanoutThreadsPercentage | `int` | The percentage of threads on each node to fan out query execution to. When set to 100%, the cluster assigns all CPUs on each node. For example, 16 CPUs on a cluster deployed on Azure D14_v2 nodes. | [`1`, `100`] | `query_fanout_threads_percent` | | MaxFanoutNodesPercentage | `int` | The percentage of nodes on the cluster to fan out query execution to. Functions in a similar manner to `MaxFanoutThreadsPercentage`. | [`1`, `100`] | `query_fanout_nodes_percent` | -| MaxResultRecords | `long` | The maximum number of records a request is allowed to return to the caller, above which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as those that result from having cross-cluster references. | [`1`, `9223372036854775807`] | `truncationmaxrecords` | -| MaxResultBytes | `long` | The maximum data size (in bytes) a request is allowed to return to the caller, above which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as those that result from having cross-cluster references.| [`1`, `9223372036854775807`] | `truncationmaxsize` | -| MaxExecutionTime | `timespan` | The maximum duration of a request.
**Notes:**
1) This can be used to place more limits on top of the [*default* limits on execution time](../concepts/query-limits.md#limit-execution-timeout), but not extend them.
2) Timeout processing isn't at the resolution of *seconds*, rather it's designed to prevent a query from running for *minutes*.
3) The time it takes to read the payload back at the client isn't treated as part of the timeout. It depends on how quickly the caller pulls the data from the stream.
4) Total execution time may exceed the configured value if aborting execution takes longer to complete. | [`00:00:00`, `01:00:00`] | `servertimeout` | +| MaxResultRecords | `long` | The maximum number of records a request is allowed to return to the caller, beyond which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as those that result from having cross-cluster references. | [`1`, `9223372036854775807`] | `truncationmaxrecords` | +| MaxResultBytes | `long` | The maximum data size (in bytes) a request is allowed to return to the caller, beyond which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as those that result from having cross-cluster references.| [`1`, `9223372036854775807`] | `truncationmaxsize` | +| MaxExecutionTime | `timespan` | The maximum duration of a request.
**Notes:**
1) This can be used to place more limits on top of the [*default* limits on execution time](../concepts/query-limits.md#limit-execution-timeout), but not extend them.
2) Timeout processing isn't at the resolution of *seconds*, rather it's designed to prevent a query from running for *minutes*.
3) The time it takes to read the payload back at the client isn't treated as part of the timeout. It depends on how quickly the caller pulls the data from the stream.
4) Total execution time can exceed the configured value if aborting execution takes longer to complete. | [`00:00:00`, `01:00:00`] | `servertimeout` | ::: moniker-end :::moniker range="microsoft-fabric" | Property | Type | Description | Supported values | Matching client request property | @@ -38,11 +38,11 @@ The following limits are configurable: | DataScope | `string` | The query's data scope. This value determines whether the query applies to all data or just the hot cache. | `All`, `HotCache`, or `null` | `query_datascope` | | MaxMemoryPerQueryPerNode | `long` | The maximum amount of memory (in bytes) a query can allocate. | [`1`, *50% of a single node's total RAM*] | `max_memory_consumption_per_query_per_node` | | MaxMemoryPerIterator | `long` | The maximum amount of memory (in bytes) a [query operator](../concepts/query-limits.md#limit-on-memory-consumed-by-query-operators-e_runaway_query) can allocate. | [`1`, *50% of a single node's total RAM*] | `maxmemoryconsumptionperiterator` | -| MaxFanoutThreadsPercentage | `int` | The percentage of threads on each node to fan out query execution to. When set to 100%, the eventhouse assigns all CPUs on each node. For example, 16 CPUs on a eventhouse deployed on Azure D14_v2 nodes. | [`1`, `100`] | `query_fanout_threads_percent` | -| MaxFanoutNodesPercentage | `int` | The percentage of nodes on the eventhouse to fan out query execution to. Functions in a similar manner to `MaxFanoutThreadsPercentage`. | [`1`, `100`] | `query_fanout_nodes_percent` | -| MaxResultRecords | `long` | The maximum number of records a request is allowed to return to the caller, above which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as those that result from having cross-eventhouse references. | [`1`, `9223372036854775807`] | `truncationmaxrecords` | -| MaxResultBytes | `long` | The maximum data size (in bytes) a request is allowed to return to the caller, above which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as those that result from having cross-eventhouse references.| [`1`, `9223372036854775807`] | `truncationmaxsize` | -| MaxExecutionTime | `timespan` | The maximum duration of a request.
**Notes:**
1) This can be used to place more limits on top of the [*default* limits on execution time](../concepts/query-limits.md#limit-execution-timeout), but not extend them.
2) Timeout processing isn't at the resolution of *seconds*, rather it's designed to prevent a query from running for *minutes*.
3) The time it takes to read the payload back at the client isn't treated as part of the timeout. It depends on how quickly the caller pulls the data from the stream.
4) Total execution time may exceed the configured value if aborting execution takes longer to complete. | [`00:00:00`, `01:00:00`] | `servertimeout` | +| MaxFanoutThreadsPercentage | `int` | The percentage of threads on each node to fan out query execution to. When set to 100%, the Eventhouse assigns all CPUs on each node. For example, 16 CPUs on an eventhouse deployed on Azure D14_v2 nodes. | [`1`, `100`] | `query_fanout_threads_percent` | +| MaxFanoutNodesPercentage | `int` | The percentage of nodes on the Eventhouse to fan out query execution to. Functions in a similar manner to `MaxFanoutThreadsPercentage`. | [`1`, `100`] | `query_fanout_nodes_percent` | +| MaxResultRecords | `long` | The maximum number of records a request is allowed to return to the caller, beyond which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as the results from having cross-eventhouse references. | [`1`, `9223372036854775807`] | `truncationmaxrecords` | +| MaxResultBytes | `long` | The maximum data size (in bytes) a request is allowed to return to the caller, beyond which the results are truncated. The truncation limit affects the final result of the query, as delivered back to the client. However, the truncation limit doesn't apply to intermediate results of subqueries, such as results from having cross-eventhouse references.| [`1`, `9223372036854775807`] | `truncationmaxsize` | +| MaxExecutionTime | `timespan` | The maximum duration of a request.
**Notes:**
1) This can be used to place more limits on top of the [*default* limits on execution time](../concepts/query-limits.md#limit-execution-timeout), but not extend them.
2) Timeout processing isn't at the resolution of *seconds*, rather it's designed to prevent a query from running for *minutes*.
3) The time it takes to read the payload back at the client isn't treated as part of the timeout. It depends on how quickly the caller pulls the data from the stream.
4) Total execution time might exceed the configured value if aborting execution takes longer to complete. | [`00:00:00`, `01:00:00`] | `servertimeout` | ::: moniker-end > [!NOTE] @@ -54,10 +54,10 @@ The following limits are configurable: Queries can use all the CPU resources within the cluster. By default, when multiple queries are running concurrently, the system employs a fair round-robin approach to distribute resources. This strategy is optimal for achieving high performance with ad-hoc queries. ::: moniker-end :::moniker range="microsoft-fabric" -Queries can use all the CPU resources within the eventhouse. By default, when multiple queries are running concurrently, the system employs a fair round-robin approach to distribute resources. This strategy is optimal for achieving high performance with ad-hoc queries. +Queries can use all the CPU resources within the Eventhouse. By default, when multiple queries are running concurrently, the system employs a fair round-robin approach to distribute resources. This strategy is optimal for achieving high performance with ad-hoc queries. ::: moniker-end -However, there are scenarios where you might want to restrict the CPU resources allocated to a specific query. For instance, if you are running a background job that can accommodate higher latencies. The request limits policy provides the flexibility to specify a lower percentage of threads or nodes to be used when executing distributed sub-query operations. The default setting is 100%. +However, there are scenarios where you might want to restrict the CPU resources allocated to a specific query. For instance, if you're running a background job that can accommodate higher latencies. The request limits policy provides the flexibility to specify a lower percentage of threads or nodes to be used when executing distributed subquery operations. The default setting is 100%. ## The `default` workload group @@ -99,11 +99,11 @@ The `default` workload group has the following policy defined by default. This p } } ``` + > [!NOTE] > * Limits in the `default` workload group must be defined and have a non-`null` value. > * All limits in the `default` workload group have `IsRelaxable` set to `true`. -> * Request limits are turned off for specific command types within the `default` workload group, such as `.export` commands and [ingest from query](data-ingestion/ingest-from-query.md) commands like `.set-or-append` and `.set-or-replace`. When these commands are assigned to a non-default workload group, the request limits specified in the policy become applicable. - +> * Request limits are turned off for specific command types within the `default` workload group, such as `.export` commands and [ingest from query](data-ingestion/ingest-from-query.md) commands like `.set-or-append` and `.set-or-replace`. When these commands are assigned to a nondefault workload group, the request limits specified in the policy become applicable. ## Example @@ -148,5 +148,9 @@ The following JSON represents a custom requests limits policy object: ## Related content +* [Workload groups](workload-groups.md) * [Client request properties](../api/rest/request-properties.md) * [.show workload_group command](show-workload-group-command.md) +* [.drop workload_group](drop-workload-group-command.md) +* [.alter-merge workload_group](alter-merge-workload-group-command.md) +* [.create-or-alter workload_group](create-or-alter-workload-group-command.md) diff --git a/data-explorer/kusto/management/request-queuing-policy.md b/data-explorer/kusto/management/request-queuing-policy.md index dd66cd9a4d..b6b9830c5c 100644 --- a/data-explorer/kusto/management/request-queuing-policy.md +++ b/data-explorer/kusto/management/request-queuing-policy.md @@ -3,7 +3,7 @@ title: Request queuing policy description: Learn how to use the request queuing policy to control queuing of requests for delayed execution. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/25/2024 --- # Request queuing policy (Preview) @@ -13,9 +13,9 @@ A workload group's request queuing policy controls queuing of requests for delay Queuing of requests can reduce the number of throttling errors during times of peak activity. It does so by queuing incoming requests up to a predefined short time period, while polling for available capacity during that time period. -The policy may be defined only for workload groups with a [request rate limit policy](request-rate-limit-policy.md) that limits the max concurrent requests at the scope of the workload group. +The policy might be defined only for workload groups with a [request rate limit policy](request-rate-limit-policy.md) that limits the max concurrent requests at the scope of the workload group. -Use the [.alter-merge workload group](alter-merge-workload-group-command.md#alter-the-request-queuing-policy) management command to enable request queuing. +Use the [.alter-merge workload group](alter-merge-workload-group-command.md#alter-the-request-queuing-policy) management command, to enable request queuing. ## The policy object @@ -25,5 +25,6 @@ The policy includes a single property: ## Related content +* [Workload groups](workload-groups.md) * [.show workload_group command](show-workload-group-command.md) * [.create-or-alter workload_group command](create-or-alter-workload-group-command.md) diff --git a/data-explorer/kusto/management/request-rate-limit-policy.md b/data-explorer/kusto/management/request-rate-limit-policy.md index 854b644acf..6337a042b4 100644 --- a/data-explorer/kusto/management/request-rate-limit-policy.md +++ b/data-explorer/kusto/management/request-rate-limit-policy.md @@ -3,7 +3,7 @@ title: Request rate limit policy description: Learn how to use the request rate limit policy to limit the number of concurrent requests classified into a workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/25/2024 --- # Request rate limit policy @@ -17,20 +17,20 @@ Rate limits are enforced at the level defined by the workload group's [Request r A request rate limit policy has the following properties: -| Name | Supported values | Description | -|------------|---------------------------------------------|--------------------------------------------| -| IsEnabled | `true`, `false` | Indicates if the policy is enabled or not. | -| Scope | `WorkloadGroup`, `Principal` | The scope to which the limit applies. | -| LimitKind | `ConcurrentRequests`, `ResourceUtilization` | The kind of the request rate limit. | -| Properties | Property bag | Properties of the request rate limit. | +| Name | Supported values | Description | +|--|--|--| +| IsEnabled | `true`, `false` | Indicates if the policy is enabled or not. | +| Scope | `WorkloadGroup`, `Principal` | The scope to which the limit applies. | +| LimitKind | `ConcurrentRequests`, `ResourceUtilization` | The kind of the request rate limit. | +| Properties | Property bag | Properties of the request rate limit. | ### Concurrent requests rate limit A request rate limit of kind `ConcurrentRequests` includes the following property: -| Name | Type | Description | Supported Values | -|-----------------------|------|--------------------------------------------|------------------| -| MaxConcurrentRequests | `int` | The maximum number of concurrent requests. | [`0`, `10000`] | +| Name | Type | Description | Supported Values | +|--|--|--|--| +| MaxConcurrentRequests | `int` | The maximum number of concurrent requests. | [`0`, `10000`] | > [!NOTE] > @@ -61,11 +61,11 @@ The following table shows a few examples of concurrent requests that exceed the A request rate limit of kind `ResourceUtilization` includes the following properties: -| Name | Type | Description | Supported Values | -|----------------|----------------|----------------|--------------| -| ResourceKind | `ResourceKind` | The resource to limit.

When `ResourceKind` is `TotalCpuSeconds`, the limit is enforced based on post-execution reports of CPU utilization of completed requests. Requests that report utilization of 0.005 seconds of CPU or lower aren't counted. The limit (`MaxUtilization`) represents the total CPU seconds that can be consumed by requests within a specified time window (`TimeWindow`). For example, a user running ad-hoc queries may have a limit of 1000 CPU seconds per hour. If this limit is exceeded, subsequent queries will be throttled, even if started concurrently, as the cumulative CPU seconds have surpassed the defined limit within the sliding window period. | `RequestCount`, `TotalCpuSeconds` | -| MaxUtilization | `long` | The maximum of the resource that can be utilized. | RequestCount: [`1`, `16777215`]; TotalCpuSeconds: [`1`, `828000`] | -| TimeWindow | `timespan` | The sliding time window during which the limit is applied. | [`00:01:00`, `1.00:00:00`] | +| Name | Type | Description | Supported Values | +|--|--|--|--| +| ResourceKind | `ResourceKind` | The resource to limit.

When `ResourceKind` is `TotalCpuSeconds`, the limit is enforced based on post-execution reports of CPU utilization of completed requests. Requests that report utilization of 0.005 seconds of CPU or lower aren't counted. The limit (`MaxUtilization`) represents the total CPU seconds that can be consumed by requests within a specified time window (`TimeWindow`). For example, a user running ad-hoc queries may have a limit of 1000 CPU seconds per hour. If this limit is exceeded, subsequent queries will be throttled, even if started concurrently, as the cumulative CPU seconds have surpassed the defined limit within the sliding window period. | `RequestCount`, `TotalCpuSeconds` | +| MaxUtilization | `long` | The maximum of the resource that can be utilized. | RequestCount: [`1`, `16777215`]; TotalCpuSeconds: [`1`, `828000`] | +| TimeWindow | `timespan` | The sliding time window during which the limit is applied. | [`00:01:00`, `1.00:00:00`] | When a request exceeds the limit on resources utilization: @@ -75,7 +75,7 @@ When a request exceeds the limit on resources utilization: The following table shows a few examples of requests that exceed the resource utilization rate limit and the error message that these requests return: | Scenario | Error message | -|----------------|----------------| +|--|--| | A throttled request that was classified to a workload group named `Automated Requests`, which has a limit of 1000 requests per hour at the scope of a principal. | The request was denied due to exceeding quota limitations. Resource: 'RequestCount', Quota: '1000', TimeWindow: '01:00:00', Origin: 'RequestRateLimitPolicy/WorkloadGroup/Automated Requests/Principal/aadapp=9e04c4f5-1abd-48d4-a3d2-9f58615b4724;6ccf3fe8-6343-4be5-96c3-29a128dd9570'. | | A throttled request, that was classified to a workload group named `Automated Requests`, which has a limit of 2000 total CPU seconds per hour at the scope of the workload group. | The request was denied due to exceeding quota limitations. Resource: 'TotalCpuSeconds', Quota: '2000', TimeWindow: '01:00:00', Origin: 'RequestRateLimitPolicy/WorkloadGroup/Automated Requests'. | @@ -173,4 +173,9 @@ The following policies will block all requests classified to the workload group: ## Related content +* [Workload groups](workload-groups.md) +* [Request rate limits enforcement policy](request-rate-limits-enforcement-policy.md) * [.show workload_group command](show-workload-group-command.md) +* [.drop workload_group command](drop-workload-group-command.md) +* [.alter-merge workload_group command](alter-merge-workload-group-command.md) +* [.create-or-alter workload_group command](create-or-alter-workload-group-command.md) diff --git a/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md b/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md index d6c8455568..596958d405 100644 --- a/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md +++ b/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md @@ -3,7 +3,7 @@ title: Request rate limits enforcement policy description: Learn how to use the request rate limits enforcement policy to enforce request rate limits. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/25/2024 --- # Request rate limits enforcement policy @@ -15,17 +15,17 @@ A workload group's request rate limits enforcement policy controls how request r A request rate limit policy has the following properties: -| Name | Supported values | Default value | Description | -|--------------------------|---------------------------------------------|----------------|-----------------------------------------------| -| QueriesEnforcementLevel | `Cluster`, `QueryHead` | `QueryHead` | Indicates the enforcement level for queries. | -| CommandsEnforcementLevel | `Cluster`, `Database` | `Database` | Indicates the enforcement level for commands. | - +| Name | Supported values | Default value | Description | +|--|--|--|--| +| QueriesEnforcementLevel | `Cluster`, `QueryHead` | `QueryHead` | Indicates the enforcement level for queries. | +| CommandsEnforcementLevel | `Cluster`, `Database` | `Database` | Indicates the enforcement level for commands. | ### Request rate limits enforcement level Request rate limits can be enforced at one of the following levels: :::moniker range="azure-data-explorer" + * `Cluster`: * Rate limits are enforced by the single cluster admin node. * `Database`: @@ -45,7 +45,7 @@ Request rate limits can be enforced at one of the following levels: ::: moniker-end :::moniker range="microsoft-fabric" * `Cluster`: - * Rate limits are enforced by the single eventhouse admin node. + * Rate limits are enforced by the single Eventhouse admin node. * `Database`: * Rate limits are enforced by the database admin node that manages the database the request was sent to. * If there are multiple database admin nodes, the configured rate limit is effectively multiplied by the number of database admin nodes. @@ -111,5 +111,7 @@ The effective rate limits for the `default` workload group are: ## Related content +* [Workload groups](workload-groups.md) * [.show workload_group command](show-workload-group-command.md) +* [.alter-merge workload_group command](alter-merge-workload-group-command.md) * [System information](system-info.md) diff --git a/data-explorer/kusto/management/show-cluster-policy-request-classification-command.md b/data-explorer/kusto/management/show-cluster-policy-request-classification-command.md index 9079e9e741..1f6a0a7c9e 100644 --- a/data-explorer/kusto/management/show-cluster-policy-request-classification-command.md +++ b/data-explorer/kusto/management/show-cluster-policy-request-classification-command.md @@ -3,7 +3,7 @@ title: .show cluster policy request_classification command description: Learn how to use the `.show cluster policy request_classification` command to show the request classification policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .show cluster policy request_classification command @@ -13,7 +13,7 @@ ms.date: 08/11/2024 Shows the cluster's request classification policy. ::: moniker-end :::moniker range="microsoft-fabric" -Shows the eventhouse's request classification policy. +Shows the Eventhouse's request classification policy. ::: moniker-end ## Permissions @@ -38,26 +38,26 @@ The command returns one row showing the details of the cluster request classific Following is the schema of the output returned: -| Name | Type | Description | -|---------------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For cluster request classification policy, this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy, this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For cluster request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For cluster request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | | ChildEntities | `string` | Child entities for which this policy is set. For cluster request classification policy, this value is an array of strings, each of which corresponds to the name of system and user databases attached to the cluster. | -| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy, this value is **Cluster**. | +| EntityType | `string` | Type of entity for which this policy is set. For cluster request classification policy, this value is **Cluster**. | ::: moniker-end :::moniker range="microsoft-fabric" -The command returns one row showing the details of the eventhouse request classification policy. +The command returns one row showing the details of the Eventhouse request classification policy. Following is the schema of the output returned: -| Name | Type | Description | -|---------------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| PolicyName | `string` | Name of the policy. For eventhouse request classification policy, this value is **ClusterRequestClassificationPolicy**. | -| EntityName | `string` | Name of the entity for which the policy is set. For eventhouse request classification policy, this value is an empty string. | -| Policy | `string` | JSON representation of the policy object. | -| ChildEntities | `string` | Child entities for which this policy is set. For eventhouse request classification policy, this value is an array of strings, each of which corresponds to the name of system and user databases attached to the eventhouse. | -| EntityType | `string` | Type of entity for which this policy is set. For eventhouse request classification policy, this value is **Cluster**. | +| Name | Type | Description | +|--|--|--| +| PolicyName | `string` | Name of the policy. For Eventhouse request classification policy, this value is **ClusterRequestClassificationPolicy**. | +| EntityName | `string` | Name of the entity for which the policy is set. For Eventhouse request classification policy, this value is an empty string. | +| Policy | `string` | JSON representation of the policy object. | +| ChildEntities | `string` | Child entities for which this policy is set. For Eventhouse request classification policy, this value is an array of strings, each of which corresponds to the name of system and user databases attached to the Eventhouse. | +| EntityType | `string` | Type of entity for which this policy is set. For Eventhouse request classification policy, this value is **Cluster**. | ::: moniker-end ## Examples @@ -68,9 +68,9 @@ Following is the schema of the output returned: Display request classification policy for the cluster: ::: moniker-end :::moniker range="microsoft-fabric" -### Display the eventhouse's request classification policy +### Display the Eventhouse's request classification policy -Display request classification policy for the eventhouse: +Display request classification policy for the Eventhouse: ::: moniker-end ```kusto @@ -79,6 +79,14 @@ Display request classification policy for the eventhouse: **Output** -| PolicyName | EntityName | Policy | ChildEntities | EntityType | -|------------------------------------|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------|------------| -| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": true, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | ["$systemdb", "KustoMonitoringPersistentDatabase", "YourDatabaseName"] | Cluster | +| PolicyName | EntityName | Policy | ChildEntities | EntityType | +|--|--|--|--|--| +| ClusterRequestClassificationPolicy | | {"ClassificationProperties": ["current_application", "request_type" ],"IsEnabled": true, "ClassificationFunction": "iff(request_properties.current_application == \"Kusto.Explorer\" and request_properties.request_type == \"Query\",\"Ad-hoc queries\",\"default\")"} | ["$systemdb", "KustoMonitoringPersistentDatabase", "YourDatabaseName"] | Cluster | + +## Related content + +* [Workload groups](workload-groups.md) +* [Request classification policy](request-classification-policy.md) +* [`.alter cluster request classification policy`](alter-cluster-policy-request-classification-command.md) +* [`.alter-merge cluster request classification policy`](alter-merge-cluster-policy-request-classification-command.md) +* [`.delete cluster request classification policy`](delete-cluster-policy-request-classification-command.md) \ No newline at end of file diff --git a/data-explorer/kusto/management/show-workload-group-command.md b/data-explorer/kusto/management/show-workload-group-command.md index 80ebfedcc8..82e3bcd135 100644 --- a/data-explorer/kusto/management/show-workload-group-command.md +++ b/data-explorer/kusto/management/show-workload-group-command.md @@ -3,7 +3,7 @@ title: .show workload_group command description: Learn how to use the `.show workload_group` command to show the specified workload group definitions. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/01/2025 --- # .show workload_group command @@ -49,3 +49,15 @@ Show definition for **MyWorkloadGroup** workload group: | WorkloadGroupName | WorkloadGroup | |--|--| | MyWorkloadGroup | {"RequestRateLimitPolicies": [{"IsEnabled": true, "Scope": "WorkloadGroup", "LimitKind": "ConcurrentRequests", "Properties": {"MaxConcurrentRequests": 30}}]} | + +## Related content + +* [Workload groups](workload-groups.md) +* [Request limits policy](request-limits-policy.md) +* [Request rate limit policy](request-rate-limit-policy.md) +* [Request rate limits enforcement policy](request-rate-limits-enforcement-policy.md) +* [Request queuing policy](request-queuing-policy.md) +* [Query consistency policy](query-consistency-policy.md) +* [.drop workload_group](drop-workload-group-command.md) +* [.alter-merge workload_group](alter-merge-workload-group-command.md) +* [.create-or-alter workload_group](create-or-alter-workload-group-command.md) From 6bb333ad119e83e006dfa0cf507d62b787623345 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Wed, 1 Jan 2025 17:40:41 +0200 Subject: [PATCH 12/19] edits --- .../request-classification-policy.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/data-explorer/kusto/management/request-classification-policy.md b/data-explorer/kusto/management/request-classification-policy.md index f721ee731b..cd2fbe01a8 100644 --- a/data-explorer/kusto/management/request-classification-policy.md +++ b/data-explorer/kusto/management/request-classification-policy.md @@ -65,15 +65,15 @@ A classification function: * `externaldata` * Has access to a special `dynamic` symbol, a property-bag named `request_properties`, with the following properties: -| | Name | Type | Description | Examples | -|--|--|--|--|--| -| | `current_database` | `string` | The name of the request database. | `"MyDatabase"` | -| | `current_application` | `string` | The name of the application that sent the request. | `"Kusto.Explorer"`, `"KusWeb"` | -| | `current_principal` | `string` | The fully qualified name of the principal identity that sent the request. | `"aaduser=1793eb1f-4a18-418c-be4c-728e310c86d3;83af1c0e-8c6d-4f09-b249-c67a2e8fda65"` | -| | `query_consistency` | `string` | For queries: the consistency of the query - `strongconsistency` or `weakconsistency`. This property is set by the caller as part of the request's [request properties](../api/rest/request-properties.md): The client request property to set is: `queryconsistency`. | `"strongconsistency"`, `"weakconsistency"` | -| | `request_description` | `string` | Custom text that the author of the request can include. The text is set by the caller as part of the request's [Client request properties](../api/rest/request-properties.md): The client request property to set is: `request_description`. | `"Some custom description"`; automatically populated for dashboards: `"dashboard:{dashboard_id};version:{version};sourceId:{source_id};sourceType:{tile/parameter}"` | -| | `request_text` | `string` | The obfuscated text of the request. Obfuscated string literals included in the query text are replaced by multiple of star (`*`) characters. **Note:** only the leading 65,536 characters of the request text are evaluated. | `".show version"` | -| | `request_type` | `string` | The type of the request - `Command` or `Query`. | `"Command"`, `"Query"` | +| Name | Type | Description | Examples | +|--|--|--|--| +| `current_database` | `string` | The name of the request database. | `"MyDatabase"` | +| `current_application` | `string` | The name of the application that sent the request. | `"Kusto.Explorer"`, `"KusWeb"` | +| `current_principal` | `string` | The fully qualified name of the principal identity that sent the request. | `"aaduser=1793eb1f-4a18-418c-be4c-728e310c86d3;83af1c0e-8c6d-4f09-b249-c67a2e8fda65"` | +| `query_consistency` | `string` | For queries: the consistency of the query - `strongconsistency` or `weakconsistency`. This property is set by the caller as part of the request's [request properties](../api/rest/request-properties.md): The client request property to set is: `queryconsistency`. | `"strongconsistency"`, `"weakconsistency"` | +| `request_description` | `string` | Custom text that the author of the request can include. The text is set by the caller as part of the request's [Client request properties](../api/rest/request-properties.md): The client request property to set is: `request_description`. | `"Some custom description"`; automatically populated for dashboards: `"dashboard:{dashboard_id};version:{version};sourceId:{source_id};sourceType:{tile/parameter}"` | +| `request_text` | `string` | The obfuscated text of the request. Obfuscated string literals included in the query text are replaced by multiple of star (`*`) characters. **Note:** only the leading 65,536 characters of the request text are evaluated. | `".show version"` | +| `request_type` | `string` | The type of the request - `Command` or `Query`. | `"Command"`, `"Query"` | > [!NOTE] > Setting the request options using `set` statements doesn't allow classification by them. The options need to be set in the client request properties object. From d644805d43edbcab6601699402f57342e4ee9785 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Thu, 2 Jan 2025 11:52:18 +0200 Subject: [PATCH 13/19] edits --- .../management/alter-merge-workload-group-command.md | 4 ++-- .../management/create-or-alter-workload-group-command.md | 2 +- .../kusto/management/drop-workload-group-command.md | 4 ++-- .../kusto/management/query-consistency-policy.md | 5 ++--- data-explorer/kusto/management/request-limits-policy.md | 6 +++--- .../kusto/management/request-rate-limit-policy.md | 6 +++--- .../management/request-rate-limits-enforcement-policy.md | 8 +++++--- .../kusto/management/show-workload-group-command.md | 2 +- 8 files changed, 19 insertions(+), 18 deletions(-) diff --git a/data-explorer/kusto/management/alter-merge-workload-group-command.md b/data-explorer/kusto/management/alter-merge-workload-group-command.md index f45427e060..6660e76816 100644 --- a/data-explorer/kusto/management/alter-merge-workload-group-command.md +++ b/data-explorer/kusto/management/alter-merge-workload-group-command.md @@ -3,7 +3,7 @@ title: .alter-merge workload_group command description: Learn how to use the ".alter-merge workload_group" command to alter a workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 01/01/2025 +ms.date: 01/02/2025 --- # .alter-merge workload_group command @@ -171,6 +171,6 @@ Specify the applicable option for the query consistency model. ## Related content * [Workload groups](workload-groups.md) +* [.create-or-alter workload_group](create-or-alter-workload-group-command.md) * [.drop workload_group](drop-workload-group-command.md) * [.show workload_group](show-workload-group-command.md) -* [.create-or-alter workload_group](create-or-alter-workload-group-command.md) diff --git a/data-explorer/kusto/management/create-or-alter-workload-group-command.md b/data-explorer/kusto/management/create-or-alter-workload-group-command.md index da0ef5003b..5d57b21699 100644 --- a/data-explorer/kusto/management/create-or-alter-workload-group-command.md +++ b/data-explorer/kusto/management/create-or-alter-workload-group-command.md @@ -183,6 +183,6 @@ Create **My Workload Group** workload group with a full definition of its reques ## Related content * [Workload groups](workload-groups.md) -* [.show workload_group](show-workload-group-command.md) * [.alter-merge workload_group](alter-merge-workload-group-command.md) * [.drop workload_group](drop-workload-group-command.md) +* [.show workload_group](show-workload-group-command.md) diff --git a/data-explorer/kusto/management/drop-workload-group-command.md b/data-explorer/kusto/management/drop-workload-group-command.md index 97703232c4..da2e613d51 100644 --- a/data-explorer/kusto/management/drop-workload-group-command.md +++ b/data-explorer/kusto/management/drop-workload-group-command.md @@ -3,7 +3,7 @@ title: .drop workload_group command description: Learn how to use the `.drop workload_group` command to drop a workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 01/01/2025 +ms.date: 01/02/2025 --- # .drop workload_group command @@ -88,5 +88,5 @@ The `internal` and `default` workload groups can't be dropped. An attempt to dro * [Workload groups](workload-groups.md) * [.alter-merge workload_group](alter-merge-workload-group-command.md) -* [.show workload_group](show-workload-group-command.md) * [.create-or-alter workload_group](create-or-alter-workload-group-command.md) +* [.show workload_group](show-workload-group-command.md) diff --git a/data-explorer/kusto/management/query-consistency-policy.md b/data-explorer/kusto/management/query-consistency-policy.md index 50f6def385..f5c6578079 100644 --- a/data-explorer/kusto/management/query-consistency-policy.md +++ b/data-explorer/kusto/management/query-consistency-policy.md @@ -3,7 +3,7 @@ title: Query consistency policy description: Learn how to use the query consistency policy to control the consistency mode of queries. ms.reviewer: yonil ms.topic: reference -ms.date: 01/01/2025 +ms.date: 01/02/2025 --- # Query consistency policy @@ -57,8 +57,7 @@ You can monitor the latency of the metadata snapshot age on nodes serving as wea * [Workload groups](workload-groups.md) * [Query consistency](../concepts/query-consistency.md) -* [.show workload_group command](show-workload-group-command.md) * [.alter-merge workload_group](alter-merge-workload-group-command.md) * [.create-or-alter workload_group command](create-or-alter-workload-group-command.md) * [.drop workload_group](drop-workload-group-command.md) - +* [.show workload_group command](show-workload-group-command.md) diff --git a/data-explorer/kusto/management/request-limits-policy.md b/data-explorer/kusto/management/request-limits-policy.md index 28874aa2d1..299d521708 100644 --- a/data-explorer/kusto/management/request-limits-policy.md +++ b/data-explorer/kusto/management/request-limits-policy.md @@ -3,7 +3,7 @@ title: Request limits policy description: Learn how to use the request limits policy to limit the resources used by the request during its execution. ms.reviewer: yonil ms.topic: reference -ms.date: 01/01/2025 +ms.date: 01/02/2025 --- # Request limits policy @@ -150,7 +150,7 @@ The following JSON represents a custom requests limits policy object: * [Workload groups](workload-groups.md) * [Client request properties](../api/rest/request-properties.md) -* [.show workload_group command](show-workload-group-command.md) -* [.drop workload_group](drop-workload-group-command.md) * [.alter-merge workload_group](alter-merge-workload-group-command.md) * [.create-or-alter workload_group](create-or-alter-workload-group-command.md) +* [.drop workload_group](drop-workload-group-command.md) +* [.show workload_group command](show-workload-group-command.md) diff --git a/data-explorer/kusto/management/request-rate-limit-policy.md b/data-explorer/kusto/management/request-rate-limit-policy.md index 6337a042b4..4a624d27f4 100644 --- a/data-explorer/kusto/management/request-rate-limit-policy.md +++ b/data-explorer/kusto/management/request-rate-limit-policy.md @@ -3,7 +3,7 @@ title: Request rate limit policy description: Learn how to use the request rate limit policy to limit the number of concurrent requests classified into a workload group. ms.reviewer: yonil ms.topic: reference -ms.date: 12/25/2024 +ms.date: 01/02/2025 --- # Request rate limit policy @@ -175,7 +175,7 @@ The following policies will block all requests classified to the workload group: * [Workload groups](workload-groups.md) * [Request rate limits enforcement policy](request-rate-limits-enforcement-policy.md) -* [.show workload_group command](show-workload-group-command.md) -* [.drop workload_group command](drop-workload-group-command.md) * [.alter-merge workload_group command](alter-merge-workload-group-command.md) * [.create-or-alter workload_group command](create-or-alter-workload-group-command.md) +* [.drop workload_group command](drop-workload-group-command.md) +* [.show workload_group command](show-workload-group-command.md) diff --git a/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md b/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md index 596958d405..071374dc44 100644 --- a/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md +++ b/data-explorer/kusto/management/request-rate-limits-enforcement-policy.md @@ -3,7 +3,7 @@ title: Request rate limits enforcement policy description: Learn how to use the request rate limits enforcement policy to enforce request rate limits. ms.reviewer: yonil ms.topic: reference -ms.date: 12/25/2024 +ms.date: 01/02/2025 --- # Request rate limits enforcement policy @@ -112,6 +112,8 @@ The effective rate limits for the `default` workload group are: ## Related content * [Workload groups](workload-groups.md) -* [.show workload_group command](show-workload-group-command.md) -* [.alter-merge workload_group command](alter-merge-workload-group-command.md) * [System information](system-info.md) +* [.alter-merge workload_group command](alter-merge-workload-group-command.md) +* [.create-or-alter workload_group command](create-or-alter-workload-group-command.md) +* [.drop workload_group command](drop-workload-group-command.md) +* [.show workload_group command](show-workload-group-command.md) diff --git a/data-explorer/kusto/management/show-workload-group-command.md b/data-explorer/kusto/management/show-workload-group-command.md index 82e3bcd135..5958bc285d 100644 --- a/data-explorer/kusto/management/show-workload-group-command.md +++ b/data-explorer/kusto/management/show-workload-group-command.md @@ -58,6 +58,6 @@ Show definition for **MyWorkloadGroup** workload group: * [Request rate limits enforcement policy](request-rate-limits-enforcement-policy.md) * [Request queuing policy](request-queuing-policy.md) * [Query consistency policy](query-consistency-policy.md) -* [.drop workload_group](drop-workload-group-command.md) * [.alter-merge workload_group](alter-merge-workload-group-command.md) * [.create-or-alter workload_group](create-or-alter-workload-group-command.md) +* [.drop workload_group](drop-workload-group-command.md) From c0a7007a3b6f8138f58da5bf3b7a5daa88c623d8 Mon Sep 17 00:00:00 2001 From: ktalmor <193799742+ktalmor@users.noreply.github.com> Date: Wed, 8 Jan 2025 15:19:02 +0200 Subject: [PATCH 14/19] added note (line 34) --- data-explorer/kusto/query/column-ifexists-function.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/data-explorer/kusto/query/column-ifexists-function.md b/data-explorer/kusto/query/column-ifexists-function.md index 8d92e43c49..0e19c12693 100644 --- a/data-explorer/kusto/query/column-ifexists-function.md +++ b/data-explorer/kusto/query/column-ifexists-function.md @@ -30,6 +30,9 @@ Takes a column name as a string and a default value. Returns a reference to the If *columnName* exists, then the column it refers to. Otherwise *defaultValue*. +> [!NOTE] +> If *columnName* has a dynamic value, then it returns *defaultValue*. + ## Example :::moniker range="azure-data-explorer" From 10600f67257a5a712a5a6b86fb9e5132f5ecfc0c Mon Sep 17 00:00:00 2001 From: ktalmor <193799742+ktalmor@users.noreply.github.com> Date: Thu, 9 Jan 2025 13:52:46 +0200 Subject: [PATCH 15/19] Added 2 output headers --- data-explorer/kusto/query/arg-min-aggregation-function.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/data-explorer/kusto/query/arg-min-aggregation-function.md b/data-explorer/kusto/query/arg-min-aggregation-function.md index bfb9501015..4464abe9dd 100644 --- a/data-explorer/kusto/query/arg-min-aggregation-function.md +++ b/data-explorer/kusto/query/arg-min-aggregation-function.md @@ -53,6 +53,8 @@ StormEvents | summarize arg_min(BeginLat, BeginLocation) by State ``` +**Output** + The results table shown includes only the first 10 rows. | State | BeginLat | BeginLocation | @@ -84,6 +86,8 @@ StormEvents | summarize arg_min(StartTime, *) by State ``` +**Output** + The results table shown includes only the first 10 rows and first 3 columns. | State | StartTime | EndTime | ... | From 0c0d19c2ee61007b221b39f05da16405fc7ab5c6 Mon Sep 17 00:00:00 2001 From: ktalmor <193799742+ktalmor@users.noreply.github.com> Date: Thu, 9 Jan 2025 14:26:42 +0200 Subject: [PATCH 16/19] Enhanced the topic --- .../kusto/query/column-ifexists-function.md | 42 ++++++++++++++----- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/data-explorer/kusto/query/column-ifexists-function.md b/data-explorer/kusto/query/column-ifexists-function.md index 0e19c12693..46fe302bf3 100644 --- a/data-explorer/kusto/query/column-ifexists-function.md +++ b/data-explorer/kusto/query/column-ifexists-function.md @@ -1,15 +1,18 @@ --- title: column_ifexists() -description: Learn how to use the column_ifexists() function to return a reference to the column if it exists. +description: Learn how to use the column_ifexists() function to return a reference to the column if it exists. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 01/09/2025 --- # column_ifexists() > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)] -Takes a column name as a string and a default value. Returns a reference to the column if it exists, otherwise - returns the default value. +Disaplays the column, if the column exists. Otherwise, it returns the default column. + +> [!NOTE] +> If the column data type is [dynamic](scalar-data-types/dynamic.md), then it returns the default column. > **Deprecated aliases:** columnifexists() @@ -23,24 +26,41 @@ Takes a column name as a string and a default value. Returns a reference to the | Name | Type | Required | Description | |--|--|--|--| -| *columnName* | `string` | :heavy_check_mark: | The name of the column to check if exists.| -| *defaultValue* | scalar | :heavy_check_mark: | The value to use if the column doesn't exist. This value can be any scalar expression. For example, a reference to another column.| +| *columnName* | `string` | :heavy_check_mark: | The name of the column to return.| +| *defaultValue* | scalar | :heavy_check_mark: | The default column to return if *columnName* doesn't exist in the table. This value can be any scalar expression. For example, a reference to another column.| ## Returns -If *columnName* exists, then the column it refers to. Otherwise *defaultValue*. - -> [!NOTE] -> If *columnName* has a dynamic value, then it returns *defaultValue*. +If *columnName* exists, then returns the column. Otherwise, it returns the *defaultValue* column. ## Example +This example returns the default **State** column, becuase a column named Capital does not exsit in the StormEvents table. + :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] -> Run the query +> Run the query ::: moniker-end ```kusto -// There's no column "Capital" in "StormEvents", therefore, the State column will be used instead StormEvents | project column_ifexists("Capital", State) ``` + +**Output** + +This output shows the first ten rows of the default **State** column. + + +| State | +|-------| +| ATLANTIC SOUTH | +| FLORIDA | +| FLORIDA | +| GEORGIA | +| MISSISSIPPI | +| MISSISSIPPI | +| MISSISSIPPI | +| MISSISSIPPI | +| AMERICAN SAMOA | +| KENTUCKY | +| ... | \ No newline at end of file From bd2ba279108a45747680c79c378ed87d9a5a46bc Mon Sep 17 00:00:00 2001 From: ktalmor <193799742+ktalmor@users.noreply.github.com> Date: Thu, 9 Jan 2025 14:29:56 +0200 Subject: [PATCH 17/19] spell check --- data-explorer/kusto/query/column-ifexists-function.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/data-explorer/kusto/query/column-ifexists-function.md b/data-explorer/kusto/query/column-ifexists-function.md index 46fe302bf3..64c608526c 100644 --- a/data-explorer/kusto/query/column-ifexists-function.md +++ b/data-explorer/kusto/query/column-ifexists-function.md @@ -9,7 +9,7 @@ ms.date: 01/09/2025 > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)] -Disaplays the column, if the column exists. Otherwise, it returns the default column. +Displays the column, if the column exists. Otherwise, it returns the default column. > [!NOTE] > If the column data type is [dynamic](scalar-data-types/dynamic.md), then it returns the default column. @@ -35,7 +35,7 @@ If *columnName* exists, then returns the column. Otherwise, it returns the *defa ## Example -This example returns the default **State** column, becuase a column named Capital does not exsit in the StormEvents table. +This example returns the default **State** column, because a column named Capital doesn't exist in the StormEvents table. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -48,7 +48,7 @@ StormEvents | project column_ifexists("Capital", State) **Output** -This output shows the first ten rows of the default **State** column. +This output shows the first 10 rows of the default **State** column. | State | From da96372d6d7d3261bc0d3e4602044c24f8bc63c8 Mon Sep 17 00:00:00 2001 From: ktalmor <193799742+ktalmor@users.noreply.github.com> Date: Thu, 9 Jan 2025 15:39:41 +0200 Subject: [PATCH 18/19] removed note after testing --- data-explorer/kusto/query/column-ifexists-function.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/data-explorer/kusto/query/column-ifexists-function.md b/data-explorer/kusto/query/column-ifexists-function.md index 64c608526c..be8e48cc0b 100644 --- a/data-explorer/kusto/query/column-ifexists-function.md +++ b/data-explorer/kusto/query/column-ifexists-function.md @@ -11,9 +11,6 @@ ms.date: 01/09/2025 Displays the column, if the column exists. Otherwise, it returns the default column. -> [!NOTE] -> If the column data type is [dynamic](scalar-data-types/dynamic.md), then it returns the default column. - > **Deprecated aliases:** columnifexists() ## Syntax @@ -35,7 +32,7 @@ If *columnName* exists, then returns the column. Otherwise, it returns the *defa ## Example -This example returns the default **State** column, because a column named Capital doesn't exist in the StormEvents table. +This example returns the default **State** column, because a column named Capital doesn't exist in the *StormEvents* table. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -50,7 +47,6 @@ StormEvents | project column_ifexists("Capital", State) This output shows the first 10 rows of the default **State** column. - | State | |-------| | ATLANTIC SOUTH | @@ -63,4 +59,4 @@ This output shows the first 10 rows of the default **State** column. | MISSISSIPPI | | AMERICAN SAMOA | | KENTUCKY | -| ... | \ No newline at end of file +| ... | From f020ccd53b253a3a152c6c897bc92a4d237efea1 Mon Sep 17 00:00:00 2001 From: Meira Josephy <144697924+mjosephym@users.noreply.github.com> Date: Thu, 9 Jan 2025 16:03:09 +0200 Subject: [PATCH 19/19] Update data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md Co-authored-by: Shlomo Sagir <51323195+shsagir@users.noreply.github.com> --- .../alter-database-streaming-ingestion-policy-command.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md index 63ae307e31..cdc39cc0ea 100644 --- a/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md +++ b/data-explorer/kusto/management/alter-database-streaming-ingestion-policy-command.md @@ -14,7 +14,7 @@ Changes the database streaming ingestion policy. The [streaming policy](../manag Streaming ingestion is best suited for low latency scenarios where the ingestion time is under 10 seconds for varying data volume. It can optimize processing for multiple tables across one or more databases, especially when the tables receive only a few records per second but the ingestion volume is thousands of records per second. -However, when the amount of data grows beyond 4 GB per hour per table, the recommendation is to switch to classic bulk ingestion instead of streaming ingestion. +However, when the amount of data grows beyond 4 GB per hour per table, we recommend switching to classic bulk ingestion instead of streaming ingestion. To learn how to implement streaming ingestion, see [streaming ingestion](/azure/data-explorer/ingest-data-streaming). Streaming ingestion must be enabled at the cluster level before it can be used at the database level.