From 16f5b56c74de221776168c0cdf15be1635892d54 Mon Sep 17 00:00:00 2001 From: andykop <165172502+AndyKop@users.noreply.github.com> Date: Wed, 18 Dec 2024 12:02:29 +0200 Subject: [PATCH 1/6] updating explanations and examples --- .../query/series-periods-detect-function.md | 37 ++++++++++++++++--- 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/data-explorer/kusto/query/series-periods-detect-function.md b/data-explorer/kusto/query/series-periods-detect-function.md index 9379c6033f..907f57f30b 100644 --- a/data-explorer/kusto/query/series-periods-detect-function.md +++ b/data-explorer/kusto/query/series-periods-detect-function.md @@ -9,7 +9,9 @@ ms.date: 08/11/2024 > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)] -Finds the most significant periods that exist in a time series. +Finds the most significant periods within a time series. + +The series_periods_detect() function is particularly useful for detecting periodic patterns in data, such as daily, weekly, or monthly cycles. ## Syntax @@ -22,9 +24,9 @@ Finds the most significant periods that exist in a time series. | Name | Type | Required | Description | |--|--|--|--| | *series* | `dynamic` | :heavy_check_mark: | An array of numeric values, typically the resulting output of the [make-series](make-series-operator.md) or [make_list](make-list-aggregation-function.md) operators.| -| *min_period* | `real` | :heavy_check_mark: | The minimal period for which to search.| -| *max_period* | `real` | :heavy_check_mark: | The maximal period for which to search.| -| *num_periods* | `long` | :heavy_check_mark: | The maximum required number of periods. This number will be the length of the output dynamic arrays.| +| *min_period* | `real` | :heavy_check_mark: | The minimal period length for which to search.| +| *max_period* | `real` | :heavy_check_mark: | The maximal period length for which to search.| +| *num_periods* | `long` | :heavy_check_mark: | The maximum number of periods to return. This number will be the length of the output dynamic arrays.| > [!IMPORTANT] > @@ -34,14 +36,37 @@ Finds the most significant periods that exist in a time series. ## Returns -The function outputs a table with two columns: +The function returns a table with two columns: * *periods*: A dynamic array containing the periods that have been found, in units of the bin size, ordered by their scores. * *scores*: A dynamic array containing values between 0 and 1. Each array measures the significance of a period in its respective position in the *periods* array. ## Example -The following query embeds a snapshot of a month of an application’s traffic, aggregated twice a day. The bin size is 12 hours. +### + +The following query analyzes the series containing aggregated daily data, and returns the most significant period within the defined period range. This means the function will consider all possible period lengths between 0 and 50 days. The returned period represents the most prominent cycle or pattern detected in the data. + +:::moniker range="azure-data-explorer" +> [!div class="nextstepaction"] +> Run the query +::: moniker-end + +```kusto +let series = dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94, 149, 95, 140, 77, 61, 50, 54, 47, 133, 72, 152, 94, 148, 105, 162, 101, 160, 87, 63, 53, 55, 54, 151, 103, 189, 108, 183, 113, 175, 113, 178, 90, 71, 62, 62, 65, 165, 109, 181, 115, 182, 121, 178, 114, 170]); +| print periods_detected = series_periods_detect(series, 0.0, 50.0, 1) +``` + +**Output** + +The query returns a significant cycle repeating every 14 days. + +| periods_detected | +| --- | +| 14 | + + +The following query embeds a snapshot of a month of an application’s traffic. The amount of traffic is aggregated twice a day, meaning the bin size is 12 hours. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] From e7d621ccf34e125e154909e2fe3dff412ddeef8b Mon Sep 17 00:00:00 2001 From: andykop <165172502+AndyKop@users.noreply.github.com> Date: Wed, 18 Dec 2024 15:14:00 +0200 Subject: [PATCH 2/6] update explanations --- .../query/series-periods-detect-function.md | 48 +++++++------------ 1 file changed, 18 insertions(+), 30 deletions(-) diff --git a/data-explorer/kusto/query/series-periods-detect-function.md b/data-explorer/kusto/query/series-periods-detect-function.md index 907f57f30b..8d7db0e8e6 100644 --- a/data-explorer/kusto/query/series-periods-detect-function.md +++ b/data-explorer/kusto/query/series-periods-detect-function.md @@ -3,7 +3,7 @@ title: series_periods_detect() description: Learn how to use the series_periods_detect() function to find the most significant periods that exist in a time series. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/18/2024 --- # series_periods_detect() @@ -11,7 +11,7 @@ ms.date: 08/11/2024 Finds the most significant periods within a time series. -The series_periods_detect() function is particularly useful for detecting periodic patterns in data, such as daily, weekly, or monthly cycles. +The series_periods_detect() function is useful for detecting periodic patterns in data, such as daily, weekly, or monthly cycles. ## Syntax @@ -26,7 +26,7 @@ The series_periods_detect() function is particularly useful for detecting period | *series* | `dynamic` | :heavy_check_mark: | An array of numeric values, typically the resulting output of the [make-series](make-series-operator.md) or [make_list](make-list-aggregation-function.md) operators.| | *min_period* | `real` | :heavy_check_mark: | The minimal period length for which to search.| | *max_period* | `real` | :heavy_check_mark: | The maximal period length for which to search.| -| *num_periods* | `long` | :heavy_check_mark: | The maximum number of periods to return. This number will be the length of the output dynamic arrays.| +| *num_periods* | `long` | :heavy_check_mark: | The maximum number of periods to return. This number is the length of the output dynamic arrays.| > [!IMPORTANT] > @@ -38,35 +38,14 @@ The series_periods_detect() function is particularly useful for detecting period The function returns a table with two columns: -* *periods*: A dynamic array containing the periods that have been found, in units of the bin size, ordered by their scores. +* *periods*: A dynamic array containing the periods found, in units of the bin size, ordered by their scores. * *scores*: A dynamic array containing values between 0 and 1. Each array measures the significance of a period in its respective position in the *periods* array. ## Example -### +### Line chart to show patterns -The following query analyzes the series containing aggregated daily data, and returns the most significant period within the defined period range. This means the function will consider all possible period lengths between 0 and 50 days. The returned period represents the most prominent cycle or pattern detected in the data. - -:::moniker range="azure-data-explorer" -> [!div class="nextstepaction"] -> Run the query -::: moniker-end - -```kusto -let series = dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94, 149, 95, 140, 77, 61, 50, 54, 47, 133, 72, 152, 94, 148, 105, 162, 101, 160, 87, 63, 53, 55, 54, 151, 103, 189, 108, 183, 113, 175, 113, 178, 90, 71, 62, 62, 65, 165, 109, 181, 115, 182, 121, 178, 114, 170]); -| print periods_detected = series_periods_detect(series, 0.0, 50.0, 1) -``` - -**Output** - -The query returns a significant cycle repeating every 14 days. - -| periods_detected | -| --- | -| 14 | - - -The following query embeds a snapshot of a month of an application’s traffic. The amount of traffic is aggregated twice a day, meaning the bin size is 12 hours. +The following query embeds a snapshot of application traffic for one month. The amount of traffic is aggregated twice a day, meaning the bin size is 12 hours. The query produces a line chart clearly showing a pattern in the data. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -81,7 +60,9 @@ print y=dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94 :::image type="content" source="media/series-periods/series-periods.png" alt-text="Series periods."::: -Running `series_periods_detect()` on this series, results in the weekly period, 14 points long. +### Find the period + +Let's run `series_periods_detect()` on the same series to identify the recurring patterns. The query returns the two most significant periods within the defined period range. This means the function considers all possible period lengths between 0 and 25 days and search for patterns within those ranges. The returned periods represent the most prominent cycles or patterns detected in the data. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] @@ -98,7 +79,14 @@ print y=dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94 | series\_periods\_detect\_y\_periods | series\_periods\_detect\_y\_periods\_scores | |-------------|-------------------| -| [14.0, 0.0] | [0.84, 0.0] | +| [14, 0] | [0.84, 0] | + +The value in series\_periods\_detect\_y\_periods\_scores is truncated. > [!NOTE] -> The daily period that can be also seen in the chart wasn't found because the sampling is too coarse (12h bin size), so a daily period of 2 bins is below the minimum period size of 4 points, required by the algorithm. +> There is also a daily pattern visible in the chart, but this is not returned by the query because the sampling is too coarse (12h bin size). A daily period of 2 bins is below the minimum period size of 4 points, required by the algorithm. + +## Related content + +* [Regular expressions](/dotnet/standard/base-types/regular-expression-language-quick-reference) +* [series_periods_validate()](series-periods-validate-function.md) From 74e02e9edf6ac17a56bb4863f289df7788b7da32 Mon Sep 17 00:00:00 2001 From: andykop <165172502+AndyKop@users.noreply.github.com> Date: Wed, 18 Dec 2024 15:21:15 +0200 Subject: [PATCH 3/6] example headings --- data-explorer/kusto/query/series-periods-detect-function.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/data-explorer/kusto/query/series-periods-detect-function.md b/data-explorer/kusto/query/series-periods-detect-function.md index 8d7db0e8e6..b700776d95 100644 --- a/data-explorer/kusto/query/series-periods-detect-function.md +++ b/data-explorer/kusto/query/series-periods-detect-function.md @@ -43,8 +43,6 @@ The function returns a table with two columns: ## Example -### Line chart to show patterns - The following query embeds a snapshot of application traffic for one month. The amount of traffic is aggregated twice a day, meaning the bin size is 12 hours. The query produces a line chart clearly showing a pattern in the data. :::moniker range="azure-data-explorer" @@ -60,8 +58,6 @@ print y=dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94 :::image type="content" source="media/series-periods/series-periods.png" alt-text="Series periods."::: -### Find the period - Let's run `series_periods_detect()` on the same series to identify the recurring patterns. The query returns the two most significant periods within the defined period range. This means the function considers all possible period lengths between 0 and 25 days and search for patterns within those ranges. The returned periods represent the most prominent cycles or patterns detected in the data. :::moniker range="azure-data-explorer" From 568170b99fd3f7397762bedf8b20323cd0666895 Mon Sep 17 00:00:00 2001 From: andykop <165172502+AndyKop@users.noreply.github.com> Date: Wed, 18 Dec 2024 15:22:30 +0200 Subject: [PATCH 4/6] tweaks --- data-explorer/kusto/query/series-periods-detect-function.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-explorer/kusto/query/series-periods-detect-function.md b/data-explorer/kusto/query/series-periods-detect-function.md index b700776d95..717f22585f 100644 --- a/data-explorer/kusto/query/series-periods-detect-function.md +++ b/data-explorer/kusto/query/series-periods-detect-function.md @@ -58,7 +58,7 @@ print y=dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94 :::image type="content" source="media/series-periods/series-periods.png" alt-text="Series periods."::: -Let's run `series_periods_detect()` on the same series to identify the recurring patterns. The query returns the two most significant periods within the defined period range. This means the function considers all possible period lengths between 0 and 25 days and search for patterns within those ranges. The returned periods represent the most prominent cycles or patterns detected in the data. +Let's run `series_periods_detect()` on the same series to identify the recurring patterns. The query returns the two most significant periods within the defined period range. This means the function considers all possible period lengths between 0 and 25 days and search for patterns within those ranges. The returned periods represent the two most prominent cycles or patterns detected in the data. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] From c0fb4f674d3a1836a2dfbd761e0f2f0c4615a020 Mon Sep 17 00:00:00 2001 From: Andy Kopelowitz <165172502+AndyKop@users.noreply.github.com> Date: Thu, 19 Dec 2024 09:37:35 +0200 Subject: [PATCH 5/6] Update data-explorer/kusto/query/series-periods-detect-function.md Co-authored-by: Shlomo Sagir <51323195+shsagir@users.noreply.github.com> --- data-explorer/kusto/query/series-periods-detect-function.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-explorer/kusto/query/series-periods-detect-function.md b/data-explorer/kusto/query/series-periods-detect-function.md index 717f22585f..85a6448fef 100644 --- a/data-explorer/kusto/query/series-periods-detect-function.md +++ b/data-explorer/kusto/query/series-periods-detect-function.md @@ -58,7 +58,7 @@ print y=dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94 :::image type="content" source="media/series-periods/series-periods.png" alt-text="Series periods."::: -Let's run `series_periods_detect()` on the same series to identify the recurring patterns. The query returns the two most significant periods within the defined period range. This means the function considers all possible period lengths between 0 and 25 days and search for patterns within those ranges. The returned periods represent the two most prominent cycles or patterns detected in the data. +You can run the `series_periods_detect()` function on the same series to identify the recurring patterns. The function searches for patterns in the specified period range and returns two values. The first indicate a detected pattern that is 14 point long and the other, zero, indicates no additional pattern was found. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"] From 608c1aec6cc791a8e99d4cd4c43af28183fa60c2 Mon Sep 17 00:00:00 2001 From: Shlomo Sagir <51323195+shsagir@users.noreply.github.com> Date: Thu, 19 Dec 2024 10:54:07 +0200 Subject: [PATCH 6/6] Update data-explorer/kusto/query/series-periods-detect-function.md --- data-explorer/kusto/query/series-periods-detect-function.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data-explorer/kusto/query/series-periods-detect-function.md b/data-explorer/kusto/query/series-periods-detect-function.md index 85a6448fef..af22c62e84 100644 --- a/data-explorer/kusto/query/series-periods-detect-function.md +++ b/data-explorer/kusto/query/series-periods-detect-function.md @@ -58,7 +58,7 @@ print y=dynamic([80, 139, 87, 110, 68, 54, 50, 51, 53, 133, 86, 141, 97, 156, 94 :::image type="content" source="media/series-periods/series-periods.png" alt-text="Series periods."::: -You can run the `series_periods_detect()` function on the same series to identify the recurring patterns. The function searches for patterns in the specified period range and returns two values. The first indicate a detected pattern that is 14 point long and the other, zero, indicates no additional pattern was found. +You can run the `series_periods_detect()` function on the same series to identify the recurring patterns. The function searches for patterns in the specified period range and returns two values. The first value indicates a detected pattern that is 14 point long with a score of approximately .84. The other value is zero that indicates no additional pattern was found. :::moniker range="azure-data-explorer" > [!div class="nextstepaction"]