diff --git a/data-explorer/get-data-storage.md b/data-explorer/get-data-storage.md index 2ba48db4dc..8e6ed993ea 100644 --- a/data-explorer/get-data-storage.md +++ b/data-explorer/get-data-storage.md @@ -3,14 +3,14 @@ title: Get data from Azure storage description: Learn how to get data from Azure storage in Azure Data Explorer. ms.reviewer: sharmaanshul ms.topic: how-to -ms.date: 12/04/2023 +ms.date: 12/04/2024 --- # Get data from Azure storage Data ingestion is the process used to load data from one or more sources into a table in Azure Data Explorer. Once ingested, the data becomes available for query. In this article, you learn how to get data from Azure storage (ADLS Gen2 container, blob container, or individual blobs) into either a new or existing table. -Ingestion can be done as a one-time operation, or as a continuous method. Continuous ingestion can be configured via portal only. +Ingestion from an Azure storage account is a one-time operation. To ingest data continuously, see [Configure streaming ingestion](ingest-data-streaming.md). For general information on data ingestion, see [Azure Data Explorer data ingestion overview](ingest-data-overview.md). @@ -59,14 +59,16 @@ Select the data source from the available list. In this example, you're ingestin | Folder path| Filters data to ingest files with a specific folder path. | | File extension| Filters data to ingest files with a specific file extension only.| - 1. If you selected **Add URI**, paste your storage connection string for a blob container or individual files in the **URI** field, and then select **+**. + 1. If you selected **Add URI**, from the storage account, generate an SAS URL for the container or individual blobs you want to ingest. Set the permissions to **Read** and **List** for containers or **Read** for individual blobs. For more information, see [Generate a SAS token](/kusto/api/connection-strings/generate-sas-token?view=azure-data-explorer&preserve-view=true). + + 1. Paste the URL into the **URI** field, and then select plus (**+**). You can add multiple URIs for individual blobs, or a single URI for a container. + + :::image type="content" source="media/get-data-storage/add-uri.png" alt-text="Screenshot of configure tab with the connection string pasted in the URI field." lightbox="media/get-data-storage/add-uri.png"::: > [!NOTE] - > > * You can add up to 10 individual blobs. Each blob can be a max of 1 GB uncompressed. > * You can ingest up to 5000 blobs from a single container. - - :::image type="content" source="media/get-data-storage/add-uri.png" alt-text="Screenshot of configure tab with the connection string pasted in the URI field." lightbox="media/get-data-storage/add-uri.png"::: + > * You can't ingest individual blobs and containers in the same ingestion. 1. Select **Next** diff --git a/data-explorer/kusto/api/connection-strings/generate-sas-token.md b/data-explorer/kusto/api/connection-strings/generate-sas-token.md index 5f2976b594..9e5377079d 100644 --- a/data-explorer/kusto/api/connection-strings/generate-sas-token.md +++ b/data-explorer/kusto/api/connection-strings/generate-sas-token.md @@ -9,27 +9,27 @@ ms.date: 08/11/2024 > [!INCLUDE [applies](../../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../../includes/applies-to-version/azure-data-explorer.md)] -This article shows you how to generate a SAS token to a resource. +This article shows you how to generate a SAS token to a storage resource. You can generate SAS tokens for containers and individual blobs. ## Azure portal 1. In the [Azure portal](https://portal.azure.com/), open **Storage accounts**. 1. Select the storage account that contains the resource for which you'd like to create a SAS token. 1. From the left menu, select **Containers**. -1. Right-click on the container for which you'd like to create a SAS token. -1. From the context menu, select **Generate SAS**. +1. Navigate to the container, or drill down to the individual blob for which you'd like to create a SAS token. +1. Right-click on the container or blob and select **Generate SAS** from the context menu. - :::image type="content" source="storage/generate-sas-storage-account.png" alt-text="Screenshot of Azure portal with Containers selected. Specific container is right-clicked and a menu opens. Generate SAS is selected from this menu."::: + :::image type="content" source="storage/generate-sas-storage-account.png" lightbox="storage/generate-sas-storage-account.png" alt-text="Screenshot of Azure portal with Containers selected. Specific container is right-clicked and a menu opens. Generate SAS is selected from this menu."::: -1. In the **Generate SAS** dialog, specify the **Permissions**, **Start and expiry date/time**, and **Allowed IP addresses**. Then, select **Generate SAS token and URL** +1. In the **Generate SAS** dialog, select **Read** and **List** permissions for containers, or **Read** for individual blobs. Optionally, specify the **Start and expiry date/time** and **Allowed IP addresses**. For more information about the optional parameters, see [best practices when authorizing using SAS](/azure/storage/common/storage-sas-overview#best-practices-when-using-sas). Then select **Generate SAS token and URL**. - :::image type="content" source="storage/generate-sas-token-and-url.png" alt-text="Screen shot of the Generate SAS dialog with information filled in and Generate SAS token and URL selected."::: + :::image type="content" source="storage/generate-sas-token-and-url.png" lightbox="storage/generate-sas-token-and-url.png" alt-text="Screenshot of the Generate SAS dialog with information filled in and Generate SAS token and URL selected."::: A new section displays at the bottom of the dialog, listing the blob SAS token and the blob SAS URL. -1. Select the icon to the right of the blob SAS URL to copy it. Then, paste it to save or use it as required. +1. Copy the **Blob SAS URL** value and save or use it as required. - :::image type="content" source="storage/copy-sas-token-and-url.png" alt-text="Screenshot of Azure portal with blob SAS URL generated."::: + :::image type="content" source="storage/copy-sas-token-and-url.png" lightbox="storage/copy-sas-token-and-url.png" alt-text="Screenshot of Azure portal with blob SAS URL generated."::: ## Related content diff --git a/data-explorer/kusto/api/connection-strings/storage/copy-sas-token-and-url.png b/data-explorer/kusto/api/connection-strings/storage/copy-sas-token-and-url.png index 1bd0cd6e6d..bae85e4b19 100644 Binary files a/data-explorer/kusto/api/connection-strings/storage/copy-sas-token-and-url.png and b/data-explorer/kusto/api/connection-strings/storage/copy-sas-token-and-url.png differ diff --git a/data-explorer/kusto/api/connection-strings/storage/generate-sas-token-and-url.png b/data-explorer/kusto/api/connection-strings/storage/generate-sas-token-and-url.png index cd2342abd3..d59daca00e 100644 Binary files a/data-explorer/kusto/api/connection-strings/storage/generate-sas-token-and-url.png and b/data-explorer/kusto/api/connection-strings/storage/generate-sas-token-and-url.png differ diff --git a/data-explorer/kusto/management/show-tables-command.md b/data-explorer/kusto/management/show-tables-command.md index 97814b8b5a..23a567d098 100644 --- a/data-explorer/kusto/management/show-tables-command.md +++ b/data-explorer/kusto/management/show-tables-command.md @@ -3,7 +3,7 @@ title: .show tables command description: Learn how to use the `.show tables` command to show a set that contains the specified tables in the database. ms.reviewer: orspodek ms.topic: reference -ms.date: 08/11/2024 +ms.date: 12/09/2024 --- # .show tables command @@ -32,6 +32,7 @@ You must have at least Database User, Database Viewer, or Database Monitor permi ## Returns + |Output parameter |Type |Description |---|---|--- |TableName | `string` |The name of the table. @@ -39,18 +40,52 @@ You must have at least Database User, Database Viewer, or Database Monitor permi |Folder | `string` |The table's folder. |DocString | `string` |A string documenting the table. -## Example +## Examples + +### Show all tables in the database + +The following example shows information about all tables in the current database. + +:::moniker range="azure-data-explorer" +> [!div class="nextstepaction"] +> Run the query +::: moniker-end ```kusto .show tables -.show tables (T1, ..., Tn) ``` -**Output example** +**Output** + +|Table Name|Database Name|Folder|DocString| +|---|---|---|---| +|StormEvents|Samples|Storm_Events|US storm events. Data source: https://www.ncdc.noaa.gov/stormevents| +|demo_make_series1|Samples|TimeSeries_and_ML|| +|demo_series2|Samples|TimeSeries_and_ML|| +|demo_series3|Samples|TimeSeries_and_ML|| +|demo_many_series1|Samples|TimeSeries_and_ML|| +|ConferenceSessions|Samples|ADX_Conferences|| + + +### Show specific tables + +The following example shows information about the specified tables. + +:::moniker range="azure-data-explorer" +> [!div class="nextstepaction"] +> Run the query +::: moniker-end + +```kusto +.show tables (PopulationData, StormEvents, Trips, GeoRegions) +``` + +**Output** |Table Name |Database Name |Folder | DocString |---|---|---|--- -|Table1 |DB1 |Logs |Contains services logs -|Table2 |DB1 | Reporting | -|Table3 |DB1 | | Extended info | -|Table4 |DB2 | Metrics| Contains services performance information +|StormEvents| Samples |Storm_Events| US storm events. Data source: https://www.ncdc.noaa.gov/stormevents| +|PopulationData|Samples| Storm_Events| +|GeoRegions| Samples| NYC Taxi | +|Trips| Samples| NYC Taxi | + diff --git a/data-explorer/kusto/query/facet-operator.md b/data-explorer/kusto/query/facet-operator.md index 6bbcdd9852..e8e417d77b 100644 --- a/data-explorer/kusto/query/facet-operator.md +++ b/data-explorer/kusto/query/facet-operator.md @@ -9,9 +9,9 @@ ms.date: 08/11/2024 > [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] [!INCLUDE [monitor](../includes/applies-to-version/monitor.md)] [!INCLUDE [sentinel](../includes/applies-to-version/sentinel.md)] -Returns a set of tables, one for each specified column. -Each table specifies the list of values taken by its column. -An additional table can be created by using the `with` clause. +Returns a set of tables, one for each column specified in the facet clause. +Each table contains the list of values taken by its column. +An additional table can be created by using the `with` clause. Facet result tables can't be renamed or referenced by any additional operators. ## Syntax diff --git a/data-explorer/kusto/query/index.md b/data-explorer/kusto/query/index.md index 61a7021297..6f331137bf 100644 --- a/data-explorer/kusto/query/index.md +++ b/data-explorer/kusto/query/index.md @@ -64,7 +64,8 @@ StormEvents | 28| > [!NOTE] -> KQL is case-sensitive for everything – table names, table column names, operators, functions, and so on. +> KQL is case-sensitive for everything – table names, table column names, operators, functions, and so on. +> Keywords can be used as identifiers by enclosing them in brackets and quotes (`['` and `']` or `["` and `"]`). For example, `['where']`. For more information, see [Identifier naming rules](/kusto/query/schema-entities/entity-names?view=azure-data-explorer&preserve-view=true#identifier-naming-rules) This query has a single tabular expression statement. The statement begins with a reference to a table called *StormEvents* and contains several operators, [`where`](where-operator.md) and [`count`](count-operator.md), each separated by a pipe. The data rows for the source table are filtered by the value of the *StartTime* column and then filtered by the value of the *State* column. In the last line, the query returns a table with a single column and a single row containing the count of the remaining rows. diff --git a/data-explorer/media/get-data-storage/add-uri.png b/data-explorer/media/get-data-storage/add-uri.png index 3d98725ab4..3255c70c91 100644 Binary files a/data-explorer/media/get-data-storage/add-uri.png and b/data-explorer/media/get-data-storage/add-uri.png differ