From 431c2e5a08893504c8b660ef935a7436068c64bf Mon Sep 17 00:00:00 2001 From: runllm Date: Fri, 18 Apr 2025 13:59:18 +0000 Subject: [PATCH 1/2] Update properties.md --- docs/features/feature-guides/properties.md | 36 ++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/docs/features/feature-guides/properties.md b/docs/features/feature-guides/properties.md index 0d961b9ceac4ff..ba15e6a67df6ba 100644 --- a/docs/features/feature-guides/properties.md +++ b/docs/features/feature-guides/properties.md @@ -57,9 +57,11 @@ By using Structured Properties, compliance and governance officers can ensure co ## Creating, Assigning, and Editing Structured Properties -Structured Properties are defined via YAML, then created and assigned to DataHub Assets via the DataHub CLI. +Structured Properties are defined via YAML, then created and assigned to DataHub Assets via the DataHub CLI. This guide will cover defining structured properties, upserting them into DataHub, associating them with datasets, and verifying their display in the UI. -Here's how we would define the above examples in YAML: +### Defining Structured Properties + +To define a structured property, create a YAML file with details such as ID, qualified name, type, cardinality, display name, entity types, description, and allowed values. Here's how we would define the above examples in YAML: @@ -119,10 +121,40 @@ Here's how we would define the above examples in YAML: +### Upserting Structured Properties + +Use the DataHub CLI to upsert the structured property definition into DataHub. For example, use the following command: + +```shell +datahub properties upsert -f {properties_yaml} +``` + +If successful, you should see `Created structured property urn:li:structuredProperty:...` + :::note To learn more about creating and assigning Structured Properties via CLI, please see the [Create Structured Properties](/docs/api/tutorials/structured-properties.md) tutorial. ::: +### Associating Structured Properties with Datasets + +To associate structured properties with datasets, use a dataset YAML file that includes the structured properties. For example: + +```yaml +- urn: urn:li:dataset:(urn:li:dataPlatform:postgres,my_database,my_table,PROD) + structured_properties: + io.acryl.privacy.retentionTime: 90 +``` + +Use the CLI to upsert this dataset YAML file: + +```shell +datahub dataset upsert -f {dataset_yaml} +``` + +### Verification in the UI + +After successfully associating the structured property with the dataset, verify its display in the DataHub UI under the dataset's properties tab or sidebar. If it's not visible, ensure that the property is configured to be displayed in the UI. + Once a Structured Property is assigned to an Asset, Users with the `Edit Properties` Metadata Privilege will be able to change Structured Property values via the DataHub UI.

From d4b1021e4475c6e6a828676a407646eef1f2c730 Mon Sep 17 00:00:00 2001 From: runllm Date: Fri, 18 Apr 2025 13:59:19 +0000 Subject: [PATCH 2/2] Update structured-properties.md --- docs/api/tutorials/structured-properties.md | 54 +++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/docs/api/tutorials/structured-properties.md b/docs/api/tutorials/structured-properties.md index 9b18aa922290b4..f16d3bc92a3de2 100644 --- a/docs/api/tutorials/structured-properties.md +++ b/docs/api/tutorials/structured-properties.md @@ -1274,6 +1274,60 @@ ELASTICSEARCH_INDEX_BUILDER_MAPPINGS_REINDEX=true ENABLE_STRUCTURED_PROPERTIES_SYSTEM_UPDATE=true ``` +## Adding Structured Properties to Datasets + +This section provides a comprehensive guide on adding structured properties to datasets in DataHub version 0.14.1, especially for users upgrading from earlier versions. It covers defining, upserting, associating, and verifying structured properties. + +### Defining Structured Properties + +To define a structured property, create a YAML file with details such as ID, qualified name, type, cardinality, display name, entity types, description, and allowed values. For example: + +```yaml +- id: io.acryl.privacy.retentionTime + qualified_name: io.acryl.privacy.retentionTime + type: number + cardinality: MULTIPLE + display_name: Retention Time + entity_types: + - dataset + description: "Retention Time is used to figure out how long to retain records in a dataset" + allowed_values: + - value: 30 + description: 30 days, usually reserved for datasets that are ephemeral and contain pii + - value: 90 + description: Use this for datasets that drive monthly reporting but contain pii + - value: 365 + description: Use this for non-sensitive data that can be retained for longer +``` + +### Upserting Structured Properties + +Use the DataHub CLI to upsert the structured property definition: + +```shell +datahub properties upsert -f {properties_yaml} +``` + +### Associating Structured Properties with Datasets + +To associate a structured property with a dataset, use a dataset YAML file: + +```yaml +- urn: urn:li:dataset:(urn:li:dataPlatform:postgres,my_database,my_table,PROD) + structured_properties: + io.acryl.privacy.retentionTime: 90 +``` + +Upsert this dataset YAML file using the CLI: + +```shell +datahub dataset upsert -f {dataset_yaml} +``` + +### Verification in the UI + +After associating the structured property, verify its display in the DataHub UI under the dataset's properties tab or sidebar. If not visible, check configuration settings or logs for errors. + ## Update Structured Property With Breaking Schema Changes This section will demonstrate how to make backwards incompatible schema changes. Making backwards incompatible