Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/actions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,15 @@ To develop a new Action, check out the [Developing an Action](guides/developing-
Contributing guidelines follow those of the [main DataHub project](docs/CONTRIBUTING.md). We are accepting contributions for Actions, Transformers, and general framework improvements (tests, error handling, etc).


## Custom Document Propagation via Actions

The DataHub Actions framework allows users to implement custom document propagation beyond the default behavior. This can be particularly useful for advanced or specific requirements. Below are the key components to leverage this capability:

- **Overview of the DataHub Actions Framework**: The framework provides a way to define custom actions that can be triggered based on specific events in DataHub.
- **Example Usage**: The `propagation_action.py` script can be used to create custom document propagation logic. This script is part of the DataHub Actions framework and can be customized to fit unique data governance needs.
- **Setup and Configuration**: To set up custom actions, users need to define their action logic and configure it within the DataHub environment. This involves specifying triggers and actions in the configuration files.
- **Common Use Cases**: Custom document propagation might be necessary in scenarios where the default propagation does not meet specific business requirements, such as complex lineage relationships or conditional propagation rules.

## Resources

Check out the [original announcement](https://www.youtube.com/watch?v=7iwNxHgqxtg&t=2189s) of the DataHub Actions Framework at the DataHub April 2022 Town Hall.
Expand Down
11 changes: 11 additions & 0 deletions docs/actions/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,17 @@ The Actions Framework consists of a few core concepts--

Each of these will be described in detail below.

### Custom Document Propagation via Actions

For users with advanced or specific requirements beyond the default behavior of column-level document propagation, the DataHub Actions framework can be leveraged for custom document propagation. This includes:

- An overview of the DataHub Actions framework.
- A specific example of how to use `propagation_action.py` for custom document propagation.
- Steps to set up and configure custom actions in DataHub.
- Common use cases or scenarios where custom document propagation might be necessary.

This addition helps users understand how to extend DataHub's capabilities to fit their unique data governance and documentation needs.


<p align="center">
<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/actions.png"/>
Expand Down
11 changes: 11 additions & 0 deletions docs/actions/guides/developing-an-action.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,17 @@ In this guide, we will outline each step to developing a Action for the DataHub
Developing a DataHub Action is a matter of extending the `Action` base class in Python, installing your
Action to make it visible to the framework, and then configuring the framework to use the new Action.

## Custom Document Propagation via Actions

For users with advanced or specific requirements that go beyond the default behavior of column-level document propagation, the DataHub Actions framework can be leveraged for custom document propagation. This involves:

- An overview of the DataHub Actions framework.
- A specific example of how to use `propagation_action.py` for custom document propagation.
- Steps to set up and configure custom actions in DataHub.
- Common use cases or scenarios where custom document propagation might be necessary.

This addition will help users understand how to extend DataHub's capabilities to fit their unique data governance and documentation needs.


## Step 1: Defining an Action

Expand Down
22 changes: 22 additions & 0 deletions docs/actions/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,28 @@ python3 -m pip install --upgrade acryl-datahub-actions
datahub actions version
```

## Custom Document Propagation via Actions

### Overview of the DataHub Actions Framework

The DataHub Actions framework allows users to define custom actions that can be triggered by specific events within DataHub. This framework is useful for extending DataHub's capabilities beyond its default behavior, such as custom document propagation.

### Example: Using `propagation_action.py` for Custom Document Propagation

To implement custom document propagation, you can use the `propagation_action.py` script. This script is part of the DataHub Actions framework and can be configured to propagate documents based on custom logic.

### Steps to Set Up and Configure Custom Actions

1. **Create an Action Configuration File**: Define the action configuration in a YAML file, specifying the source, filter, and action type.
2. **Run the Action**: Use the `datahub actions` command to execute the action based on the configuration file.
3. **Monitor and Adjust**: Review the logs and adjust the configuration as needed to ensure the desired propagation behavior.

### Common Use Cases for Custom Document Propagation

- Propagating documents based on complex lineage relationships not covered by default settings.
- Implementing organization-specific rules for document propagation.
- Integrating with external systems to enhance document metadata.

### Hello World

DataHub ships with a "Hello World" Action which logs all events it receives to the console.
Expand Down
22 changes: 21 additions & 1 deletion docs/automations/docs-propagation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,32 @@

## Introduction

Documentation Propagation is an automation automatically propagates column and asset (coming soon) descriptions based on downstream column-level lineage and sibling relationships.
Documentation Propagation is an automation that automatically propagates column and asset (coming soon) descriptions based on downstream column-level lineage and sibling relationships.
It simplifies metadata management by ensuring consistency and reducing the manual effort required for documenting data assets to aid
in Data Governance & Compliance along with Data Discovery.

This feature is enabled by default in Open Source DataHub.

## Custom Document Propagation via Actions

For users with advanced or specific requirements that go beyond the default behavior, the DataHub Actions framework can be leveraged for custom document propagation.

### Overview of the DataHub Actions Framework
The DataHub Actions framework allows users to define custom actions that can be triggered based on specific events or conditions within DataHub. This provides flexibility to extend DataHub's capabilities to meet unique data governance and documentation needs.

### Example: Using `propagation_action.py` for Custom Document Propagation
The `propagation_action.py` script is an example of how to implement custom document propagation using the DataHub Actions framework. This script can be customized to define specific propagation logic that suits your requirements.

### Steps to Set Up and Configure Custom Actions
1. **Define Your Action**: Create a Python script that implements the desired propagation logic using the DataHub Actions framework.
2. **Configure the Action**: Set up the action in your DataHub environment by specifying the conditions under which it should be triggered.
3. **Deploy and Test**: Deploy the action and test it to ensure it behaves as expected.

### Common Use Cases for Custom Document Propagation
- Propagating documentation based on complex lineage relationships not covered by default settings.
- Implementing custom rules for documentation updates based on specific business logic.
- Integrating with external systems to fetch or update documentation dynamically.

## Capabilities

### Open Source
Expand Down