Move Airflow to TFXA #186
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| #### SIG TFX-Addons | ||
| # Project Proposal | ||
|
|
||
| **Your name:** Varun Murthy | ||
|
|
||
| **Your email:** murthyvs@google.com | ||
|
|
||
| **Your company/organization:** Google Core ML TFX Team (MTV + Seoul) | ||
|
|
||
| **Project name:** Airflow Orchestration | ||
|
|
||
| ## Project Description | ||
| Moving Airflow to TFX add-ons due to decreasing native support. | ||
|
|
||
| ## Project Category | ||
| Orchestrator | ||
|
|
||
| ## Project Use-Case(s) | ||
| We are moving Airflow from tfx/orchestration to tfx-addons. Native support for Airflow won't be provided in the near future. | ||
|
Contributor
There was a problem hiding this comment. A use-case describes how the project will be used by developers. In this case it should be something more like:
Contributor
There was a problem hiding this comment. Imagine that you're a developer who is new to TFX. What do you need to know about Airflow, and how it fits with TFX? Why should you consider installing this module? |
||
|
|
||
| ## Project Implementation | ||
| 1. Copy tfx/orchestration/airflow to tfx-addons/airflow. | ||
|
Contributor
There was a problem hiding this comment.
Author
There was a problem hiding this comment. So ... this is where it gets a bit interesting because the tutorials on the TF website need to reflect the deprecation too. It's a multi-part effort in which we need to first move the orchestrator to TFXA and THEN update the documentation, delete/remove dependencies from tfx/. We can't provide an accurate, line-by-line description as of today ... but we need to create a project in TFXA to get started.
Contributor
There was a problem hiding this comment. That's ok, just describe the issues and some options for dealing with them, and note that we haven't settled on a particular option yet. At some point however, we will need a solid plan to do this so that we can be successful. |
||
| 2. Mark Airflow as deprecated in tfx/ and indicate that support will be dropped in 1-2 releases. | ||
| 3. Update TFX tutorials on www.tensorflow.org to indicate deprecated and moving to TFXA | ||
|
|
||
| ## Project Dependencies | ||
|
Contributor
There was a problem hiding this comment. These aren't dependencies. For comparison, see: etc. |
||
| TensorFlow Extended (1.9.1) | ||
|
|
||
| ## Project Team | ||
| murthyvs: Varun Murthy (murthyvs@google.com) | ||
|
Comment on lines
+29
to
+30
Member
There was a problem hiding this comment. Will Google continue to support it from TFX-Addons? Or is the idea it will be abandoned? We don't want to reach a place where Airflow is released in TFX-Addons and then abandoned by maintainers. We will need some garantees on maintainance from Google or whoever is maintaining this component of the project. The wording on
Member
There was a problem hiding this comment. btw, unless Google can provide with a clear path of support when moved to TFX-Addons for long term maintainability of this component, expect my -1 on this. |
||
|
|
||
| # Note | ||
| Please be aware of the processes and requirements which are outlined here: | ||
|
|
||
| * [SIG-TFX-Addons](https://github.com/tensorflow/tfx-addons) | ||
| * [Contributing Guidelines](https://github.com/tensorflow/tfx-addons/blob/main/CONTRIBUTING.md) | ||
| * [TensorFlow Code of Conduct](https://github.com/tensorflow/tfx-addons/blob/main/CODE_OF_CONDUCT.md) | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please describe what the results of the project will be, and a short introduction to what Airflow is and how it's used. You can also discuss the transition from having Airflow support included in TFX directly, versus having it as a separate install.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It can we installed and used as a separate package under tfxa/.