Migrate toolkit-data modules in library

modules/accelerators/industrial_tools/cdf_location_filter_asset_centric/classic/root.Asset.yaml

@@ -0,0 +1,9 @@
+- externalId: {{ root_asset }}
+  name: Asset Centric Location Filter Example
+  description: This is an example of an asset centric location filter
+  dataSetExternalId: {{ ds_asset }}
+- externalId: {{ root_asset2 }}
+  name: Asset Centric Location Filter Example 2
+  description: This is an example of an asset centric location filter 2
+  dataSetExternalId: {{ ds_asset }}
+

modules/accelerators/industrial_tools/cdf_location_filter_asset_centric/data_sets/data_sets.DataSet.yaml

@@ -0,0 +1,3 @@
+- externalId: {{ ds_asset }}
+  name: LocationFilter Asset DataSet
+  description: This is an example of a location filter asset centric data set

modules/accelerators/industrial_tools/cdf_location_filter_asset_centric/default.config.yaml

@@ -0,0 +1,4 @@
+ds_asset: ds_asset_location_filter
+root_asset: my_root_asset
+root_asset2: my_root_asset2
+ts_prefix: sensor

modules/accelerators/industrial_tools/cdf_location_filter_asset_centric/locations/individualFilters.LocationFilter.yaml

@@ -0,0 +1,19 @@
+externalId: asset_centric_individual_filter_location_filter
+name: Asset Centric Individual Filter Location Example
+description: This is an example of an asset centric individual filter location
+assetCentric:
+ assets:
+   # Typically, you would use either dataSetExternalIds, assetSubtreeIds, or externalIdPrefix
+   # not all of them. They are shown in this example for demonstration purposes.
+   dataSetExternalIds:
+     - '{{ ds_asset }}'
+   assetSubtreeIds:
+     - externalId: '{{  root_asset }}'
+   externalIdPrefix: pump_
+ timeseries:
+   dataSetExternalIds:
+     - '{{  ds_asset }}'
+   assetSubtreeIds:
+     - externalId: '{{  root_asset2 }}'
+   externalIdPrefix: '{{  ts_prefix }}'
+# events, sequences, and files are also supported.

modules/accelerators/industrial_tools/cdf_location_filter_asset_centric/locations/oneFilter.LocationFilter.yaml

@@ -0,0 +1,12 @@
+externalId: asset_centric_shared_location_filter
+name: Asset Centric Shared Location Example
+description: This is an example of an asset centric shared location
+assetCentric:
+  # Typically, you would use either dataSetExternalIds, assetSubtreeIds, or externalIdPrefix
+  # not all of them. They are shown in this example for demonstration purposes.
+ dataSetExternalIds:
+   - '{{  ds_asset }}'
+ assetSubtreeIds:
+   - externalId: '{{  root_asset }}'
+   - externalId: '{{  root_asset2 }}'
+ externalIdPrefix: pump_

modules/accelerators/industrial_tools/cdf_location_filter_asset_centric/module.toml

@@ -0,0 +1,4 @@
+[module]
+title = "Example location filter asset centric"
+id = "dp:acc:industrial_tools:cdf_location_filter_asset_centric"
+package_id = "dp:industrial_tools"

modules/accelerators/industrial_tools/cdf_location_filter_datamodel_based/data_models/instance.Space.yaml

@@ -0,0 +1,2 @@
+- space: {{ instance_space1 }}
+- space: {{ instance_space2 }}

modules/accelerators/industrial_tools/cdf_location_filter_datamodel_based/default.config.yaml

@@ -0,0 +1,2 @@
+instance_space1: my_instance_space1
+instance_space2: my_instance_space2

modules/accelerators/industrial_tools/cdf_location_filter_datamodel_based/locations/dataModel.LocationFilter.yaml

@@ -0,0 +1,17 @@
+externalId: view_based_location_filter
+name: Data Model Based Location Filter Example
+description: This is an example of a data model based location filter
+dataModels:
+ - externalId: CogniteProcessIndustries
+   space: cdf_idm
+   version: v1
+instanceSpaces:
+ - {{ instance_space1 }}
+ - {{ instance_space2 }}
+# If you have your own data model, with your own views, you can
+# specify what the views are representing. This is used by Cognite Applications.
+#views:
+#  - externalId: CogniteAsset
+#    space: cdf_cdm
+#    version: v1
+#    representsEntity: ASSET # Can be MAINTENANCE_ORDER, OPERATION, NOTIFICATION, ASSET

modules/accelerators/industrial_tools/cdf_location_filter_datamodel_based/module.toml

@@ -0,0 +1,4 @@
+[module]
+title = "Example location filter datamodel based"
+id = "dp:acc:industrial_tools:cdf_location_filter_datamodel_based"
+package_id = "dp:industrial_tools"

modules/accelerators/industrial_tools/cdf_search/module.toml

@@ -2,4 +2,4 @@
 title = "Search configuration with location filter"
 is_selected_by_default = false
 id = "dp:acc:industrial_tools:cdf_search"
-package_id = "dp:accelerators"
+package_id = "dp:industrial_tools"

modules/accelerators/inrobot/README.md

@@ -0,0 +1,41 @@
+# InRobot Module
+
+This module allows you to quickly set up an InRobot project. There are a few pre-requisites:
+
+- You must have an asset heirarchy already set up. Specifically you will need the external id of the root asset for
+  your location.
+- You must have a 3D model already uploaded in Fusion. Note its model id, its revision id, and its name.
+- You must have created 4 groups in your source system: Users*<Location>, Admin*<Location>, Robot*1*<Location>,
+ Run_Function_User. The naming does not matter specifically, but you may need to add more locations and/or robots in
+ the future, so it would be ideal to name the groups accordingly.
+- You must have created an app registration for the robot, and added the app registration to the robot user group. Note
+  the client id and the secret for the robot app registration.
+- You must have already activated the functions service for your project. This can be done in Fusion, and can take up
+  to 2 hours to become activated.
+
+This module is meant to be used in conjunction with the toolkit common module and the cdf_apm_base module.
+
+For now, until the next version release of toolkit, you must also enable the following toolkit feature flags:
+
+cdf features set fun-schedule --enable
+cdf features set robotics --enable
+
+## Configuration Variables
+
+Specific inrobot variables you will need to define in your config YAML file:
+
+| Variable Name                             | Description                                                                                                                  |
+|-------------------------------------------|------------------------------------------------------------------------------------------------------------------------------|
+| `first_root_asset_external_id`            | This is the asset external ID for your root asset.                                                                           |
+| `first_location`                          | A human readable name that will be included as part of different location-specific spaces and groups.                        |
+| `inrobot_admin_first_location_source_id`  | The ID for the admin group for the location.                                                                                 |
+| `inrobot_users_first_location_source_id`  | The ID for the users group for the location.                                                                                 |
+| `robot_1_first_location_source_id`        | The ID for the robot group for the location.                                                                                 |
+| `run_function_user_group_source_id`       | The ID for the run function group.                                                                                           |
+| `run_function_client_id`                  | The run function client ID (app registration in Azure).                                                                      |
+| `run_function_secret`                     | The secret for the run function app registration. This will be stored in your env file and should be referenced securely.    |
+| `robot_1_dataset_ext_id`                  | This is the data set for your robot. You can give this whatever value you want.                                              |
+| `three_d_model_name`                      | The name of the 3D model as named in Fusion.                                                                                 |
+| `three_d_type`                            | The type of 3D model. This will be either `THREEDMODEL` or `POINTCLOUD`.                                                     |
+| `three_d_model_id`                        | The model ID of your 3D model.                                                                                               |
+| `three_d_revision_id`                     | The revision ID of your 3D model.                                                                                            |

modules/accelerators/inrobot/cdf_inrobot_common/README.md

@@ -0,0 +1,53 @@
+# cdf_inrobot_common
+
+This module contains shared configurations across multiple locations for InRobot.
+
+## Auth
+
+The module creates one group that needs one matching group in the identity provider that the CDF
+project is configured with. The group is:
+
+- Run Function User. This role is used by our functions when they need to interact with CDF, for example, to add
+  annotations to images, to create timeseries, etc. This makes reference to the robot's data set, so each new robot
+  requires a new run function user.
+
+## Data models
+
+There is one space created in this module called cognite_app_data. This is the space where user and user profile
+data is stored. We also populate the APM_Config node with our default config, including the information about the
+location. If new locations are added, this node must be updated to include the new locations.
+
+## Data sets
+
+This module creates a new data set for the robot. All the robot data in CDF will be stored in this data set. The
+external id is specified in the config.yaml file, and this data_set is referred to in many other modules. Any new
+robots must have a new data set.
+
+## Functions
+
+This module contains four (4) functions that we deploy: Contextualize Robot Data, Gauge Reading, Get IR Data from
+IR Raw, and ThreeSixty. These are functions that run every minute on files with certain labels that
+the robot uploads to CDF.
+
+Because the function itself must be stored in a dataset and we use the dataset of the robot, a new function must be
+defined for each robot in the robots.functions.yaml file. Additionally, most schedules use the robot data set id as a
+data parameters - this means that a new schedule must be created for every new robot.
+
+## Labels
+
+This module creates two labels that are needed for InRobot to work: robot_gauge and read_ir. These are scoped to
+the robot's data set. This means that new labels must be created for each additional robot.
+
+## Robotics
+
+This module creates some robotics specific resources: RobotCapability and DataPostProcessing.
+
+Currently we support the following RobotCapability resources: acoustic_video, pt_ir_video, ptz_ir, ptz_video,
+ptz, threesixty, and threesixty_video.
+
+Currently we support the following DataPostProcessing resources:
+process_threesixty, read_dial_gauge, read_digital_gauge, read_level_gauge, read_valve.
+
+If there are any of these robot capabilities you do not want, you can remove the YAML file. Note that the
+process_threesixty data post processing requires the threesixty capability to be present. Similarly,
+the various gauge data post processing options require the ptz robot capability.

modules/accelerators/inrobot/cdf_inrobot_common/auth/run-function-user.Group.yaml

@@ -0,0 +1,89 @@
+name: "gp_Run_Function_User"
+sourceId: "{{run_function_user_group_source_id}}"
+metadata:
+  origin: "cdf-project-templates"
+capabilities:
+  - datasetsAcl:
+      actions:
+        - READ
+      scope:
+        all: {}
+  - groupsAcl:
+      actions:
+        - LIST
+      scope:
+        all: {}
+  - sessionsAcl:
+      actions:
+        - CREATE
+        - LIST
+      scope:
+        all: {}
+  - projectsAcl:
+      actions:
+        - LIST
+      scope:
+        all: {}
+  - functionsAcl:
+      actions:
+        - READ
+        - WRITE
+      scope:
+        all: {}
+  - labelsAcl:
+      actions:
+        - READ
+        - WRITE
+      scope:
+        all: {}
+  - assetsAcl:
+      actions:
+        - READ
+      scope:
+        all: {}
+  - timeSeriesAcl:
+      actions:
+        - READ
+        - WRITE
+      scope:
+        all: {}
+  - filesAcl:
+      actions:
+        - READ
+        - WRITE
+      scope:
+        all: {}
+  - visionModelAcl:
+      actions:
+        - READ
+      scope:
+        all: {}
+  - dataModelsAcl:
+      actions:
+        - READ
+      scope:
+        all: {}
+  - dataModelInstancesAcl:
+      actions:
+        - READ
+        - WRITE
+      scope:
+        all: {}
+  - eventsAcl:
+      actions:
+        - READ
+        - WRITE
+      scope:
+        datasetScope:
+          ids:
+            - "{{ robot_1_dataset_ext_id }}"
+  - roboticsAcl:
+      actions:
+        - READ
+      scope:
+        all: {}
+  - annotationsAcl:
+      actions:
+        - WRITE
+      scope:
+        all: {}

modules/accelerators/inrobot/cdf_inrobot_common/classic/robot.Label.yaml

@@ -0,0 +1,6 @@
+- name: robot_gauge
+  externalId: robot_gauge
+  dataSetExternalId: "{{ robot_1_dataset_ext_id }}"
+- name: read_ir
+  externalId: read_ir
+  dataSetExternalId: "{{ robot_1_dataset_ext_id }}"

modules/accelerators/inrobot/cdf_inrobot_common/data_models/cogniteAppData.space.yaml

@@ -0,0 +1,3 @@
+space: cognite_app_data
+name: cognite_app_data
+description: Space for User and User Preferences Data

modules/accelerators/inrobot/cdf_inrobot_common/data_models/inrobotApmConfig.node.yaml

@@ -0,0 +1,17 @@
+- space: "APM_Config"
+  externalId: "default-config"
+  sources:
+    - source:
+        space: APM_Config
+        externalId: APM_Config
+        version: "1"
+        type: view
+      properties:
+        featureConfiguration:
+          rootLocationConfigurations:
+            - assetExternalId: "{{ first_root_asset_external_id }}"
+              appDataInstanceSpace: sp_{{ first_location }}_app_data
+              sourceDataInstanceSpace: sp_{{ first_location }}_source_data
+        customerDataSpaceId: APM_SourceData
+        customerDataSpaceVersion: "1"
+        name: InRobot APM App Config

modules/accelerators/inrobot/cdf_inrobot_common/data_sets/robot_1.dataset.yaml

@@ -0,0 +1,2 @@
+- externalId: "{{ robot_1_dataset_ext_id }}"
+  name: "Robot 1 Dataset"

modules/accelerators/inrobot/cdf_inrobot_common/default.config.yaml

@@ -0,0 +1,3 @@
+run_function_user_group_source_id: <change_me>
+run_function_client_id: <change_me>
+run_function_secret: <change_me>

modules/accelerators/inrobot/cdf_inrobot_common/functions/fn_contextualize_robot_data/README.md

@@ -0,0 +1,30 @@
+# Contextualize robot files
+
+Function for contextualizing files that are uploaded from a robot to CDF. The function connects the file to
+the correct asset, and connects the correct time series to the file. If no time series exist, they are created.
+The function also assigns the correct action label to the file, indicating what action should be taken on the file,
+e.g., read dial gauge, read valve.
+
+List all files with metadata field `{"processed":"false"}` in input data set. If the image has an asset id in the
+"asset_id"  metadata field, check that the asset exists and add that as "assetId" to the file.
+
+If the image is a gauge reading image, check if the asset has a time series with label "GAUGE_TIME_SERIES". If it does,
+add metadata field "ts_external_id" to the image.
+If not, create the timeseries.
+
+In this case we could write metadata to the time series if we are able to read it in the gauge reader service. So the
+gauge reader would check if the TS has metadata or the file has metadata.
+
+1. If the TS has metadata, use ts metadata.
+2. If the TS does not have metadata and the file has metadata, use file metadata and write that metadata to TS.
+3. If the TS does not have metadata and the file does not have metadata complete metadata, use incomplete metadata set
+4. and read remainig metadata and write metadata to TS if successful
+
+(Assume always complete or no metadata on timeseries)
+
+With this we could instruct the user to take a very close and good picture of the gauge initially and that would work
+in some cases. Optionally, the user could take en image of the gauge, add correct metadata in the vision app and
+that metadata will be written to the timeseries.
+
+It will not be easy to change the metadata if we read wrong metadata initially. An option could be to not not write
+metadata to timeseries in case 3 (when metadata is read from image).

modules/accelerators/inrobot/cdf_inrobot_common/functions/fn_contextualize_robot_data/__init__.py

@@ -0,0 +1 @@
+"""Contextualize robot data."""