From c4a20671a01685746e07291912758601b82e5e99 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Wed, 18 Aug 2021 18:56:59 -0700 Subject: [PATCH 01/13] DOCS-11794 process build connector --- modules/ROOT/nav.adoc | 5 + .../ROOT/pages/rest-sdk/tutorial-design.adoc | 477 ++++++++++++++++++ .../ROOT/pages/rest-sdk/tutorial-intro.adoc | 13 + .../ROOT/pages/rest-sdk/tutorial-iterate.adoc | 160 ++++++ .../ROOT/pages/rest-sdk/tutorial-prereq.adoc | 9 + .../ROOT/pages/rest-sdk/tutorial-release.adoc | 122 +++++ 6 files changed, 786 insertions(+) create mode 100644 modules/ROOT/pages/rest-sdk/tutorial-design.adoc create mode 100644 modules/ROOT/pages/rest-sdk/tutorial-intro.adoc create mode 100644 modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc create mode 100644 modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc create mode 100644 modules/ROOT/pages/rest-sdk/tutorial-release.adoc diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 10194cdd..0140f5cb 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -2,6 +2,11 @@ * xref:index.adoc[Mule SDK] * xref:rest-sdk/rest-sdk-connectivity.adoc[REST SDK] ** xref:rest-sdk/project-setup.adoc[Getting Started] + ** xref:rest-sdk/tutorial-intro.adoc[Build a Connector from Start to Finish] + *** xref:rest-sdk/tutorial-prereq.adoc[Step 1: Prerequisites] + *** xref:rest-sdk/tutorial-design.adoc[Step 2: Design the Connector] + *** xref:rest-sdk/tutorial-iterate.adoc[Step 3: Iterate Over the Connector Design] + *** xref:rest-sdk/tutorial-release.adoc[Step 4: Release the Connector] ** xref:rest-sdk/customize-the-connector.adoc[Customize the Connector] *** xref:rest-sdk/customize-connector-descriptor.adoc[Content] *** xref:rest-sdk/configure-operations.adoc[Operations] diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc new file mode 100644 index 00000000..dc4df498 --- /dev/null +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -0,0 +1,477 @@ += Step 2: Design the Connector + +Start the process by designing the connector you would like to build. + +== Step 2.1: Choose an Anypoint Connector + +Choose an existing Anypoint Connector to modify, such as +https://github.com/mulesoft/mule4-trello-connector[Trello Connector]. + +== Step 2.2: Add L3 Capabilities + +Add L3 capabilities with customization-layer.yaml, in which you can modify the +original connector by setting names, adding descriptions, filtering operations, +changing operation names, or doing input and output transformations. Refer to +xref:rest-sdk/customize-the-connector.adoc[Customize the Connector] for detailed +information on this process. + +This https://github.com/mulesoft/mule4-trello-connector/blob/feature/citizen-connector/definitions/citizen-descriptor.yaml[link] +shows an example of the L3 capabilities added to Trello Connector. + +== Step 2.3: Generate Java Code + +Use customization-layer.yaml to generate a Connector with specific Java code. +The generated code is packaged inside `target/generated-sources`. + +Optionally, extend the code using custom Java. It is important to note that +in order to preserve the custom code when regenerating the connector, you must +refactor the code by moving the Java class that needs to be extended into an +overwrite folder. + +== Step 2.4: Generate Models + +Generate models using auto generation. Note that this generation takes +inputs from customization-layer.yaml, which eliminates the need to manually fill +in the gaps from generated models. + +. Get the extension model from Exchange and use it to create the customization +model. +. Use the same extension model with the new customization model to generate +the Composer and transformation models. +. Store the generated models for later use. + +== Step 2.5: Onboard the Connector to Exchange + +Onboard the connector to Exchange using the following steps: + +. Clone the connector code into your machine. ++ +`git clone git@github.com:mulesoft/mule4-trello-connector.git` +. Checkout the feature branch you want to release from. ++ +`git checkout feature/release/1.2.0` +. Create a branch based off it for the release following the `citizen/release/$VERSION` +naming convention. ++ +`git checkout -b citizen/release/1.2.0-DEV3` +. Go to the project's main pom.xml and change the version to the release version. ++ +`1.2.0-DEV3` +. Commit the change in this new branch. ++ +`git commit -m "Set version number for deploy"` +. Build this branch. ++ +`mvn clean install -Dmule.disableSdkComponentIgnore=true -Duser.timezone=UTC -DskipTests` +. Deploy to devx using the bash script. ++ +`./deploy.sh to devx` +. Verify it is in the public org in Anypoint Platform and Exchange in +https://devx.anypoint.mulesoft.com/exchange/. +. Tag the version. Begin by fetching the latest tags. ++ +`git fetch --tags` +. Check the latest tags to understand the standard in the previous version, +for example in mule-trello-connector-1.1.0. ++ +`git tag -l` +. Create a new tag following the `git tag $CONNECTOR_NAME-$VERSION` naming +convention. ++ +`git tag mule-trello-connector-1.2.0-DEV3` +. Push the tag using `git push origin $TAG_NAME`. ++ +`git push origin mule-trello-connector-1.2.0-DEV3` +. If you want to deploy to qax, use the `./deploy.sh to qax` bash script. If +you want to deploy to stgx, use the ``./deploy.sh to stgx` bash script. +. Validate in either Exchange QAX or Exchange STGX using any Anypoint Platform +user. + +== Step 2.6: Auto Generate Auth Schemas and Onboard to CFM + +Use Connection Schema and Connection Schema Metadata to generate a +connection creation UI. + +The CFM Connectivity Service generates a Connection Schema Model +based on an existing Connection Schema and an optional Connection Schema Metadata. + +To create a connection for a connector, the UI requests the Connection Schema +Model for that connector. The UI reads that model and generates a form +in the frontend for creating the connection. + +Ensure the Connection Schema is already onboarded before proceeding with the +following steps: + +. Part 1: Standard Connection UI (no manual work needed) +.. Once the Connection Schema is onboarded, the Connectivity Service generates +a standard Connection Schema Model, inferring as much information as possible +from the schema. +.. The UI retrieves that Connection Schema Model when the user wants to create +a connection for the schema's connector. +.. The UI reads the retrieved Connection Schema Model from the API's response +and generates a form with the fields needed for creating a connection for that +connector. + +. Part 2: Customize the Connection UI + +Customize the standard UI by providing a Connection Schema Metadata file +through a pull request. The following +https://github.com/mulesoft/citizen-platform-xapi-service/pull/278[pull request] +shows an example of that for Trello Connector. + +.. Create a JSON file with the Connection Schema Metadata. Refer to the following +https://github.com/mulesoft/cfm/blob/master/docs/connection-ui-autogeneration/creating-connection-schema-metadata.md[link] +to learn how to create a Connection Schema Metadata file. The file name must be +based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. +.. Validate the Connection Schema Metadata with the xAPI by pushing a `test/` +branch with your metadata and hitting the endpoint. ++ +`curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ +--header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` ++ +As a result, the connectionSchemaModel property appears in the response with the +generated Connection Schema Model with its Metadata. The customizations are shown +in the metadata of the endpoint's response. Alternatively, validate it against +the CFM Connectivity Service. + +. Part 3: Validate the Auto Generated Connection UI + +To validate that the Connection Creation UI is generated as expected, you must: + +.. Open the Composer UI. +.. Create a new flow. +.. Select the connector which the Connection Schema was created for. Note that if +the connector does not have any triggers, it will not appear here and you will need +to search for the connector inside the flow. +.. Once, you have selected your connector, a loading spinner may appear and +when it goes away the form is generated. + +If you added metadata and want to test it integrated with the UI, push a `test/` +or `integration/` branch with the added metadata and repeat the previous steps. + +== Step 2.7: Onboard onto OCS (Optional) + +Optionally onboard onto OCS using the following +https://github.com/mulesoft/ocs/blob/master/docs/common/onboarding-a-new-service-provider.md[steps]. + +[[onboardlower]] +== Step 2.8: Onboard Connector Onto Lower Environments + +Use the following steps to onboard either a new connector or a new version of +an already onboarded connector. + +=== Onboard a New Connector + +. Create a customization model, for example, by using the +`./cli template trello` command. This creates a new file inside the +customization folder: + +[source,yaml] +---- +name: trello +displayName: Trello +description: MuleSoft Composer connector for Trello. +iconUrl: www.fakeurl.com/icon.jpg +artifactGav: # TODO + groupId: ??? # Usually 'com.mulesoft.connectors' + artifactId: ??? # Probably either 'mule-trello-connector' or 'mule4-trello-connector' + version: ??? +configuration: # TODO + name: ??? # Usually 'config' + +sources: [] # TODO +# - name: ??? +# displayName: ??? # Optional +# description: ??? # Optional +# fields: # Optional +# - name: ??? +# displayName: ??? # Optional +# description: ??? # Optional +# visibility: SHOW +# # ... +# # ... + +operations: [] # TODO +# - name: ??? +# displayName: ??? # Optional +# description: ??? # Optional +# fields: # Optional +# - name: ??? +# displayName: ??? # Optional +# description: ??? # Optional +# visibility: SHOW +# # ... +# # ... +---- + +You can remove the `TODO`s and placeholder `???`s. For now, fill in the +`artifactGav` with the information of the specialist Connector that +corresponds to the Composer Connector. The following example is for Trello +Connector: + +[source,yaml] +---- +# ... +artifactGav: + groupId: com.mulesoft.connectors + artifactId: mule-trello-connector + version: 1.2.0 +# ... +---- + +. Optionally, download the extension model to use as a reference while completing +the rest of the customization model by using the `./cli extension trello` command. +This downloads the file into a JSON file. +. Fill in the rest of the customization model. Use the technical specification +and extension model to get the names of the operations and sources and their +respective fields. + +[source,yaml] +---- +name: trello +displayName: Trello +description: MuleSoft Composer connector for Trello. +iconUrl: www.fakeurl.com/icon.jpg +artifactGav: + groupId: com.mulesoft.connectors + artifactId: mule-trello-connector + version: 1.2.0 +configuration: + name: config + +sources: + - name: source1 + displayName: Source 1 + description: Source 1's description. + fields: + - name: field1 + displayName: Field 1 + description: Field 1's description. + visibility: SHOW + # ... + # ... + +operations: + - name: operation1 + displayName: Operation 1 + description: Operation 1's description. + fields: + - name: field1 + displayName: Field 1 + description: Field 1's description. + visibility: SHOW + # ... + # ... +---- + +Run validations of the customization model while building it by using the +`./cli validate foo` command. + + +. Generate models using the cli tool. + +`./cli connector trello +./cli transformation trello` + +Alternatively, use `./cli both trello` to generate both the conenctor and +transformation. + +. Patch the generated output. +.. Rename the version folder. +.. Add the *schedulingStrategy* transformation. For each source in the +`transformation_model.json`, add an entry on the transformations entry like this: + +[source,json5] +---- +{ + "type": "multipleTypedField", + "name": "schedulingStrategy", + "path": "General.schedulingStrategy", + "dynamic": false, + "typeId": "FixedFrequencyScheduler", + "value": { + "FixedFrequencyScheduler": { + "frequency": 15, + "timeUnit": "SECONDS", + "startDelay": 0 + }, + "CronScheduler": { + "expression": "*****", + "timeZone": "-3 GTM" + } + }, + "subtypes": { + "FixedFrequencyScheduler": "org.mule.runtime.core.api.source.scheduler.FixedFrequencyScheduler", + "CronScheduler": "org.mule.runtime.core.api.source.scheduler.CronScheduler" + } +} +---- + + +The final transformation would then look like this: + +[source,json5] +---- +{ + // ... + "components": { + // ... + "source1": { + "type": "source", + "transformations": [ + // ... + { + "type": "multipleTypedField", + "name": "schedulingStrategy", + "path": "General.schedulingStrategy", + "dynamic": false, + "typeId": "FixedFrequencyScheduler", + "value": { + "FixedFrequencyScheduler": { + "frequency": 15, + "timeUnit": "SECONDS", + "startDelay": 0 + }, + "CronScheduler": { + "expression": "*****", + "timeZone": "-3 GTM" + } + }, + "subtypes": { + "FixedFrequencyScheduler": "org.mule.runtime.core.api.source.scheduler.FixedFrequencyScheduler", + "CronScheduler": "org.mule.runtime.core.api.source.scheduler.CronScheduler" + } + } + // ... + ] + } + // ... + } + // ... +} +---- + + +. Expose the new connector by adding the relevant entries to +https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/main/resources/application.yml[application.yaml] +under the path's `connectors.environments..foo` and adding the +new version. + +[source,yaml] +---- +# ... +connectors: + environments: + KDEV: + # ... + trello: 1.1.0 + KQA: + # ... + trello: 1.2.0 + # ... +# ... +---- + +. Add the connector to the unit tests, for which you should add similar entries +to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: + +[source,yaml] +---- +KDEV: + # ... + trello: 1.1.0 +KQA: + # ... + trello: 1.2.0 +# ... +---- + +. Run `mvn clean test` to validate that the build passes. +. Create a pull request for your newly onboarded connector. +. Merge the pull request. + + +=== Onboard a New Version of an Already Onboarded Connector + +The following steps apply only when: + +* A schema already exists that is valid for a connector. +* A new version of the same connector has been released. +* The new version of the connector is compatible with the CFM schema. To check +schema compatibility: + ** Verify that there are no changes in the `connectionProviders` part of the + connector's extension model. Note that it is okay if other parts of the model + have been modified. + ** If there are changes in the `connectionProviders` part, verify that the + connection provider with the same name as the schema's `labels.connectionProvider` + has not been modified. + ** If the matching connection provider has been modified, identify the + differences and reach out the CFM team to further analyze the case. + + If all of the above are true, you can proceed to onboard the new version using + the following steps: + + . Open the schema. + . Under `assets`, add the GAV of the new connector version. For example, for + the following schema: + + [source,yaml] + ---- + { + "groupId": "com.mulesoft.schemas", + "artifactId": "mule-trello-connector-platform-oauth", + "version": "1.1", + ... + "assets": [ + { + "groupId": "com.mulesoft.connectors", + "assetId": "mule4-trello-connector", + "version": "1.1.0" + } + ], + ... + } +---- + +To onboard connector version `1.2.0`, the schema would look like this: + + +[source,yaml] +---- +{ + "groupId": "com.mulesoft.schemas", + "artifactId": "mule-trello-connector-platform-oauth", + "version": "1.1", + ... + "assets": [ + { + "groupId": "com.mulesoft.connectors", + "assetId": "mule4-trello-connector", + "version": "1.1.0" + }, + { + "groupId": "com.mulesoft.connectors", + "assetId": "mule4-trello-connector", + "version": "1.2.0" + } + ], + ... +} +---- + +Note that some schema fields were omitted with `...` for brevity. Only the +assets section should be modified. + +. Get your changes deployed. + +* For lower environments (kdev, kqa): Check out branch `integration/main`, commit +your changes, and push. Your changes will be automatically deployed. +* For high environments (kstg, kprod, kprod-eu): Branch out from `master`, push +your changes, and create a pull request. You will need approval from the CFM team to merge. +After merging, merge `master` back into `integration/main` to prevent regressions +to older connector versions in lower environments. + +== What's Next? + +Now that you have fully designed and deployed your connector, you can iterate +over the design to add more features or fix bugs. diff --git a/modules/ROOT/pages/rest-sdk/tutorial-intro.adoc b/modules/ROOT/pages/rest-sdk/tutorial-intro.adoc new file mode 100644 index 00000000..878fd986 --- /dev/null +++ b/modules/ROOT/pages/rest-sdk/tutorial-intro.adoc @@ -0,0 +1,13 @@ += Build a Connector from Start to Finish + +REST SDK enables you to build a connector using an API specification within +just a few simple steps. + +The following topics show how to build a connector after the Anypoint Connector +development is complete. For more information on how to build Anypoint Connectors, +refer to xref:rest-sdk/project-setup.adoc[Getting Started]. + +. xref:rest-sdk/tutorial-prereq.adoc[Prerequisites] +. xref:rest-sdk/tutorial-design.adoc[Design the Connector] +. xref:rest-sdk/tutorial-iterate.adoc[Iterate Over the Connector Design] +. xref:rest-sdk/tutorial-release.adoc[Release the Connector] diff --git a/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc new file mode 100644 index 00000000..8fbd6b36 --- /dev/null +++ b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc @@ -0,0 +1,160 @@ += Step 3: Iterate Over the Connector Design + +Iterate over the connector design by using a new version of the API specification +to add more features via metadata or Java code and fix bugs. + +== Step 3.1: Add Desired Changes + +To add your desired changes, you can either upload a new API specification or +make changes to customization-layer.yaml (L3). Custom code changes moved to the +overwrite folder will be preserved. + +== Step 3.2: Compare your Changes + +For conflicts and changes with custom code, use diffs view in IntelliJ to compare +the differences and manually fix them. + +== Step 3.3: Update the Connector Onto Lower Environments + +Update the existing connector using the following steps: + +. Go to the customizations folder. +. Update the version. + +[source,yaml] +---- +# ... +artifactGav: + groupId: com.mulesoft.connectors + artifactId: mule-trello-connector + version: 1.2.0 # <- Change this +# ... +---- + +. Make any necessary changes to the model, if any. +. Generate models using the cli tool. + +`./cli connector trello +./cli transformation trello` + +Alternatively, use `./cli both trello` to generate both the conenctor and +transformation. + +. Patch the generated output. +.. Rename the version folder. +.. Add the *schedulingStrategy* transformation. For each source in the +`transformation_model.json`, add an entry on the transformations entry like this: + +[source,json5] +---- +{ + "type": "multipleTypedField", + "name": "schedulingStrategy", + "path": "General.schedulingStrategy", + "dynamic": false, + "typeId": "FixedFrequencyScheduler", + "value": { + "FixedFrequencyScheduler": { + "frequency": 15, + "timeUnit": "SECONDS", + "startDelay": 0 + }, + "CronScheduler": { + "expression": "*****", + "timeZone": "-3 GTM" + } + }, + "subtypes": { + "FixedFrequencyScheduler": "org.mule.runtime.core.api.source.scheduler.FixedFrequencyScheduler", + "CronScheduler": "org.mule.runtime.core.api.source.scheduler.CronScheduler" + } +} +---- + + +The final transformation would then look like this: + +[source,json5] +---- +{ + // ... + "components": { + // ... + "source1": { + "type": "source", + "transformations": [ + // ... + { + "type": "multipleTypedField", + "name": "schedulingStrategy", + "path": "General.schedulingStrategy", + "dynamic": false, + "typeId": "FixedFrequencyScheduler", + "value": { + "FixedFrequencyScheduler": { + "frequency": 15, + "timeUnit": "SECONDS", + "startDelay": 0 + }, + "CronScheduler": { + "expression": "*****", + "timeZone": "-3 GTM" + } + }, + "subtypes": { + "FixedFrequencyScheduler": "org.mule.runtime.core.api.source.scheduler.FixedFrequencyScheduler", + "CronScheduler": "org.mule.runtime.core.api.source.scheduler.CronScheduler" + } + } + // ... + ] + } + // ... + } + // ... +} +---- + + +. Expose the new connector by adding the relevant entries to +https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/main/resources/application.yml[application.yaml] +under the path's `connectors.environments..foo` and adding the +new version. + +[source,yaml] +---- +# ... +connectors: + environments: + KDEV: + # ... + trello: 1.2.0 + KQA: + # ... + trello: 1.3.0 + # ... +# ... +---- + +. Update the connector on the unit tests, for which you should add similar entries +to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: + +[source,yaml] +---- +KDEV: + # ... + trello: 1.2.0 +KQA: + # ... + trello: 1.3.0 +# ... +---- + +. Run `mvn clean test` to validate that the build passes. +. Create a pull request for your newly onboarded connector. +. Merge the pull request. + +== What's Next? + +After adding your final touches while creating new features or fixing bugs, you +can start the process to release the connector to staging and production. diff --git a/modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc b/modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc new file mode 100644 index 00000000..81f10e0e --- /dev/null +++ b/modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc @@ -0,0 +1,9 @@ += Step 1. Prerequisites + +Before you start building your connector, verify that you have the basic tools +needed. + +* https://www.jetbrains.com/idea/download/#section=mac[IntelliJ IDEA] ++ +You need IntelliJ IDEA to begin this process. Install the IntelliJ plugin to +unlock autocomplete and the project structure. diff --git a/modules/ROOT/pages/rest-sdk/tutorial-release.adoc b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc new file mode 100644 index 00000000..3d7d0239 --- /dev/null +++ b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc @@ -0,0 +1,122 @@ += Step 4: Release the Connector + +Release the connector to staging and production by following these steps. + +== Step 4.1: Promote the Connector to Staging + +. Ensure the connector is onboarded onto lower environments. Refer to +xref:rest-sdk/tutorial-design.adoc#onboardlower[Onboard Connector Onto Lower Environments]. +. Expose the connector by adding or changing the relevant entry on +https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/main/resources/application.yml[application.yml] +under the path `connectors.environments.KSTG.foo` and putting in the version +that should be promoted. + +[source,yaml] +---- +# ... +connectors: + environments: + # ... + KSTG: + # ... + trello: 1.3.0 + # ... +# ... +---- + +. Add or update the connector to the unit tests, for which you should add a similar entry +to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: + +[source,yaml] +---- +# ... +KSTG: + # ... + trello: 1.3.0 +# ... +---- + +. Run `mvn clean test` to validate that the build passes. +. Create a pull request for your newly promoted connector. +. Merge the pull request. + +== Step 4.2: Release the Connector to Production + +. Ensure the connector is onboarded onto staging. +. Expose the connector by adding or changing the relevant entries on https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/main/resources/application.yml[application.yml], +under the paths connectors.environments..foo and putting in the +version that should be released. + +[source,yaml] +---- +# ... +connectors: + environments: + # ... + KPROD_EU: + # ... + trello: 1.2.0 + KPROD: + # ... + trello: 1.3.0 +# ... +---- + +. Add the connector to the unit tests, for which you should add similar entries +to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: + +[source,yaml] +---- +# ... +KPROD_EU: + # ... + trello: 1.2.0 +KPROD: + # ... + trello: 1.3.0 +---- + +. Create or update the corresponding prod test for the connector. Changes should +be similar to these diffs: + +https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/mocks/ConnectorsMocks.java[ConnectorsMocks.java] + +[source, java, linenums] +---- +// ... ++ ++ public static ListedConnector getFooConnector() { ++ return ListedConnector.builder("trello", "Trello", "MuleSoft Composer connector for Trello.", "1.3.0") ++ .withConnection(???) // Whether the connector requires a connection (see CONNECTORS_WITHOUT_CONNECTION in ./citizen-platform-connectors-models-service/src/main/java/com/mulesoft/citizen/platform/connectors_models/configuration/RegisteredConnectorConfig.java) ++ .withTrigger(???) // Whether the connector has sources. ++ .withAction(???) // Whether the connector has operations. ++ .build(); ++ } + // ... +---- + +https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/controllers/ConnectorsControllerTest.java[ConnectorsControllerTest.java] + +[source, java, linenums] +---- +// ... + + @Test + public void connectorsEndpointReturnsAListOfConnectorsWithAllAvailableConnectors() throws Exception { + // ... + List expectedConnectorsList = Arrays.asList( + // ... +- ConnectorsMocks.getOtherConnector() // Deleted ++ ConnectorsMocks.getOtherConnector(), ++ ConnectorsMocks.getTrelloConnector() // <- This is our connector + ); + + // ... + } + + // ... +---- + +. Run `mvn clean test` to validate that the build passes. +. Create a pull request for your soon to be released connector. +. Merge the pull request. From d727580ed278ca29ae4994c1c604575a814fae1c Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Wed, 18 Aug 2021 20:07:55 -0700 Subject: [PATCH 02/13] formatting --- .../ROOT/pages/rest-sdk/tutorial-design.adoc | 76 +++++++++---------- .../ROOT/pages/rest-sdk/tutorial-iterate.adoc | 12 +-- 2 files changed, 38 insertions(+), 50 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index dc4df498..ae35b155 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -103,27 +103,28 @@ Ensure the Connection Schema is already onboarded before proceeding with the following steps: . Part 1: Standard Connection UI (no manual work needed) -.. Once the Connection Schema is onboarded, the Connectivity Service generates + +* Once the Connection Schema is onboarded, the Connectivity Service generates a standard Connection Schema Model, inferring as much information as possible from the schema. -.. The UI retrieves that Connection Schema Model when the user wants to create +* The UI retrieves that Connection Schema Model when the user wants to create a connection for the schema's connector. -.. The UI reads the retrieved Connection Schema Model from the API's response +* The UI reads the retrieved Connection Schema Model from the API's response and generates a form with the fields needed for creating a connection for that connector. -. Part 2: Customize the Connection UI +. Part 2: Customize the Connection UI. -Customize the standard UI by providing a Connection Schema Metadata file -through a pull request. The following +Customize the standard UI by providing +a Connection Schema Metadata file through a pull request. The following https://github.com/mulesoft/citizen-platform-xapi-service/pull/278[pull request] shows an example of that for Trello Connector. -.. Create a JSON file with the Connection Schema Metadata. Refer to the following +* Create a JSON file with the Connection Schema Metadata. Refer to the following https://github.com/mulesoft/cfm/blob/master/docs/connection-ui-autogeneration/creating-connection-schema-metadata.md[link] to learn how to create a Connection Schema Metadata file. The file name must be based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. -.. Validate the Connection Schema Metadata with the xAPI by pushing a `test/` +* Validate the Connection Schema Metadata with the xAPI by pushing a `test/` branch with your metadata and hitting the endpoint. + `curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ @@ -138,12 +139,12 @@ the CFM Connectivity Service. To validate that the Connection Creation UI is generated as expected, you must: -.. Open the Composer UI. -.. Create a new flow. -.. Select the connector which the Connection Schema was created for. Note that if +* Open the Composer UI. +* Create a new flow. +* Select the connector which the Connection Schema was created for. Note that if the connector does not have any triggers, it will not appear here and you will need to search for the connector inside the flow. -.. Once, you have selected your connector, a loading spinner may appear and +* Once, you have selected your connector, a loading spinner may appear and when it goes away the form is generated. If you added metadata and want to test it integrated with the UI, push a `test/` @@ -164,7 +165,10 @@ an already onboarded connector. . Create a customization model, for example, by using the `./cli template trello` command. This creates a new file inside the -customization folder: +customization folder. You can remove all of the `TODO` and placeholder `???`. +For now, fill in the `artifactGav` with the information of the specialist Connector that +corresponds to the Composer Connector. The following example is for Trello +Connector: [source,yaml] ---- @@ -204,11 +208,6 @@ operations: [] # TODO # # ... ---- -You can remove the `TODO`s and placeholder `???`s. For now, fill in the -`artifactGav` with the information of the specialist Connector that -corresponds to the Composer Connector. The following example is for Trello -Connector: - [source,yaml] ---- # ... @@ -264,22 +263,16 @@ operations: # ... ---- -Run validations of the customization model while building it by using the +. Run validations of the customization model while building it by using the `./cli validate foo` command. - - -. Generate models using the cli tool. - -`./cli connector trello -./cli transformation trello` - -Alternatively, use `./cli both trello` to generate both the conenctor and -transformation. - +. Generate models using the cli tool by running `./cli connector trello` and then +`./cli transformation trello`. Alternatively, run `./cli both trello` to generate +both the conenctor and transformation. . Patch the generated output. .. Rename the version folder. .. Add the *schedulingStrategy* transformation. For each source in the -`transformation_model.json`, add an entry on the transformations entry like this: +`transformation_model.json`, add an entry on the transformations entry like this. +The following examples show the entry and the final transformation: [source,json5] ---- @@ -307,9 +300,6 @@ transformation. } ---- - -The final transformation would then look like this: - [source,json5] ---- { @@ -408,15 +398,16 @@ schema compatibility: ** If the matching connection provider has been modified, identify the differences and reach out the CFM team to further analyze the case. - If all of the above are true, you can proceed to onboard the new version using - the following steps: - . Open the schema. - . Under `assets`, add the GAV of the new connector version. For example, for +If all of the above are true, you can proceed to onboard the new version using +the following steps: + +. Open the schema. +. Under `assets`, add the GAV of the new connector version. For example, for the following schema: - [source,yaml] - ---- +[source,yaml] +---- { "groupId": "com.mulesoft.schemas", "artifactId": "mule-trello-connector-platform-oauth", @@ -433,7 +424,10 @@ schema compatibility: } ---- -To onboard connector version `1.2.0`, the schema would look like this: + +. To onboard connector version `1.2.0`, the schema would look like this. Note that +some schema fields were omitted with `...` for brevity. Only the +assets section should be modified: [source,yaml] @@ -459,8 +453,6 @@ To onboard connector version `1.2.0`, the schema would look like this: } ---- -Note that some schema fields were omitted with `...` for brevity. Only the -assets section should be modified. . Get your changes deployed. diff --git a/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc index 8fbd6b36..4fc399bf 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc @@ -32,13 +32,9 @@ artifactGav: ---- . Make any necessary changes to the model, if any. -. Generate models using the cli tool. - -`./cli connector trello -./cli transformation trello` - -Alternatively, use `./cli both trello` to generate both the conenctor and -transformation. +. Generate models using the cli tool by running `./cli connector trello` and then +`./cli transformation trello`. Alternatively, run `./cli both trello` to generate +both the conenctor and transformation. . Patch the generated output. .. Rename the version folder. @@ -157,4 +153,4 @@ KQA: == What's Next? After adding your final touches while creating new features or fixing bugs, you -can start the process to release the connector to staging and production. +can start the process to release the connector to staging and production. From 13743adad998e08ffb4b3d2e944b31ad158e75af Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Wed, 18 Aug 2021 21:01:23 -0700 Subject: [PATCH 03/13] formatting --- .../ROOT/pages/rest-sdk/tutorial-design.adoc | 24 ++++++++++--------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index ae35b155..545e5000 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -120,31 +120,33 @@ a Connection Schema Metadata file through a pull request. The following https://github.com/mulesoft/citizen-platform-xapi-service/pull/278[pull request] shows an example of that for Trello Connector. -* Create a JSON file with the Connection Schema Metadata. Refer to the following +** Create a JSON file with the Connection Schema Metadata. Refer to the following https://github.com/mulesoft/cfm/blob/master/docs/connection-ui-autogeneration/creating-connection-schema-metadata.md[link] to learn how to create a Connection Schema Metadata file. The file name must be based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. -* Validate the Connection Schema Metadata with the xAPI by pushing a `test/` -branch with your metadata and hitting the endpoint. +** Validate the Connection Schema Metadata with the xAPI by pushing a `test/` +branch with your metadata and hitting the endpoint. As a result, the +connectionSchemaModel property appears in the response with the +generated Connection Schema Model with its Metadata. The customizations are shown +in the metadata of the endpoint's response. Alternatively, validate it against +the CFM Connectivity Service. + `curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ --header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` + -As a result, the connectionSchemaModel property appears in the response with the -generated Connection Schema Model with its Metadata. The customizations are shown -in the metadata of the endpoint's response. Alternatively, validate it against -the CFM Connectivity Service. + + . Part 3: Validate the Auto Generated Connection UI To validate that the Connection Creation UI is generated as expected, you must: -* Open the Composer UI. -* Create a new flow. -* Select the connector which the Connection Schema was created for. Note that if +** Open the Composer UI. +** Create a new flow. +** Select the connector which the Connection Schema was created for. Note that if the connector does not have any triggers, it will not appear here and you will need to search for the connector inside the flow. -* Once, you have selected your connector, a loading spinner may appear and +** Once, you have selected your connector, a loading spinner may appear and when it goes away the form is generated. If you added metadata and want to test it integrated with the UI, push a `test/` From 808593d36950b84a25f6763a3efd941c9bddb888 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 09:03:10 -0700 Subject: [PATCH 04/13] formatting --- .../ROOT/pages/rest-sdk/tutorial-design.adoc | 76 +++++++++++-------- .../ROOT/pages/rest-sdk/tutorial-prereq.adoc | 4 +- 2 files changed, 45 insertions(+), 35 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index 545e5000..7496c12e 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -20,7 +20,7 @@ shows an example of the L3 capabilities added to Trello Connector. == Step 2.3: Generate Java Code -Use customization-layer.yaml to generate a Connector with specific Java code. +Use customization-layer.yaml to generate a connector with specific Java code. The generated code is packaged inside `target/generated-sources`. Optionally, extend the code using custom Java. It is important to note that @@ -103,55 +103,57 @@ Ensure the Connection Schema is already onboarded before proceeding with the following steps: . Part 1: Standard Connection UI (no manual work needed) - -* Once the Connection Schema is onboarded, the Connectivity Service generates ++ +.. Once the Connection Schema is onboarded, the Connectivity Service generates a standard Connection Schema Model, inferring as much information as possible from the schema. -* The UI retrieves that Connection Schema Model when the user wants to create +.. The UI retrieves that Connection Schema Model when the user wants to create a connection for the schema's connector. -* The UI reads the retrieved Connection Schema Model from the API's response +.. The UI reads the retrieved Connection Schema Model from the API's response and generates a form with the fields needed for creating a connection for that connector. - ++ . Part 2: Customize the Connection UI. - ++ Customize the standard UI by providing a Connection Schema Metadata file through a pull request. The following https://github.com/mulesoft/citizen-platform-xapi-service/pull/278[pull request] shows an example of that for Trello Connector. - -** Create a JSON file with the Connection Schema Metadata. Refer to the following ++ +.. Create a JSON file with the Connection Schema Metadata. Refer to the following https://github.com/mulesoft/cfm/blob/master/docs/connection-ui-autogeneration/creating-connection-schema-metadata.md[link] to learn how to create a Connection Schema Metadata file. The file name must be based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. -** Validate the Connection Schema Metadata with the xAPI by pushing a `test/` -branch with your metadata and hitting the endpoint. As a result, the -connectionSchemaModel property appears in the response with the -generated Connection Schema Model with its Metadata. The customizations are shown -in the metadata of the endpoint's response. Alternatively, validate it against -the CFM Connectivity Service. +.. Validate the Connection Schema Metadata with the xAPI by pushing a `test/` +branch with your metadata and hitting the endpoint. + `curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ --header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` + +As a result, the +connectionSchemaModel property appears in the response with the +generated Connection Schema Model with its Metadata. The customizations are shown +in the metadata of the endpoint's response. Alternatively, validate it against +the CFM Connectivity Service. ++ . Part 3: Validate the Auto Generated Connection UI To validate that the Connection Creation UI is generated as expected, you must: - -** Open the Composer UI. -** Create a new flow. -** Select the connector which the Connection Schema was created for. Note that if ++ +.. Open the Composer UI. +.. Create a new flow. +.. Select the connector which the Connection Schema was created for. Note that if the connector does not have any triggers, it will not appear here and you will need to search for the connector inside the flow. -** Once, you have selected your connector, a loading spinner may appear and +.. Once, you have selected your connector, a loading spinner may appear and when it goes away the form is generated. If you added metadata and want to test it integrated with the UI, push a `test/` or `integration/` branch with the added metadata and repeat the previous steps. - ++ == Step 2.7: Onboard onto OCS (Optional) Optionally onboard onto OCS using the following @@ -167,10 +169,7 @@ an already onboarded connector. . Create a customization model, for example, by using the `./cli template trello` command. This creates a new file inside the -customization folder. You can remove all of the `TODO` and placeholder `???`. -For now, fill in the `artifactGav` with the information of the specialist Connector that -corresponds to the Composer Connector. The following example is for Trello -Connector: +customization folder: [source,yaml] ---- @@ -209,6 +208,12 @@ operations: [] # TODO # # ... # # ... ---- ++ +You can remove all of the `TODO` and placeholder `???`. +For now, fill in the `artifactGav` with the information of the specialist Connector that +corresponds to the Composer Connector. The following example is for Trello +Connector: ++ [source,yaml] ---- @@ -219,7 +224,7 @@ artifactGav: version: 1.2.0 # ... ---- - ++ . Optionally, download the extension model to use as a reference while completing the rest of the customization model by using the `./cli extension trello` command. This downloads the file into a JSON file. @@ -264,7 +269,7 @@ operations: # ... # ... ---- - ++ . Run validations of the customization model while building it by using the `./cli validate foo` command. . Generate models using the cli tool by running `./cli connector trello` and then @@ -273,8 +278,7 @@ both the conenctor and transformation. . Patch the generated output. .. Rename the version folder. .. Add the *schedulingStrategy* transformation. For each source in the -`transformation_model.json`, add an entry on the transformations entry like this. -The following examples show the entry and the final transformation: +`transformation_model.json`, add an entry on the transformations entry like this: [source,json5] ---- @@ -302,6 +306,11 @@ The following examples show the entry and the final transformation: } ---- ++ + +The final transformation would then look like this: ++ + [source,json5] ---- { @@ -342,7 +351,7 @@ The following examples show the entry and the final transformation: // ... } ---- - ++ . Expose the new connector by adding the relevant entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/main/resources/application.yml[application.yaml] @@ -364,6 +373,7 @@ connectors: # ... ---- ++ . Add the connector to the unit tests, for which you should add similar entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: @@ -377,7 +387,7 @@ KQA: trello: 1.2.0 # ... ---- - ++ . Run `mvn clean test` to validate that the build passes. . Create a pull request for your newly onboarded connector. . Merge the pull request. @@ -425,7 +435,7 @@ the following steps: ... } ---- - ++ . To onboard connector version `1.2.0`, the schema would look like this. Note that some schema fields were omitted with `...` for brevity. Only the @@ -454,7 +464,7 @@ assets section should be modified: ... } ---- - ++ . Get your changes deployed. diff --git a/modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc b/modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc index 81f10e0e..8018d8cb 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-prereq.adoc @@ -1,9 +1,9 @@ = Step 1. Prerequisites -Before you start building your connector, verify that you have the basic tools +Before you start building your new connector, verify that you have the basic tools needed. * https://www.jetbrains.com/idea/download/#section=mac[IntelliJ IDEA] + You need IntelliJ IDEA to begin this process. Install the IntelliJ plugin to -unlock autocomplete and the project structure. +unlock autocomplete and the project structure. From ccec2d769e8f011048615c1a598743b068a5f2fb Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 09:04:53 -0700 Subject: [PATCH 05/13] formatting --- modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc index 4fc399bf..9ea18c1f 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc @@ -30,12 +30,11 @@ artifactGav: version: 1.2.0 # <- Change this # ... ---- - ++ . Make any necessary changes to the model, if any. . Generate models using the cli tool by running `./cli connector trello` and then `./cli transformation trello`. Alternatively, run `./cli both trello` to generate both the conenctor and transformation. - . Patch the generated output. .. Rename the version folder. .. Add the *schedulingStrategy* transformation. For each source in the @@ -66,9 +65,10 @@ both the conenctor and transformation. } } ---- - ++ The final transformation would then look like this: ++ [source,json5] ---- @@ -110,7 +110,7 @@ The final transformation would then look like this: // ... } ---- - ++ . Expose the new connector by adding the relevant entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/main/resources/application.yml[application.yaml] @@ -131,7 +131,7 @@ connectors: # ... # ... ---- - ++ . Update the connector on the unit tests, for which you should add similar entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: @@ -145,7 +145,7 @@ KQA: trello: 1.3.0 # ... ---- - ++ . Run `mvn clean test` to validate that the build passes. . Create a pull request for your newly onboarded connector. . Merge the pull request. From 2d58ce55f8005d7ad873e25aeeadc8f5cb231c72 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 09:06:01 -0700 Subject: [PATCH 06/13] formatting --- .../ROOT/pages/rest-sdk/tutorial-release.adoc | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-release.adoc b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc index 3d7d0239..2a16ba3f 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-release.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc @@ -23,7 +23,7 @@ connectors: # ... # ... ---- - ++ . Add or update the connector to the unit tests, for which you should add a similar entry to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: @@ -35,7 +35,7 @@ KSTG: trello: 1.3.0 # ... ---- - ++ . Run `mvn clean test` to validate that the build passes. . Create a pull request for your newly promoted connector. . Merge the pull request. @@ -61,7 +61,7 @@ connectors: trello: 1.3.0 # ... ---- - ++ . Add the connector to the unit tests, for which you should add similar entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: @@ -75,12 +75,12 @@ KPROD: # ... trello: 1.3.0 ---- - ++ . Create or update the corresponding prod test for the connector. Changes should be similar to these diffs: https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/mocks/ConnectorsMocks.java[ConnectorsMocks.java] - ++ [source, java, linenums] ---- // ... @@ -94,9 +94,9 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast + } // ... ---- - ++ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/controllers/ConnectorsControllerTest.java[ConnectorsControllerTest.java] - ++ [source, java, linenums] ---- // ... @@ -116,7 +116,7 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast // ... ---- - ++ . Run `mvn clean test` to validate that the build passes. . Create a pull request for your soon to be released connector. -. Merge the pull request. +. Merge the pull request. From 0b106d11b3053cf16ecefd4d3395f6b6f433e0e0 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 09:51:50 -0700 Subject: [PATCH 07/13] Update tutorial-design.adoc --- .../ROOT/pages/rest-sdk/tutorial-design.adoc | 31 ++++++++++++++++--- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index 7496c12e..d5ae5c89 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -113,25 +113,29 @@ a connection for the schema's connector. and generates a form with the fields needed for creating a connection for that connector. + + . Part 2: Customize the Connection UI. + + Customize the standard UI by providing a Connection Schema Metadata file through a pull request. The following https://github.com/mulesoft/citizen-platform-xapi-service/pull/278[pull request] shows an example of that for Trello Connector. + + .. Create a JSON file with the Connection Schema Metadata. Refer to the following https://github.com/mulesoft/cfm/blob/master/docs/connection-ui-autogeneration/creating-connection-schema-metadata.md[link] to learn how to create a Connection Schema Metadata file. The file name must be based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. .. Validate the Connection Schema Metadata with the xAPI by pushing a `test/` branch with your metadata and hitting the endpoint. + + `curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ --header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` + - ++ As a result, the connectionSchemaModel property appears in the response with the generated Connection Schema Model with its Metadata. The customizations are shown @@ -141,8 +145,10 @@ the CFM Connectivity Service. . Part 3: Validate the Auto Generated Connection UI ++ To validate that the Connection Creation UI is generated as expected, you must: + + .. Open the Composer UI. .. Create a new flow. .. Select the connector which the Connection Schema was created for. Note that if @@ -151,14 +157,18 @@ to search for the connector inside the flow. .. Once, you have selected your connector, a loading spinner may appear and when it goes away the form is generated. ++ If you added metadata and want to test it integrated with the UI, push a `test/` or `integration/` branch with the added metadata and repeat the previous steps. + + == Step 2.7: Onboard onto OCS (Optional) + Optionally onboard onto OCS using the following https://github.com/mulesoft/ocs/blob/master/docs/common/onboarding-a-new-service-provider.md[steps]. + [[onboardlower]] == Step 2.8: Onboard Connector Onto Lower Environments @@ -171,6 +181,7 @@ an already onboarded connector. `./cli template trello` command. This creates a new file inside the customization folder: ++ [source,yaml] ---- name: trello @@ -208,6 +219,8 @@ operations: [] # TODO # # ... # # ... ---- ++ + + You can remove all of the `TODO` and placeholder `???`. For now, fill in the `artifactGav` with the information of the specialist Connector that @@ -215,6 +228,7 @@ corresponds to the Composer Connector. The following example is for Trello Connector: + ++ [source,yaml] ---- # ... @@ -225,6 +239,7 @@ artifactGav: # ... ---- + + . Optionally, download the extension model to use as a reference while completing the rest of the customization model by using the `./cli extension trello` command. This downloads the file into a JSON file. @@ -232,6 +247,7 @@ This downloads the file into a JSON file. and extension model to get the names of the operations and sources and their respective fields. ++ [source,yaml] ---- name: trello @@ -270,6 +286,7 @@ operations: # ... ---- + + . Run validations of the customization model while building it by using the `./cli validate foo` command. . Generate models using the cli tool by running `./cli connector trello` and then @@ -280,6 +297,7 @@ both the conenctor and transformation. .. Add the *schedulingStrategy* transformation. For each source in the `transformation_model.json`, add an entry on the transformations entry like this: ++ [source,json5] ---- { @@ -305,12 +323,13 @@ both the conenctor and transformation. } } ---- - + ++ The final transformation would then look like this: + ++ [source,json5] ---- { @@ -358,6 +377,7 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast under the path's `connectors.environments..foo` and adding the new version. ++ [source,yaml] ---- # ... @@ -372,11 +392,12 @@ connectors: # ... # ... ---- - + + . Add the connector to the unit tests, for which you should add similar entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: ++ [source,yaml] ---- KDEV: @@ -388,6 +409,7 @@ KQA: # ... ---- + + . Run `mvn clean test` to validate that the build passes. . Create a pull request for your newly onboarded connector. . Merge the pull request. @@ -418,6 +440,7 @@ the following steps: . Under `assets`, add the GAV of the new connector version. For example, for the following schema: ++ [source,yaml] ---- { @@ -441,7 +464,7 @@ the following steps: some schema fields were omitted with `...` for brevity. Only the assets section should be modified: - ++ [source,yaml] ---- { From cbb0bf33a24b394bc9d06e57d61d01fa20361926 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 09:58:23 -0700 Subject: [PATCH 08/13] Update tutorial-design.adoc --- modules/ROOT/pages/rest-sdk/tutorial-design.adoc | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index d5ae5c89..cb439b75 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -130,12 +130,12 @@ based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. .. Validate the Connection Schema Metadata with the xAPI by pushing a `test/` branch with your metadata and hitting the endpoint. -+ + `curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ --header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` -+ -+ + + As a result, the connectionSchemaModel property appears in the response with the generated Connection Schema Model with its Metadata. The customizations are shown @@ -157,10 +157,9 @@ to search for the connector inside the flow. .. Once, you have selected your connector, a loading spinner may appear and when it goes away the form is generated. -+ If you added metadata and want to test it integrated with the UI, push a `test/` or `integration/` branch with the added metadata and repeat the previous steps. -+ + == Step 2.7: Onboard onto OCS (Optional) @@ -325,9 +324,9 @@ both the conenctor and transformation. ---- + -+ + The final transformation would then look like this: -+ + + [source,json5] From 5ea39f5e6de73b6d84294dc1132e7d9988b8b1fc Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 10:02:06 -0700 Subject: [PATCH 09/13] Update tutorial-design.adoc --- modules/ROOT/pages/rest-sdk/tutorial-design.adoc | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index cb439b75..62d6e280 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -130,7 +130,7 @@ based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. .. Validate the Connection Schema Metadata with the xAPI by pushing a `test/` branch with your metadata and hitting the endpoint. - ++ `curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ --header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` @@ -157,8 +157,10 @@ to search for the connector inside the flow. .. Once, you have selected your connector, a loading spinner may appear and when it goes away the form is generated. ++ If you added metadata and want to test it integrated with the UI, push a `test/` or `integration/` branch with the added metadata and repeat the previous steps. ++ == Step 2.7: Onboard onto OCS (Optional) @@ -324,11 +326,10 @@ both the conenctor and transformation. ---- + - ++ The final transformation would then look like this: -+ [source,json5] ---- { From 1b88414341abe416f12152198413f6890dd30736 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 10:07:11 -0700 Subject: [PATCH 10/13] Update tutorial-design.adoc --- .../ROOT/pages/rest-sdk/tutorial-design.adoc | 22 +++++++------------ 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index 62d6e280..22e2fc76 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -128,20 +128,16 @@ https://github.com/mulesoft/cfm/blob/master/docs/connection-ui-autogeneration/cr to learn how to create a Connection Schema Metadata file. The file name must be based on the connection schema's GAV: `{groupId}-{artifactId}-{version}.json`. .. Validate the Connection Schema Metadata with the xAPI by pushing a `test/` -branch with your metadata and hitting the endpoint. - -+ -`curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ ---header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` - - - -As a result, the +branch with your metadata and hitting the endpoint. As a result, the connectionSchemaModel property appears in the response with the generated Connection Schema Model with its Metadata. The customizations are shown in the metadata of the endpoint's response. Alternatively, validate it against the CFM Connectivity Service. -+ + + +`curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ +--header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` + . Part 3: Validate the Auto Generated Connection UI @@ -297,7 +293,7 @@ both the conenctor and transformation. .. Rename the version folder. .. Add the *schedulingStrategy* transformation. For each source in the `transformation_model.json`, add an entry on the transformations entry like this: - +The following examples show the entry and the final transformation: + [source,json5] ---- @@ -324,10 +320,8 @@ both the conenctor and transformation. } } ---- -+ -+ -The final transformation would then look like this: + [source,json5] From 492596ce74f6396b08443694ca87090215112367 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 10:09:07 -0700 Subject: [PATCH 11/13] Update tutorial-design.adoc --- modules/ROOT/pages/rest-sdk/tutorial-design.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc index 22e2fc76..42d93510 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-design.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-design.adoc @@ -134,10 +134,10 @@ generated Connection Schema Model with its Metadata. The customizations are show in the metadata of the endpoint's response. Alternatively, validate it against the CFM Connectivity Service. - ++ `curl --location --request GET 'https://devx.composer.mulesoft.com/citizen-xapi/api/v1/organizations/ORG_ID/connection-schemas/connectors/CONNECTOR_NAME' \ --header 'Authorization: Bearer ANYPOINT_ACCESS_TOKEN'` - ++ . Part 3: Validate the Auto Generated Connection UI @@ -320,10 +320,10 @@ The following examples show the entry and the final transformation: } } ---- ++ - - ++ [source,json5] ---- { From 2d16476581af7671066620febd9d4c469358b7d3 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 10:12:45 -0700 Subject: [PATCH 12/13] format --- modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc | 11 ++++++++--- modules/ROOT/pages/rest-sdk/tutorial-release.adoc | 12 ++++++++++++ 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc index 9ea18c1f..5233f614 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-iterate.adoc @@ -21,6 +21,7 @@ Update the existing connector using the following steps: . Go to the customizations folder. . Update the version. ++ [source,yaml] ---- # ... @@ -31,6 +32,7 @@ artifactGav: # ... ---- + + . Make any necessary changes to the model, if any. . Generate models using the cli tool by running `./cli connector trello` and then `./cli transformation trello`. Alternatively, run `./cli both trello` to generate @@ -39,7 +41,8 @@ both the conenctor and transformation. .. Rename the version folder. .. Add the *schedulingStrategy* transformation. For each source in the `transformation_model.json`, add an entry on the transformations entry like this: - +The following examples show the entry and the final transformation: ++ [source,json5] ---- { @@ -67,9 +70,7 @@ both the conenctor and transformation. ---- + -The final transformation would then look like this: + - [source,json5] ---- { @@ -117,6 +118,7 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast under the path's `connectors.environments..foo` and adding the new version. ++ [source,yaml] ---- # ... @@ -132,9 +134,11 @@ connectors: # ... ---- + + . Update the connector on the unit tests, for which you should add similar entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: ++ [source,yaml] ---- KDEV: @@ -146,6 +150,7 @@ KQA: # ... ---- + + . Run `mvn clean test` to validate that the build passes. . Create a pull request for your newly onboarded connector. . Merge the pull request. diff --git a/modules/ROOT/pages/rest-sdk/tutorial-release.adoc b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc index 2a16ba3f..af3d98cc 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-release.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc @@ -11,6 +11,7 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast under the path `connectors.environments.KSTG.foo` and putting in the version that should be promoted. ++ [source,yaml] ---- # ... @@ -24,9 +25,11 @@ connectors: # ... ---- + + . Add or update the connector to the unit tests, for which you should add a similar entry to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: ++ [source,yaml] ---- # ... @@ -36,6 +39,7 @@ KSTG: # ... ---- + + . Run `mvn clean test` to validate that the build passes. . Create a pull request for your newly promoted connector. . Merge the pull request. @@ -47,6 +51,7 @@ KSTG: under the paths connectors.environments..foo and putting in the version that should be released. ++ [source,yaml] ---- # ... @@ -62,9 +67,11 @@ connectors: # ... ---- + + . Add the connector to the unit tests, for which you should add similar entries to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/resources/connectors-configuration-test.yml[connectors-configuration-test.yml]: ++ [source,yaml] ---- # ... @@ -76,10 +83,12 @@ KPROD: trello: 1.3.0 ---- + + . Create or update the corresponding prod test for the connector. Changes should be similar to these diffs: https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/mocks/ConnectorsMocks.java[ConnectorsMocks.java] + + [source, java, linenums] ---- @@ -95,7 +104,9 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast // ... ---- + + https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/controllers/ConnectorsControllerTest.java[ConnectorsControllerTest.java] + + [source, java, linenums] ---- @@ -117,6 +128,7 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast // ... ---- + + . Run `mvn clean test` to validate that the build passes. . Create a pull request for your soon to be released connector. . Merge the pull request. From 130b801e4045b7dee2309fada255beb8cc679fa6 Mon Sep 17 00:00:00 2001 From: sathya0 <78386709+sathya0@users.noreply.github.com> Date: Thu, 19 Aug 2021 10:15:11 -0700 Subject: [PATCH 13/13] Update tutorial-release.adoc --- modules/ROOT/pages/rest-sdk/tutorial-release.adoc | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/modules/ROOT/pages/rest-sdk/tutorial-release.adoc b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc index af3d98cc..458a22fe 100644 --- a/modules/ROOT/pages/rest-sdk/tutorial-release.adoc +++ b/modules/ROOT/pages/rest-sdk/tutorial-release.adoc @@ -84,10 +84,11 @@ KPROD: ---- + -. Create or update the corresponding prod test for the connector. Changes should -be similar to these diffs: - +. Create or update the corresponding prod test for the connector. Refer to https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/mocks/ConnectorsMocks.java[ConnectorsMocks.java] +and https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/controllers/ConnectorsControllerTest.java[ConnectorsControllerTest.java]. +Changes should be similar to these diffs: + + [source, java, linenums] @@ -105,8 +106,6 @@ https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/mast ---- + -https://github.com/mulesoft/citizen-platform-connectors-models-service/blob/master/citizen-platform-connectors-models-service/src/test/java/com/mulesoft/citizen/platform/connectors_models/controllers/ConnectorsControllerTest.java[ConnectorsControllerTest.java] - + [source, java, linenums] ----