From 021ec664d4124b47d1c6a9e536907396d7e9375c Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Mon, 19 Apr 2021 18:03:40 +0100 Subject: [PATCH 1/8] add examples to sh step --- .../durable_task/ShellStep/help-script.html | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html index 908c2653..e5a67ef3 100644 --- a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html +++ b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html @@ -9,4 +9,33 @@ Otherwise the system default shell will be run, using the -xe flags (you can specify set +e and/or set +x to disable those).

+

A few examples of usage include:

+ + Creating an output folder +

sh "mkdir -p output"

+ + Returning the current directory Pipeline is running in. +

sh "ls -la ${pwd()}"

+ + Making HTTP requests +

sh "curl -X POST --data-urlencode \'payload=${payload}\' ${slackURL}"

+ + Running tests in the same workspace that the project was built +

sh 'mvn test'

+ + Escaping script content from groovy interpretation > +

The triple-double-quote (""") string literal syntax allows for variable/expression substitution (interpolation), so the backslash (\) is interpreted as a special character "escape".

+

Since the first open parentheses is not such a special character, Groovy compilation fails. If your intent is to have literal backslashes in the resulting string, you need to escape the backslashes. That is, use a double-backslash (\\) to substitute for one literal backslash

+

sh (""" + sed "s/(AssemblyInformationalVersion\\(\\")(.*)(\\")/\\1${productVersion}\\3/g" + AssemblyInfoGlobal/AssemblyInfoGlobal.cs -r + """) +

+ + Adding shell script as part of Groovy interpolation > +

For double quoted string, Groovy will do interpolation on the string firstly.

+

Because the variables are runtime variables of the shell, rather than the variables of Groovy runtime. Groovy can't find the responding value from Groovy variable stack to replace the variables during interpolation.

+

So you need to escape all $ if you use double quotes or simply use single quotes which does not support interpolation

+

sh(returnStdout: true, script: "cd \$it; PLAN=\$(terragrunt plan --terragrunt-source-update | landscape); + echo \$PLAN; CHANGES=\$(echo \$PLAN | tail -2); echo \$CHANGES"

\ No newline at end of file From fed44ed56d9506a292de478847e0b239b9332734 Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Sun, 25 Apr 2021 01:12:33 +0100 Subject: [PATCH 2/8] Add Some Documentation To The Nodes And Processes Plugin --- README.md | 98 +++++++++++++++++++ .../durable_task/ShellStep/help-script.html | 1 + 2 files changed, 99 insertions(+) diff --git a/README.md b/README.md index d1290d98..301e715e 100644 --- a/README.md +++ b/README.md @@ -11,8 +11,106 @@ Pipeline steps locking agents and workspaces, and running external processes tha ## Documentation +The nodes and processes pluginalso called workflow durable task steps locks jenkins agents with the processes running on them, thereby adding durability to workflows. Pipelines can resume after an unforseeable restart. + +### Pipelines + +The plugin provides a `ws` step to allocate a workspace, however, this is created automatically using the `node` step. The [Pipeline Syntax Snippet Generator](https://www.jenkins.io/doc/book/pipeline/getting-started/#snippet-generator) guides the user on usage, practical examples and more information. + +### Node + +A node is the machine on which Jenkins runs, it is part of the Jenkins environment and is capable of executing a Pipeline. +The `agent` syntax is equivalent to the `node` syntax except in different spheres. agent is a [declarative syntax](https://www.jenkins.io/doc/book/pipeline/#declarative-pipeline-fundamentals) that is used to specify the executor and workspace on which the Pipeline will be executed while node is [scripted syntax](https://www.jenkins.io/doc/book/pipeline/#scripted-pipeline-fundamentals) that is used to execute Pipelines (and any stages contained within it), on any available agent/node + +### Processes + +A process in Jenkins can be defined as the continuous integration and continuous delivery of builds, tests, analysis, and deployment of projects and any other scenario that can be automated. The steps in these processes are documented in the jenkinsfile which can be created manually or added automatically using Blue Ocean. + +Examples of nodes and processes in respect to declarative and scripted pipeline +1. A scripted pipeline example can be found [here](https://www.jenkins.io/doc/book/pipeline/#scripted-pipeline-fundamentals) + +2. A declarative pipeline example can be found [here](https://www.jenkins.io/doc/book/pipeline/#declarative-pipeline-fundamentals), refer to the [Pipeline Syntax Snippet Generator](https://www.jenkins.io/doc/book/pipeline/getting-started/#snippet-generator) and for more information + +### Using multiple agents and setting labels + +It is possible to run Jenkins pipeline on multiple agents. Pipelines that can run on low resource can be paired with equal powered agents and high resource agents with equal powered pipelines to avoid unnecessarily long build time. + +A declarative pipeline with multiple agents: + +''' +pipeline { + agent none + stages { + stage('Build') { + agent any + steps { + checkout scm + sh 'make' + stash includes: '**/target/*.jar', name: 'app' + } + } + stage('Test on Linux') { + agent { + label 'linux' + } + steps { + unstash 'app' + sh 'make check' + } + post { + always { + junit '**/target/*.xml' + } + } + } + stage('Test on Windows') { + agent { + label 'windows' + } + steps { + unstash 'app' + bat 'make check' + } + post { + always { + junit '**/target/*.xml' + } + } + } + } +} +''' +A scripted pipeline with multiple agents + +''' +stage('Test') { + node('linux') { + checkout scm + try { + unstash 'app' + sh 'make check' + } + finally { + junit '**/target/*.xml' + } + } + node('windows') { + checkout scm + try { + unstash 'app' + bat 'make check' + } + finally { + junit '**/target/*.xml' + } + } +} +''' +Refer to this [article](https://docs.cloudbees.com/docs/admin-resources/latest/automating-with-jenkinsfile/using-multiple-agents) for a detailed explanation. + * [Changelog](https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/master/CHANGELOG.md) + ## License [MIT License](https://opensource.org/licenses/mit-license.php) diff --git a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html index e5a67ef3..2d02ad73 100644 --- a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html +++ b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html @@ -13,6 +13,7 @@ Creating an output folder

sh "mkdir -p output"

+ Returning the current directory Pipeline is running in.

sh "ls -la ${pwd()}"

From 3483e51d848c1b07b753dad79fb71ad32498fd2c Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Sun, 25 Apr 2021 01:30:21 +0100 Subject: [PATCH 3/8] Add Some Documentation To The Nodes And Processes Plugin --- README.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 301e715e..decb5b7f 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Pipeline steps locking agents and workspaces, and running external processes tha ## Documentation -The nodes and processes pluginalso called workflow durable task steps locks jenkins agents with the processes running on them, thereby adding durability to workflows. Pipelines can resume after an unforseeable restart. +The nodes and processes plugin also called workflow durable task steps locks jenkins agents with the processes running on them, thereby adding durability to workflows. This ensures Pipelines can resume after an unforseeable restart. ### Pipelines @@ -37,7 +37,7 @@ It is possible to run Jenkins pipeline on multiple agents. Pipelines that can ru A declarative pipeline with multiple agents: -''' +``` pipeline { agent none stages { @@ -79,10 +79,10 @@ pipeline { } } } -''' +``` A scripted pipeline with multiple agents -''' +``` stage('Test') { node('linux') { checkout scm @@ -105,10 +105,12 @@ stage('Test') { } } } -''' +``` Refer to this [article](https://docs.cloudbees.com/docs/admin-resources/latest/automating-with-jenkinsfile/using-multiple-agents) for a detailed explanation. -* [Changelog](https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/master/CHANGELOG.md) +## Changelog + +[Changelog](https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/master/CHANGELOG.md) ## License From c7a230bfad1da2bf53038e5db8303d2661e27b65 Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Wed, 28 Apr 2021 08:34:35 +0100 Subject: [PATCH 4/8] Add Snippet Generator Hints To Step Help --- .../workflow/steps/durable_task/ShellStep/help-script.html | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html index 2d02ad73..f890f605 100644 --- a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html +++ b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/ShellStep/help-script.html @@ -9,11 +9,11 @@ Otherwise the system default shell will be run, using the -xe flags (you can specify set +e and/or set +x to disable those).

-

A few examples of usage include:

+

Use the Pipeline Snippet Generator to generate a sample pipeline script for the sh step + A few examples of usage include:

Creating an output folder

sh "mkdir -p output"

- Returning the current directory Pipeline is running in.

sh "ls -la ${pwd()}"

@@ -27,6 +27,7 @@ Escaping script content from groovy interpretation >

The triple-double-quote (""") string literal syntax allows for variable/expression substitution (interpolation), so the backslash (\) is interpreted as a special character "escape".

Since the first open parentheses is not such a special character, Groovy compilation fails. If your intent is to have literal backslashes in the resulting string, you need to escape the backslashes. That is, use a double-backslash (\\) to substitute for one literal backslash

+

Use the Pipeline Snippet Generator to generate this example

sh (""" sed "s/(AssemblyInformationalVersion\\(\\")(.*)(\\")/\\1${productVersion}\\3/g" AssemblyInfoGlobal/AssemblyInfoGlobal.cs -r @@ -37,6 +38,7 @@

For double quoted string, Groovy will do interpolation on the string firstly.

Because the variables are runtime variables of the shell, rather than the variables of Groovy runtime. Groovy can't find the responding value from Groovy variable stack to replace the variables during interpolation.

So you need to escape all $ if you use double quotes or simply use single quotes which does not support interpolation

+

Use the Pipeline Snippet Generator to generate this example

sh(returnStdout: true, script: "cd \$it; PLAN=\$(terragrunt plan --terragrunt-source-update | landscape); echo \$PLAN; CHANGES=\$(echo \$PLAN | tail -2); echo \$CHANGES"

\ No newline at end of file From 14ab91627e1741ff489ffb3c729a33e4fe13f997 Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Wed, 28 Apr 2021 14:23:56 +0100 Subject: [PATCH 5/8] Add Examples For Bat Step --- .../BatchScriptStep/help-script.html | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/BatchScriptStep/help-script.html b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/BatchScriptStep/help-script.html index e1327aa1..8fbca4b3 100644 --- a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/BatchScriptStep/help-script.html +++ b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/BatchScriptStep/help-script.html @@ -3,3 +3,50 @@ When using the returnStdout flag, you probably wish to prefix this with @, lest the command itself be included in the output. + +

The bat step is used to run pipelines on a windows environment, + if your pipeline will run on a Linux environment, use the sh step. +

To use the bat step, you have to first point it to the path where your bat file exists, and then 'call' command to trigger that bat file. Examples below:

+This example calls mybat.bat file inside the example folder, extra backslashes are always added to every backslash +

Use the Pipeline Snippet Generator to generate the example

+

bat 'C:\\example\\mybat.bat'

+ +

An example of a declarative pipeline executing a bat step with the command set

+

+

+ pipeline { + agent any + stages { + stage('Build') { + steps { + bat 'set' + } + } + } + } +
+

+ +

Use the Pipeline Snippet Generator to generate this example

+

Print out all the environment variables seen by Windows.

+

echo bat(returnStdout: true, script: 'set')

+ +

Print out the content of the PATH environment variable on Windows

+

bat 'echo %PATH%'

+ +

Add the keyword call when running a multi-line script and need the commands to execute sequentially +

+ +

Failure to do so will result in only the first line + executing. You can also use ''&'' to chain commands into a single line +but this will make your commands hard to read and prone to mistakes.

+

Use the Pipeline Snippet Generator to generate this example

+

+

+ bat """ + call c:\path\to\conda activate my_env + cd c:\\path\\to\\scripts + call python test.py ${argument} + """ +
+

\ No newline at end of file From 7cdfc978cd50d0f389057dea341a561fb09d0711 Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Thu, 29 Apr 2021 12:03:24 +0100 Subject: [PATCH 6/8] Add Examples and Documentation To The Return Status Argument --- .../DurableTaskStep/help-returnStatus.html | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStatus.html b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStatus.html index db19c343..dd1fba3b 100644 --- a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStatus.html +++ b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStatus.html @@ -3,3 +3,23 @@ If this option is checked, the return value of the step will instead be the status code. You may then compare it to zero, for example. +

The return status step is used to return the exit status of a script. 0 equals true in shell. An exit code + of 1 will cause jenkins to abort. +

+

Use the + Pipeline Snippet Generator + to generate this example by checking the returnStatus option +

+

sh returnStatus: true, script: 'mvn test'

+ +

You can use an if statement to ascertain if the exit status returned 0 or not by assigning your script to + a variable. +

+

int status = sh returnStatus: true, script: 'mvn test'

+ + if (status != 0){ + //Take Action + } + +

While it is not advisable, use set +e to ignore the exit status code 1 when running shell script

+

See sh documentation for this value

\ No newline at end of file From fcaf9503402004f45b36f6b616084fa3ac74676a Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Fri, 30 Apr 2021 09:51:25 +0100 Subject: [PATCH 7/8] Add Return Stdout Help --- .../DurableTaskStep/help-returnStdout.html | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html index 1f891dfe..0ddf1b81 100644 --- a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html +++ b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html @@ -4,3 +4,27 @@ (Standard error, if any, will still be printed to the log.) You will often want to call .trim() on the result to strip off a trailing newline. +

The stdout command is usually unchecked by default when using +the pipeline syntax generator. It can be used with any of the steps in the nodes and processes plugin. +Refer to the steps reference page for more information on all the steps available.

+ +

An example of a declarative pipeline using the stdout argument.

+ +

script { + VARIABLE = sh ( + script: //Run script here, + returnStdout: true + ).trim() + echo "Example Output: ${Variable}" +}

+ +

An example of scripted pipeline(without the script block)

+

+ VARIABLE = sh ( + script: //Run script here, + returnStdout: true + ).trim() + echo "Example Output: ${Variable}" +

+ +

Use variables to store pipeline steps to encapsulate results.

\ No newline at end of file From e9ba8f71c903187fdba27a3979e53306157514e3 Mon Sep 17 00:00:00 2001 From: ejidike esther Date: Fri, 30 Apr 2021 10:02:47 +0100 Subject: [PATCH 8/8] Add New Example Using Bat Step --- .../durable_task/DurableTaskStep/help-returnStdout.html | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html index 0ddf1b81..2b5e1159 100644 --- a/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html +++ b/src/main/resources/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep/help-returnStdout.html @@ -27,4 +27,9 @@ echo "Example Output: ${Variable}"

-

Use variables to store pipeline steps to encapsulate results.

\ No newline at end of file +

Use variables to store pipeline steps to encapsulate results.

+

Using stdout with the bat step

+

The backslashes are used to escape the quotation marks (if you need them as part + of the script) +

+

bat returnStdout: true, script: '\'set\''

\ No newline at end of file