Cookies & Privacy

We are using cookies on your computer to help improving this website. If you agree to our use of cookies, please clos this message and continue to use this site. More information on how we use cookies can be found in our privacy statement

more details

Essential

These tags are required for the basic functions of the website.

Marketing

Marketing/target cookies are usually used to show you advertisements that meet your interests. When you visit another website, your browser's cookie is recognized and selected ads are displayed to you based on the information stored in this cookie (art. 6 par. 1 p. 1 a) GDPR).

Imprint | privacy statement

Continuous Delivery with Jenkins and AEM

With step by step explanation

Cycling and Helping

eggs unimedia supports the fund raising "Cycling and helping".
All Blog Entries

Introduction

 

We all know CI - Continuous Integration and the next step is: CD - Continuous Delivery. Which is a combination of many steps to automate the build, test, release and deploy process.

 

There are some options we can choose from such as Jenkins or the GitLab Integration (https://stackoverflow.com/questions/37429453/gitlab-ci-vs-jenkins).

 

We use Jenkins Pipelines (https://jenkins.io/doc/book/pipeline/) because I discovered the GitLab integration too late and at that time the Jenkins process was already finished. Otherwise I would strongly recommend to try the GitLab integration to prevent the "Jenkins Plugin Hell".

 

(Another alternative is https://de.atlassian.com/software/bamboo)

 

Basics

 

The goal is to see the CD process as part of the code and not as a seperate part of development. Because of this reason the process is described in script files which are commited to GIT.

 

Now we have the advantage that we don't need to edit the Jenkins-Job-Configuration everytime we need to change the process. If we have a new server we can (theoretically) easily start our CD process without creating new jobs on the new server.

 

In the case of Jenkins we need to store a "Jenkinsfile" under the root directory of our project. This Jenkinsfile can be written in the Jenkins-Pipeline language or in groovy or a mix of both.

 

Requirements    

 

Jenkins needs to be at least at version 2 (because we need the Jenkins Pipeline)    

Following Plugins are needed to get the example Jenkinsfile running          

https://wiki.jenkins.io/display/JENKINS/Credentials+Plugin    

https://wiki.jenkins.io/display/JENKINS/Config+File+Provider+Plugin    

In the Jenkins "Global Tool Configuration" maven needs to be defined:

 

 

Example Jenkinsfile

 

Following steps are executed   

  1. Update the project/maven/pom version    
  2. Create a GIT-Tag with the new version    
  3. Push the new version to the branch
  4. Upload the artifact to the Nexus Repository
  5. Commit the new version to the child/develop branch
  6. Deploy the new version to the AEM Author and Publish instance

 

The complete file

#!groovy​

node {

    def version
    def webAppTarget = "xxx"
    def sourceBranch = "develop"
    def releaseBranch = "quality-assurance"
    def nexusBaseRepoUrl = "http://xxx"
    def repositoryUrl = "http://xxx"
    def gitCredentialsId = "xxx"
    def nexusRepositoryId = "xxx"
    def configFileId = "xxx"
    def mvnHome = tool 'M3'

    def updateQAVersion = {
        def split = version.split('\\.')
        //always remove "-SNAPSHOT"
        split[2] = split[2].split('-SNAPSHOT')[0]
        //increment the middle number of version by 1
        split[1] = Integer.parseInt(split[1]) + 1
        //reset the last number to 0
        split[2] = 0
        version = split.join('.')
    }

    //FIXME: use SSH-Agent
   //FIXME: use SSH-Agent

sh "git config --replace-all credential.helper cache"
sh "git config --global --replace-all user.email gituser@xxx.de; git config --global --replace-all user.name gituser"

configFileProvider([configFile(fileId: "${configFileId}", variable: "MAVEN_SETTINGS")]) {

    stage('Clean') {
        deleteDir()
    }

    dir('qa') {
        stage('Checkout QA') {
                echo 'Load from GIT'
                git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${releaseBranch}"
       }

            stage('Increment QA version') {
                version = sh(returnStdout: true, script: "${mvnHome}/bin/mvn -q -N org.codehaus.mojo:exec-maven-plugin:1.3.1:exec -Dexec.executable='echo' -Dexec.args='\${project.version}'").toString().trim()
                echo 'Old Version:'
                echo version
                updateQAVersion()
                echo 'New Version:'
                echo version
            }

            stage('Set new QA version') {
                echo 'Clean Maven'
                sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"

                echo 'Set new version'
                sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}"
            }

            stage('QA Build') {
                echo 'Execute maven build'
                sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
            }

            stage('Push new QA version') {
                echo 'Commit and push branch'
                sh "git commit -am \"New release candidate ${version}\""
                sh "git push origin ${releaseBranch}"
            }

            stage('Push new tag') {
                echo 'Tag and push'
                sh "git tag -a ${version} -m 'release tag'"
                sh "git push origin ${version}"
            }

            stage('QA artifact deploy') {
                echo 'Deploy artifact to Nexus repository'
                try {
                    sh "${mvnHome}/bin/mvn deploy:deploy-file -DpomFile=pom.xml -DrepositoryId=${nexusRepositoryId} -Durl=${nexusBaseRepoUrl} -Dfile=${webAppTarget}/target/${webAppTarget}-${version}.zip -Dpackaging=zip -s '$MAVEN_SETTINGS'"
                } catch (ex) {
                    println("Artifact could not be deployed to the nexus!")
                    println(ex.getMessage())
                }
            }

            stage('Deploy AEM Author') {
                echo 'deploy on author'
                withCredentials([usernamePassword(credentialsId: '6a613b0f-631b-453a-9f34-6a69e8676877', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
                    sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://doom.eggs.local:64592/crx/packmgr/service.jsp"
                }
            }

            stage('Deploy AEM Publish') {
                echo 'deploy on publish'
                withCredentials([usernamePassword(credentialsId: '3a25eefc-d446-4793-a621-9f15e4774126', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
                    sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://doom.eggs.local:64594/crx/packmgr/service.jsp"
                }
            }
        }

        dir('develop') {
            stage('Checkout develop') {
                echo 'Load from GIT'
                git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${sourceBranch}"
            }

            stage('Set new develop version') {
                echo 'Clean Maven'
                sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"

                echo 'Set new version'
                sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}-SNAPSHOT"
            }

            stage('Develop Build') {
                echo 'Execute maven build'
                sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
            }

            stage('Push new develop version') {
                echo 'Commit and push branch'
                sh "git commit -am \"New QA release candidate ${version}\""
                sh "git push origin ${sourceBranch}"
            }
        }
    }

}

 

Step by step explanation

 

Code Snippet #1

#!groovy

Our pipeline script is a groovy script (declarative) so we annotate this file as groovy for our development environment.

 

Code Snippet #2

node {

With the node we declare this script as scripted pipeline (see declarative pipeline for comparrision)

 

Code Snippet #3

def version
    def webAppTarget = "xxx"
    def sourceBranch = "develop"
    def releaseBranch = "quality-assurance"
    def nexusBaseRepoUrl = "http://xxx"
    def repositoryUrl = "http://xxx"
    def gitCredentialsId = "xxx"
    def nexusRepositoryId = "xxx"
    def configFileId = "xxx"
    def mvnHome = tool 'M3'

    def updateQAVersion = {
        def split = version.split('\\.')
        //always remove "-SNAPSHOT"
        split[2] = split[2].split('-SNAPSHOT')[0]
        //increment the middle number of version by 1
        split[1] = Integer.parseInt(split[1]) + 1
        //reset the last number to 0
        split[2] = 0
        version = split.join('.')
    }

Some variables like the branch names and the credential-ids that are used to log into GIT. And a function updateQAVersion that removes the "-SNAPSHOT" and increments the middle number (2.1.12-SNAPSHOT → 2.2.0)

 

Code Snippet #4

//FIXME: use SSH-Agent

sh "git config --replace-all credential.helper cache"
sh "git config --global --replace-all user.email gituser@xxx.de; git config --global --replace-all user.name gituser"

configFileProvider([configFile(fileId: "${configFileId}", variable: "MAVEN_SETTINGS")]) {

    stage('Clean') {
        deleteDir()
    }

    dir('qa') {
        stage('Checkout QA') {
                echo 'Load from GIT'
                git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${releaseBranch}"
       }

Set the git credentials with the help of the credential.helper. Clean the directory for a fresh checkout.

 

First stage (Jenkins Pipelines are grouped by stages) which will load the project sources with the help of the Jenkins Credentials Plugin (https://wiki.jenkins.io/display/JENKINS/Credentials+Plugin)

 

With dir('qa') we set the workspace/location (because we use two separate branches in this script) 

 

Code Snippet #5

stage('Increment QA version') {
    version = sh(returnStdout: true, script: "${mvnHome}/bin/mvn -q -N org.codehaus.mojo:exec-maven-plugin:1.3.1:exec -Dexec.executable='echo' -Dexec.args='\${project.version}'").toString().trim()
    echo 'Old Version:'
    echo version
    updateQAVersion()
    echo 'New Version:'
    echo version
}

stage('Set new QA version') {
    echo 'Clean Maven'
    sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"

    echo 'Set new version'
    sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}"
}

stage('QA Build') {
    echo 'Execute maven build'
    sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
}

Now we us the exec-maven-plugin to read the project version from the pom.xml. We set "returnStdout" to return the terminal output and place it into the "version" variable.

 

Then we use our function "updateQAVersion()" to get the next version and update our pom-files with the "versions:set" goal of maven.

 

After that we build the project to get the build package.

 

Code Snippet #6

stage('Push new QA version') {
    echo 'Commit and push branch'
    sh "git commit -am \"New release candidate ${version}\""
    sh "git push origin ${releaseBranch}"
}

stage('Push new tag') {
    echo 'Tag and push'
    sh "git tag -a ${version} -m 'release tag'"
    sh "git push origin ${version}"
}

The next stage is used to push the project with the changed version to the branch and create a tag. This only works because we set the "credential.helper cache". The get() function of groovy/pipeline does not support push (see Code Snippet #4)

 

Code Snippet #7

stage('QA artifact deploy') {
    echo 'Deploy artifact to Nexus repository'
    try {
        sh "${mvnHome}/bin/mvn deploy:deploy-file -DpomFile=pom.xml -DrepositoryId=${nexusRepositoryId} -Durl=${nexusBaseRepoUrl} -Dfile=${webAppTarget}/target/${webAppTarget}-${version}.zip -Dpackaging=zip -s '$MAVEN_SETTINGS'"
    } catch (ex) {
        println("Artifact could not be deployed to the nexus!")
        println(ex.getMessage())
    }
}

This stage deploys the created artifact (a .zip-file) to our Nexus Repository with the help of the maven command line command deploy:deploy-file. The configFileProvider is once again a Jenkins-Plugin which provides us with the maven settings.xml in which the credentials for the Nexus are defined.

 

Code Snippet #8

stage('Deploy AEM Author') {
    echo 'deploy on author'
    withCredentials([usernamePassword(credentialsId: 'xxx', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
        sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://xxx/crx/packmgr/service.jsp"
    }
}

stage('Deploy AEM Publish') {
    echo 'deploy on publish'
    withCredentials([usernamePassword(credentialsId: 'xxx', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
        sh "curl -u ${USERNAME}:${PASSWORD} -F file=@\"${webAppTarget}/target/${webAppTarget}-${version}.zip\" -F force=true -F install=true http://xxx/crx/packmgr/service.jsp"
    }
}

Now in the AEM-Deploy stage we call curl commands and deploys it to the AEM-Servers.

 

Code Snippet #9

dir('develop') {
    stage('Checkout develop') {
        echo 'Load from GIT'
        git url: "${repositoryUrl}", credentialsId: "${gitCredentialsId}", branch: "${sourceBranch}"
    }

    stage('Set new develop version') {
        echo 'Clean Maven'
        sh "${mvnHome}/bin/mvn -B clean -s '$MAVEN_SETTINGS'"

        echo 'Set new version'
        sh "${mvnHome}/bin/mvn -B versions:set -DnewVersion=${version}-SNAPSHOT"
    }

    stage('Develop Build') {
        echo 'Execute maven build'
        sh "${mvnHome}/bin/mvn -B install -s '$MAVEN_SETTINGS'"
    }

    stage('Push new develop version') {
        echo 'Commit and push branch'
        sh "git commit -am \"New QA release candidate ${version}\""
        sh "git push origin ${sourceBranch}"
    }
}

Because we changed the version from "1.2.12" to "1.3.0" in the qa-branch we want to change the version in the develop-branch too. In the develop-branch we add the "-SNAPSHOT" to the new version.

 

 

Conclusion

 

We have functioning release process, from incrementing the maven/project version, creating tags, deploying to the nexus repository until we deploy it to the AEM-Instances. Even other branches can be updated. We have full control and are very flexible.

 

This saves a lot of work for developers.

 

Buuuut this process is not perfect    

 

  • The error handling is none existing (can be optimized)    
  • We need a seperate Jenkins-Trigger-Job that calls the pipeline or else the pipeline calls itself after it commits to the branch. (We would need to implement something like ci-skip into our pipeline)
  • We need to configure a lot of credentials and maven settings in Jenkins

Author

Sebastian Binder

AEM Consultant

Published on

12.6.2018

Topics

Jenkins Adobe Experience Manager Continuous Delivery