Jenkins – Pipelines 1

Congratulation, by now, you should have a fully working Jenkins installation, if not, please see this post. With that in mind, this post will focus on setting up our very first pipeline. In this example, we’ll setup a bare bones pipeline which we will integrate with GitLab and Heroku.

For those wondering why I chose GitLab and Heroku, the answer is simple, they provide a free-tier (until recently, GitHub’s free option did not allow for private repos) of their services. With this free tier, you can actually mimic a light-weight production environment (minus the scale).

Reminder, my goal in these posts, is not to supplant jenkins.io own docs, but to provide context as related to my experience. The documentation is actually very good, but documentation and implementation always differ slightly.

Credentials

Before we can do much of anything, we need to setup the proper credentials in our Jenkins installation. For our example we need to setup global credentials as described here.

Disclaimer, improper credential setup is a security vector. If an attacker gains access to your Jenkins installation, and your credentials are not properly setup, your entire pipeline will be at risk. Security is beyond the scope of this article (though we’ll have a dedicated article on the subject later), in the meantime you can read this to learn more about proper credentials setup and the risks associated with them.

With all of the above in mind, we need to setup two credentials:

1. GitLab SSH and API keys to access our git repo and update on progress
2. Heroku SSH key to interact with Heroku’s CLI

GitLab SSH and API Keys

The SSH key will enable you to clone, push and pull on the repo given the access restrictions for the user.

The API key will allow Jenkins post status updates to GitLab.

GitLab recommends creating a second user to which you can give collaborator access to the repo(s) in question. In this way, if issues arise, you can simply remove collaborator access to the Jenkins-GitLab user, which is much easier than dealing with issues on the Owner account.

Assuming you have created a new GitLab user, your Jenkins-GitLab account, you will need to do the following to setup collaboration:

1. Login to GitLab as the owner user
2. Navigate to the project and look for Members (click on that)
3. In the Members page enter the username and set the role.
   1. For this example choose Maintainer (as you need to be able to push to protected branches)
4. Click invite and leave the expiration field empty

Now you need to setup the SSH key pair to allow your Jenkins instance to access GitLab. In order to get Heroku to work properly, I had to setup the SSH Key pair as Jenkins. Login to the Jenkins server:

sudo su jenkins; 
cd; 
ssh-keygen -t rsa -b 2048 -C "match_your_gitlab_and_heroku_email_address";
cat .ssh/id_rsa.pub;

The last command will give you a copy of your public key. You will need to add this public key to your Jenkins-GitLab user:

1. Login to GitLab as the Jenkins-GitLab user
2. Go to settings and click on SSH keys
3. Add the public key and exit

Now you need to get the newly-created private key and add it to Jenkins global keys. To get the key simply type: cat .ssh/id_rsa (THIS IS YOUR PRIVATE KEY. BE CAREFUL).

At this point you have completed the setup of the GitLab SSH key pair. The next step is to setup the GitLab integration API key as described here. The instructions seem to be based on an older version of Jenkins so here is what I actually entered:

We are done with GitLab. The next step is to setup Heroku.

Heroku SSH Keys

In general, Heroku works a little different than GitLab as it uses its own CLI for interacting with Heroku and for deployments. To setup Heroku you will need to download the CLI to the Jenkins instance. I installed it using the command below:

curl https://cli-assets.heroku.com/install-ubuntu.sh | sh

Now you need to add your SSH public key to Heroku logged in as jenkins.

sudo su jenkins;
heroku keys:add;
Found an SSH public key at /var/lib/jenkins/.ssh/id_rsa.pub
? Would you like to upload it to Heroku? Yes
Uploading /var/lib/jenkins/.ssh/id_rsa.pub SSH key... !
 ▸    Invalid credentials provided.
heroku: Press any key to open up the browser to login or q to exit:
Opening browser to https://cli-auth.heroku.com/auth/cli/browser/
 ›   Warning: Cannot open browser.
Logging in... done
Logged in as heroku_user
Found an SSH public key at /var/lib/jenkins/.ssh/id_rsa.pub
? Would you like to upload it to Heroku? Yes
Uploading /var/lib/jenkins/.ssh/id_rsa.pub SSH key... done

If you installed Jenkins on an Ubuntu Server, you will get this error message:

Warning: Cannot open browser.

Simply copy and paste the URL and open it on any browser to login. Upon successful login, the SSH key will be added to Heroku. Once we have a workspace and a pipeline going we will need to do perform one final manual step, but we’re done with the credentials setup.

Our First Pipeline – Freestyle

These type of projects, Freestyle projects, were the original-style projects of Jenkins. Our first example will be a Freestyle project.

To create a Freestyle project go to Jenkins (http://localhost:8080) and click on “New Item“. Enter “Freestyle-Master” for the project name, select Freestyle Project, and click on “OK“.

In the next window enter or select the following:

• GitLab Connection: Select the correct connection from the drop-down
• GitLab Repository Name: Enter the complete repository name, basically the path of your GitLab URL: https://gitlab.com/<everything_here>
• Under source code management: Select git
• Repository URL: Enter your git clone URL ([email protected]:<repo_name>.git)
• Credentials: Select the Jenkins key from the dropdown
• Branches to build: Should be */master

Since Heroku generally builds the project upon deployment, in my example, we only need to run the tests prior to deployment. To execute the tests we’ll use the Gradle Wrapper under the Build section like this:

• Click Add Build Step: Select Invoke Gradle Script
• Use Gradle Wrapper: Selected
• Wrapper Location: . (current location)
• Tasks: clean test

To deploy our project to Heroku do the following:

• Click Add Build Step: Select Execute Shell
• In the command section enter:

heroku git:remote heroku-project-name --ssh-git;
git pull heroku HEAD:master;
git push heroku HEAD:master;

Finally, we need to keep track of our tests and to let GitLab know the status of the build. In the Post-build Actions do the following:

• Click Add Post-Build Action: Select Publish JUnit test result report
• Test Report XMLs: Enter

build/test-results/**/*.xml

• Click Add Post-Build Action: Select Publish Build Status to GitLab

Click Save and run your first build.

NOTE

It is possible your first build will fail when the job tries to deploy to heroku. This is because the first time you add the Heroku remote to the repo you have to accept the remote host’s fingerprint into the known_hosts file. If this happens simply login to the Jenkins hosts and execute the remote -add manually so that you can accept the fingerprint:

sudo su jenkins;
cd /var/lib/jenkins/workspace/Freestyle-Master;
heroku git:remote heroku-project-name --ssh-git;

With the fingerprint added to the known_hosts file, you should now be able to execute future builds without an issue.