Creating a Secure Development Pipeline with Jenkins and SonarQube

In my last post, I talked about integrating security tools with an agile process, and mentioned some ways to automate security checks during development. In this article, I'll walk through a basic Jenkins setup for CI/CD, and integrate SonarQube and DependencyCheck for security scanning. In my next article, I'll expand this to include DAST tools.

Quick infrastructure setup

If you want to follow along, I have a pre-made docker-compose file for you that will create both a Jenkins and a SonarQube instance. To get these running, you will need git, docker, and docker-compose installed on your machine. Then run the following commands:

$ git clone git@github.com:Charlie-belmer/secure-pipeline-example.git
$ cd secure-pipeline-example/
$ docker-compose up

Watch the startup data for the Jenkins initial admin password - you will need this to get setup.

Configuring Jenkins To Build WebGoat

We're going to scan a known vulnerable webapp, WebGoat, which is an OWASP project used for learning basic web penetration testing skills and vulnerabilities. A good scanner should find a lot of things!

A quick aside: I was initially going to use Mutillidae, another vulnerable app written in PHP. However I couldn't find any good open source PHP Static analyzers that would catch the vulnerabilities. Make sure whatever SAST solution you implement can actually catch vulnerabilities in the languages your business uses!

Anyway, let's get on with Jenkins. Navigate in your browser to http://localhost:8080 and enter the admin password shown in the terminal running docker. Go ahead and install the default plugins (for a deployed instance, I would recommend only installing plugins you will actually use) and create your first admin user.

WebGoat requires Java 11 to build, which Jenkins won't install automatically. Head over to the main page -> Manage Jenkins -> Global Tool Configuration. There are two sections here we will update now:

The JDK installation: we need to add a link to a Java 11 installer - I used https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz. I sometimes see Jenkins have trouble installing a JDK this way if more than one JDK is installed in the system. If this is the first one, there should be no problems.
Maven installations: we can use the default maven, but do need to specify it.

Your config should look like this:

Jenkins JDK11 Install configuration

Jenkins Maven install configuration

Finally, we have to set the JAVA_HOME variable. In the Jenkins -> Manage Jenkins -> Configure System menu, enable environment variables and set JAVA_HOME equal to /var/jenkins_home/tools/hudson.model.JDK/openjdk11/jdk-11.0.1/.

***Update April 2020***
Following these same steps, I found my path to be /var/jenkins_home/tools/hudson.model.JDK/JDK11/jdk-11.0.1/. You can check on your local jenkins instance by logging in to the docker image and exploring that directory tree. You can get a local shell by using the command docker exec -it sast_pipeline_example_jenkins_1 bash
**************************

Now let's create a pipeline for WebGoat and make sure it builds successfully. Back on the main page choose new item -> freestyle project.

The initial setup is pretty simple:

Add Webgoat to the various github setting locations (https://github.com/WebGoat/WebGoat/)
Set the target branch to */develop
Create a maven build step ("Invoke top level maven targets") and give it the command "clean install"

***Update April 2020***
It seems that WebGoat has added some integration tests that depend on running services since I originally wrote this post. Building from the latest branch will yield connection errors. To get around this, we can disable the maven testing step by appending -DskipTests to the maven command. In a real environment, you would want to setup the pipeline to be able to execute these tests.

So, the final command in the "Goals" field should be clean install -DskipTests.
*************************

Here is my full pipeline configuration:

Basic WebGoat Pipeline

Try running it and making sure that everything builds successfully.

Adding SonarQube and DependencyCheck

SonarQube setup & security

We already have a SonarQube instance running, we just need to link and configure Jenkins to use it. Log in to http://localhost:9000 and use the default sonarQube login of admin/admin.

Although this is only for practice, I still want to secure our SonarQube instance, so do the following:

Change the admin password
Go to administration-> security and turn on "Force user authentication"
Create a new user for Jenkins.
Log into the new user, go to the profile -> security section, and generate a token. Copy this for later use.

Finally, create a project named "webgoat" with your jenkins user.

Configure the plugins for Jenkins

We will need two new plugins for jenkins. In the Jenkins home page, go to Mange Jenkins -> Manage Plugins. On the Available tab find and select "OWASP Dependency-Check Plugin" and "SonarQube Scanner for Jenkins". Install them without restarting.

Back on the Jenkins home, go to Manage Jenkins -> Global Tool Configuration. You should see a new option for SonarQube Scanner. Add an installation here (I just chose the latest from Maven Central) and save.

Finally, head over to Jenkins -> Manage Jenkins -> Configure System and add a sonarqube instance. The URL with our docker container is http://sonarqube:9000 and the token should be the one you saved while setting up the Jenkins user in SonarQube. Here is my setup:

SonarQube Settings

One other thing I had to do to get SonarQube working properly. For some reason I couldn't completely determine, the SonarQube startup script was truncating the JAVA_HOME path incorrectly, causing errors during the pipeline. To solve this, log into the docker container manually and update the sonar script to the proper JAVA_HOME.

$ docker exec -it secure_pipeline_jenkins_1 bash
jenkins@2ea0acb5905d:/$ cd /var/jenkins_home/tools/hudson.plugins.sonar.SonarRunnerInstallation/sonarqube
jenkins@2ea0acb5905d:~/tools/hudson.plugins.sonar.SonarRunnerInstallation/sonarqube$ head bin/sonar-scanner
#!/bin/sh
#
# SonarQube Scanner Startup Script for Unix
#
# Optional ENV vars:
#   SONAR_SCANNER_OPTS - Parameters passed to the Java VM when running the SonarQube Scanner
#   SONAR_SCANNER_DEBUG_OPTS - Extra parameters passed to the Java VM for debugging
#   JAVA_HOME - Location of Java's installation

JAVA_HOME="/var/jenkins_home/tools/hudson.model.JDK/openjdk11-remote/jdk-11.0.1"

Add SonarQube and DependencyCheck to the pipeline

Now we can add these to our pipeline and start scanning with every build.

In the pipeline created earlier, add two new build steps - Invoke Dependency-Check analysis and Execute SonarQube Scanner. In the SonarQube scanner, add the configuration settings required - the project key and name should match the project you created in SonarQube.

sonar.projectKey=webgoat
sonar.projectName=webgoat
sonar.projectVersion=1.0
sonar.language=java
sonar.java.binaries=**/target/classes
sonar.exclusions=**/*.ts

I am excluding the TypeScript files above since we did not setup Node or a JS build step for our project. In a real project, we would want to ensure that they were also scannable.

In the DependencyCheck advanced section, check to generate HTML reports as well for easier viewing.

Here is my full pipeline configuration now:

Full Secure Pipeline

Kick off a build and make sure it runs correctly. Afterwards, you should be able to see results.

Viewing Reports

If all runs successfully, logging into SonarQube will show you security scan details (with plenty of findings!) and the pipeline can show you the dependencyCheck results in the workspace -> dependency-check-report.html file.

DependencyCheck Sample Data

SonarQube Findings

You can and should at this point consider additional SonarQube plugins (or other SAST tools) that are specifically for your languages and frameworks.

Breaking the Build

We want to know when something isn't working right at the build phase. SonarQube gives us this for free with the plugin (you should see a nice red ERROR tag under the SonarQube Quality gate) but DependencyCheck requires one more configuration.

Add a post-build check for "Publish Dependency Check Results" and expand the advanced tabs. Just add some threshold data and the build will fail or be marked unstable according to the rules set.

Here is my final pipeline configuration, fully expanded.

Final Thoughts

Getting a CI/CD pipeline running with some basic security checks can be done within a few minutes. This will help keep your published artifacts in better shape and ensure the team has an opportunity to learn about security issues as soon as they emerge.

Next time, I will extend this pipeline to include basic dynamic security scans.