Checking projects for vulnerabilities

You can also read this blog post on the Hotels.com Technology Blog

Maybe you have already heard about OWASP, the Open Web Application Security Project. It is a not-for-profit charitable organization focused on supporting web application security. They have a top ten list about critical security risks, last updated in 2017.

There are some widely known items on it such as Injection, Data exposure or Cross-site scripting that mostly can be avoided by using frameworks and other software components the right way. Now we will focus on using the right ones by discovering Components with Known Vulnerabilities used by our applications.

There’s a tool for that

OWASP has created a Dependency-check tool which is available for Ant, Maven, Gradle, Jenkins and there’s even a standalone command line version of it. On our project we use Maven so it was obvious that we should use the Maven version. The plugin scans through the dependencies of the project and checks if any of them is present in the National Vulnerability Database (NVD). Then it generates an HTML or XML report and optionally breaks the build if several previously set conditions are met about vulnerabilities.

So far so good, but the catchy part is it needs to download the whole database for that, which can take up to 400 MB. We can agree that 400 MB of disk space is not a big deal nowadays, but 340 MB of network traffic can be very painful.

First steps towards continous integration

The Maven plugin caches the NVD as an h2 database in the .m2 folder, so it shouldn’t be an issue running it locally, but on our Bamboo CI server it can cause some anxiety because of the .m2 folder being re-downloaded during each build. Is it really a good idea to run this plugin in all CI builds?

There’s another problem as well: what should we do with the results? Should we break the build? It will certainly slow our delivery down a lot, not to mention in extreme cases (e.g. when there’s no workaround yet) we wouldn’t be able to release our app at all! Conditional failure might be a solution but how can we figure out the right config (and there’s many) for that? Also, if we’re suppressing some vulnerabilities we might forget about it later when there’d be a workaround.

Anyways, I hope you got the point here.

Jenkins to the rescue!

The idea of using Jenkins came from an article by Prakhash Sivakumar. Since we are using our own dockerized Jenkins on our project for site generation, I didn’t see why not give it a shot and create a new Jenkins container that is responsible for running this plugin on our apps.

The problem with time and space

So now we separate the dependency check from our CI pipeline, but the problem with re-downloading the NVD over and over again still persists. Good thing that the Jenkins plugin has a solution for that. We are able to create a job that’s only responsible for download and update the NVD located in an arbitrary directory other than .m2. With a separate job managing the NVD, the dependency-checker jobs don’t need to download it all the time.

After adding dependency-check-jenkins-plugin to our Jenkins plugin list, we can create a job with a build step “Invoke Dependency-check update only”. Set a build trigger for your own liking, like a day or even a week, so the database stays up-to-date. The interesting part of the xml config should look like this:

<triggers>
  <hudson.triggers.TimerTrigger>
    <!-- Triggers build around every midnight -->
    <spec>H 23 * * *</spec>
  </hudson.triggers.TimerTrigger>
</triggers>
<builders>
  <org.jenkinsci.plugins.DependencyCheck.DependencyCheckUpdateOnlyBuilder plugin="dependency-check-jenkins-plugin@3.3.2">
    <!-- Where to save the database -->
    <datadir>/usr/share/nvd</datadir>
  </org.jenkinsci.plugins.DependencyCheck.DependencyCheckUpdateOnlyBuilder>
</builders>

Setting up the pipeline

I used the declarative pipeline for creating the jobs for dependency-checking, just like our site generator Jenkins container:

pipeline {
  agent any
  stages {
    stage('Checkout') {
      steps {
        git credentialsId: 'stash', 
          url: ''
      }   
    }
    /** Omitting some config steps **/
    stage('Build') {
      steps {
         sh 'mvn clean install'
      }
    }
    stage('Dependency Check') {
      steps {
          sh 'mvn org.owasp:dependency-check-maven:check
               -Dformat=XML -DdataDirectory=/usr/share/nvd -DautoUpdate=false'
          step([$class: 'DependencyCheckPublisher', unstableTotalAll: '0'])
        }
      }
    /** Omitting additional cleanup steps **/
  }
}

Unfortunately for multi-module projects you need the build phase so Maven can find dependent artifacts. It slows down the execution of the job; you can still speed it by skipping some plugin executions, such as unit, integration, and mutation tests or coverage checks. These steps are obsolete anyways as they will be performed on the CI pipeline. You can also skip some modules that are irrelevant for dependency check (e.g. release).

The last stage is where the magic happens: we run the dependency-check plugin with some custom settings:

• format=XML tells the plugin to generate the report in an XML format instead of HTML, so Jenkins could digest it
• dataDirectory=/usr/share/nvd should point to the same directory as our NVD updater job uses, the plugin will try to load the database from there
• autoUpdate=false prevents the plugin to update the NVD – it’s out of scope of this job

Then we call the publisher and generate nice human-readable pages and diagrams about the vulnerabilities of our app. For the sake of simplicity it’s set that on any vulnerability, the build will be marked unstable. Feel free to tweak it to your liking.

Summary

Now we have a standalone Jenkins that tells us if we use any components with known vulnerabilities without affecting our delivery at all. There’s room for improvement of course: you can send notifications, open automated SEC tickets, send direct messages to your Security Manager, and so on.