The simplest way to get started with a docker security program is to start with static analysis: Analyzing docker files for insecure software and making sure we have a level of trust in the base images our organization uses.

This is part of the series on how to build a comprehensive docker security program.

In this section, we'll get hands on with Clair and run our first static scan of an insecure image. Then, we'll take a look at a few ways to think about trusting base images.

Getting Setup

The only thing you should need is to install Docker locally. I run Ubuntu at home, so these  are instructions for that platform. For other platforms, head on over  to the Docker install instructions.

sudo apt install docker docker-compose

You probably also want to add a non-root user to  the docker group, so you can run docker commands without sudo, but this  is optional.

sudo usermod -a -G docker `whoami`

You'll need to log out and log back in to have the group change take effect.

Finally, You can get all the source files I will share in this article from github.

Find vulnerable components using Clair

Clair is a vulnerability scanner for Docker containers and images.

It works best as a hosted solution integrated with your own container registry. But I like to start local, so today we're going to do a little work to get a sample install on our own machine.

There's a nice project that helps with running Clair locally called clairctl, which we'll use here. I pulled and modified their compose and configuration file for this example.

Automatic setup

To get started, copy and paste these commands into your terminal:

git clone [email protected]:Charlie-belmer/Docker-security-example.git   
cd Docker-security-example/clair/   
docker-compose up

This will pull down all the files you need to get started.

Manual setup

If you did the automatic setup above, you can safely skip this step.

If you prefer to build it from scratch, then create the following compose file - it will specify three containers: Clair, Clairctl, and postgres to store the vulnerability data in.

In a new base directory you created for this exercise, copy/paste the following into a file named "docker-compose.yml". This file is lightly modified from the clairctl example.

version: '2.1'

services:
  postgres:
    image: postgres:9.6
    restart: unless-stopped
    volumes:
      - ./docker-compose-data/postgres-data/:/var/lib/postgresql/data:rw
    environment:
      - POSTGRES_PASSWORD=ChangeMe
      - POSTGRES_USER=clair
      - POSTGRES_DB=clair
    
  clair:
    image: quay.io/coreos/clair:v2.0.6
    restart: unless-stopped
    volumes:
      - ./docker-compose-data/clair-config/:/config/:ro
      - ./docker-compose-data/clair-tmp/:/tmp/:rw
    depends_on: 
      postgres:
        condition: service_started
    command: [--log-level=debug, --config, /config/config.yml]
    user: root

  clairctl:
    image: jgsqware/clairctl:latest
    restart: unless-stopped
    environment: 
      - DOCKER_API_VERSION=1.24
    volumes:
      - ./docker-compose-data/clairctl-reports/:/reports/:rw
      - /var/run/docker.sock:/var/run/docker.sock:ro
    depends_on: 
      clair: 
        condition: service_started
    user: root

next, create a couple of sub-folders to house the volume and configuration file:

mkdir -p docker-compose-data/clair-config

Now we need to save the clair configuration file in the directory you just created. Copy and paste the below to a file named "config.yml" in the clair-config directory.

clair:
  database:
    type: pgsql
    options:
      source: postgresql://clair:ChangeMe@postgres:5432/clair?sslmode=disable
      cachesize: 16384
  api:
    port: 6060
    healthport: 6061
    timeout: 900s
  updater:
    interval: 2h
  notifier:
    attempts: 3
    renotifyinterval: 2h

You should now be ready to run the containers. From the base directory (the one containing your docker-compose file):

docker-compose up

Download a vulnerable container

We also need a vulnerable container to scan. Bad Docker is a project that includes a number of known vulnerable software versions. We'll pull it into our local registry.

docker pull imiell/bad-dockerfile

Once you have Clair up and running, it may take a little while before its vulnerability database is populated. As a result, your specific vulnerability counts may differ from mine.

Run your first Clair scan on Bad Docker

We're now ready to run an analysis. Make sure Clair has been running on it's own for a while so its had a chance to download the vulnerability database, then run the following from within the Docker-security-example directory:

$ docker-compose exec clairctl clairctl analyze -l imiell/bad-dockerfile   
Image: /imiell/bad-dockerfile:latest
   20 layers found

    ➜ Analysis [3c11e3990120] found 158 vulnerabilities.
    ➜ Analysis [087f5c5ab040] found 158 vulnerabilities.
    ➜ Analysis [46efa89923a0] found 158 vulnerabilities.
    ➜ Analysis [4ebbde339cd8] found 158 vulnerabilities.
    ➜ Analysis [8887a944aca2] found 157 vulnerabilities.
    ➜ Analysis [c5ebf3e7cf20] found 157 vulnerabilities.
    ➜ Analysis [2572e5c618b4] found 157 vulnerabilities.
    ➜ Analysis [28acc1686c10] found 157 vulnerabilities.
    ➜ Analysis [d1e4932b4c29] found 157 vulnerabilities.
    ➜ Analysis [cb60957c24fd] found 157 vulnerabilities.
    ➜ Analysis [6de830bc1a1e] found 157 vulnerabilities.
    ➜ Analysis [1a558b1331ce] found 102 vulnerabilities.
    ➜ Analysis [f4422be92a61] found 72 vulnerabilities.
    ➜ Analysis [bf644db833fa] found 72 vulnerabilities.
    ➜ Analysis [3946650a2ff4] found 72 vulnerabilities.
    ➜ Analysis [4797c6ceebf5] found 72 vulnerabilities.
    ➜ Analysis [ca25017ea953] found 72 vulnerabilities.
    ➜ Analysis [835cb2910d4f] found 68 vulnerabilities.
    ➜ Analysis [6389d277b72c] found 33 vulnerabilities.
    ➜ Analysis [f655f4bc1226] found 114 vulnerabilities.    
$ docker-compose exec clairctl clairctl report -l imiell/bad-dockerfile  
HTML report at /reports/html/analysis-imiell-bad-dockerfile-latest.html

A new report has been created for you in the sub-directory docker-compose-data/clairctl-reports/html. Open it in your browser and you should see something like this:

Great! You've run your first vulnerability scan.

Clair can be integrated directly with your container registry or into CI/CD pipelines to catch known vulnerabilities quickly. This is likely what we will want to do when automating our enterprise security strategy implementation.

Ensure teams know to use a trusted image

If  you looked at a few Dockerfile examples, you probably noticed the first line specifies an image that will be downloaded from a registry you don't control. It looks something like:

FROM   ubuntu:latest

How do you know that you are really getting an Ubuntu image? What about all those application specific containers like node:11? Dockerfile poisoning is a real concern, but there are a few things you can to to mitigate this.

There are three options I tend to recommend:

1. Use official images from Dockerhub
2. Build your own base images
3. Use a trusted third party, such as CIS.

Official Docker images

If you are using official docker images, Dockerhub does do a lot to mitigate poisoning, so the first and simplest option is making sure you are using a trusted image. Say you want an  Ubuntu image - you should search the registry and look for the  "Official: OK"  tag.

This will give you confidence that the image  is maintained by the organization that creates the technology -  Canonical in this case.

$ docker search ubuntu   
NAME              DESCRIPTION     STARS   OFFICIAL    AUTOMATED   
ubuntu            Ubuntu is a D   8851    [OK]                 
dorowu/ubuntu-d   Ubuntu with o   248                 [OK]   
rastasheep/ubun   Dockerized SS   184                 [OK]   
consol/ubuntu-x   Ubuntu contai   136                 [OK]   
ansible/ubuntu1   Ubuntu 14.04    95                  [OK]   
ubuntu-upstart    Upstart is an   93      [OK]                   
neurodebian       NeuroDebian p   55      [OK]                  
1and1internet/u   ubuntu-16-ngi   47                  [OK]   
...

You can also log into the Docker Registry to see the latest security scans of these official images.

Building your own base image

If the idea of pulling those images still keeps you up at night, you might consider building your own base image and hosting it in a self-hosted registry.

This will also mean you will need to continuously update those base images as new patches come out, and may need to maintain several kinds of baseline OS and application images depending on the organizations needs.

Third party trusted containers

Some vendors have started offering trusted registry and pre-hardened images. I don't have a lot of hands on experience with these solutions, and would treat them like any other vendor offering.

Just last week, CIS announced a hardened image for AWS.

Putting together a rollout strategy

Now that you have an idea of what's possible, we can start with the first pillar of our strategy pretty quickly.

I find starting small and manual is the best way to make change happen, so here are the steps I recommend as an effective roll out strategy, assuming the organization is already using Docker.

1. Pick a few internal projects and products that are using Docker and you have a good relationship with. Let them know you want to start a static analysis program and get their buy in as early adopters.
2. Manually check if they are using some kind of trusted image. If not, discuss and learn.
3. Run Clair from your local machine against their images, and see what turns up.

From there, you should have a reasonable idea about where to go next. At some point, you will want to consider adding Clair as a CI/CD component or as part of a trusted registry you maintain.

Final Thoughts

Now that we have a way to analyze images for known vulnerable software, we  will turn to monitoring those containers after they are deployed.

In the next article, I discuss ways to integrate security monitoring tools with containers, and run some sample tests.

Did I miss anything or get something wrong? I want to hear from you on what has worked, what what hasn't, or what other threats I should consider.