The simplest way to get started with a docker security program is to start with static analysis: Analyzing docker files for insecure software and making sure we have a level of trust in the base images our organization uses.
This is part of the series on how to build a comprehensive docker security program.
In this section, we'll get hands on with Clair and run our first static scan of an insecure image. Then, we'll take a look at a few ways to think about trusting base images.
The only thing you should need is to install Docker locally. I run Ubuntu at home, so these are instructions for that platform. For other platforms, head on over to the Docker install instructions.
sudo apt install docker docker-compose
You probably also want to add a non-root user to the docker group, so you can run docker commands without sudo, but this is optional.
sudo usermod -a -G docker `whoami`
You'll need to log out and log back in to have the group change take effect.
Finally, You can get all the source files I will share in this article from github.
Find vulnerable components using Clair
Clair is a vulnerability scanner for Docker containers and images.
It works best as a hosted solution integrated with your own container registry. But I like to start local, so today we're going to do a little work to get a sample install on our own machine.
There's a nice project that helps with running Clair locally called clairctl, which we'll use here. I pulled and modified their compose and configuration file for this example.
To get started, copy and paste these commands into your terminal:
git clone [email protected]:Charlie-belmer/Docker-security-example.git cd Docker-security-example/clair/ docker-compose up
This will pull down all the files you need to get started.
If you did the automatic setup above, you can safely skip this step.
If you prefer to build it from scratch, then create the following compose file - it will specify three containers: Clair, Clairctl, and postgres to store the vulnerability data in.
In a new base directory you created for this exercise, copy/paste the following into a file named "docker-compose.yml". This file is lightly modified from the clairctl example.
version: '2.1' services: postgres: image: postgres:9.6 restart: unless-stopped volumes: - ./docker-compose-data/postgres-data/:/var/lib/postgresql/data:rw environment: - POSTGRES_PASSWORD=ChangeMe - POSTGRES_USER=clair - POSTGRES_DB=clair clair: image: quay.io/coreos/clair:v2.0.6 restart: unless-stopped volumes: - ./docker-compose-data/clair-config/:/config/:ro - ./docker-compose-data/clair-tmp/:/tmp/:rw depends_on: postgres: condition: service_started command: [--log-level=debug, --config, /config/config.yml] user: root clairctl: image: jgsqware/clairctl:latest restart: unless-stopped environment: - DOCKER_API_VERSION=1.24 volumes: - ./docker-compose-data/clairctl-reports/:/reports/:rw - /var/run/docker.sock:/var/run/docker.sock:ro depends_on: clair: condition: service_started user: root
next, create a couple of sub-folders to house the volume and configuration file:
mkdir -p docker-compose-data/clair-config
Now we need to save the clair configuration file in the directory you just created. Copy and paste the below to a file named "config.yml" in the clair-config directory.
clair: database: type: pgsql options: source: postgresql://clair:ChangeMe@postgres:5432/clair?sslmode=disable cachesize: 16384 api: port: 6060 healthport: 6061 timeout: 900s updater: interval: 2h notifier: attempts: 3 renotifyinterval: 2h
You should now be ready to run the containers. From the base directory (the one containing your docker-compose file):
Download a vulnerable container
We also need a vulnerable container to scan. Bad Docker is a project that includes a number of known vulnerable software versions. We'll pull it into our local registry.
docker pull imiell/bad-dockerfile
Once you have Clair up and running, it may take a little while before its vulnerability database is populated. As a result, your specific vulnerability counts may differ from mine.
Run your first Clair scan on Bad Docker
We're now ready to run an analysis. Make sure Clair has been running on it's own for a while so its had a chance to download the vulnerability database, then run the following from within the Docker-security-example directory:
$ docker-compose exec clairctl clairctl analyze -l imiell/bad-dockerfile Image: /imiell/bad-dockerfile:latest 20 layers found ➜ Analysis [3c11e3990120] found 158 vulnerabilities. ➜ Analysis [087f5c5ab040] found 158 vulnerabilities. ➜ Analysis [46efa89923a0] found 158 vulnerabilities. ➜ Analysis [4ebbde339cd8] found 158 vulnerabilities. ➜ Analysis [8887a944aca2] found 157 vulnerabilities. ➜ Analysis [c5ebf3e7cf20] found 157 vulnerabilities. ➜ Analysis [2572e5c618b4] found 157 vulnerabilities. ➜ Analysis [28acc1686c10] found 157 vulnerabilities. ➜ Analysis [d1e4932b4c29] found 157 vulnerabilities. ➜ Analysis [cb60957c24fd] found 157 vulnerabilities. ➜ Analysis [6de830bc1a1e] found 157 vulnerabilities. ➜ Analysis [1a558b1331ce] found 102 vulnerabilities. ➜ Analysis [f4422be92a61] found 72 vulnerabilities. ➜ Analysis [bf644db833fa] found 72 vulnerabilities. ➜ Analysis [3946650a2ff4] found 72 vulnerabilities. ➜ Analysis [4797c6ceebf5] found 72 vulnerabilities. ➜ Analysis [ca25017ea953] found 72 vulnerabilities. ➜ Analysis [835cb2910d4f] found 68 vulnerabilities. ➜ Analysis [6389d277b72c] found 33 vulnerabilities. ➜ Analysis [f655f4bc1226] found 114 vulnerabilities. $ docker-compose exec clairctl clairctl report -l imiell/bad-dockerfile HTML report at /reports/html/analysis-imiell-bad-dockerfile-latest.html
A new report has been created for you in the sub-directory docker-compose-data/clairctl-reports/html. Open it in your browser and you should see something like this:
Great! You've run your first vulnerability scan.
Clair can be integrated directly with your container registry or into CI/CD pipelines to catch known vulnerabilities quickly. This is likely what we will want to do when automating our enterprise security strategy implementation.
Ensure teams know to use a trusted image
If you looked at a few Dockerfile examples, you probably noticed the first line specifies an image that will be downloaded from a registry you don't control. It looks something like:
How do you know that you are really getting an Ubuntu image? What about all those application specific containers like node:11? Dockerfile poisoning is a real concern, but there are a few things you can to to mitigate this.
There are three options I tend to recommend:
- Use official images from Dockerhub
- Build your own base images
- Use a trusted third party, such as CIS.
Official Docker images
If you are using official docker images, Dockerhub does do a lot to mitigate poisoning, so the first and simplest option is making sure you are using a trusted image. Say you want an Ubuntu image - you should search the registry and look for the "Official: OK" tag.
This will give you confidence that the image is maintained by the organization that creates the technology - Canonical in this case.
$ docker search ubuntu NAME DESCRIPTION STARS OFFICIAL AUTOMATED ubuntu Ubuntu is a D 8851 [OK] dorowu/ubuntu-d Ubuntu with o 248 [OK] rastasheep/ubun Dockerized SS 184 [OK] consol/ubuntu-x Ubuntu contai 136 [OK] ansible/ubuntu1 Ubuntu 14.04 95 [OK] ubuntu-upstart Upstart is an 93 [OK] neurodebian NeuroDebian p 55 [OK] 1and1internet/u ubuntu-16-ngi 47 [OK] ...
You can also log into the Docker Registry to see the latest security scans of these official images.
Building your own base image
If the idea of pulling those images still keeps you up at night, you might consider building your own base image and hosting it in a self-hosted registry.
This will also mean you will need to continuously update those base images as new patches come out, and may need to maintain several kinds of baseline OS and application images depending on the organizations needs.
Third party trusted containers
Some vendors have started offering trusted registry and pre-hardened images. I don't have a lot of hands on experience with these solutions, and would treat them like any other vendor offering.
Just last week, CIS announced a hardened image for AWS.
Putting together a rollout strategy
Now that you have an idea of what's possible, we can start with the first pillar of our strategy pretty quickly.
I find starting small and manual is the best way to make change happen, so here are the steps I recommend as an effective roll out strategy, assuming the organization is already using Docker.
- Pick a few internal projects and products that are using Docker and you have a good relationship with. Let them know you want to start a static analysis program and get their buy in as early adopters.
- Manually check if they are using some kind of trusted image. If not, discuss and learn.
- Run Clair from your local machine against their images, and see what turns up.
From there, you should have a reasonable idea about where to go next. At some point, you will want to consider adding Clair as a CI/CD component or as part of a trusted registry you maintain.
Now that we have a way to analyze images for known vulnerable software, we will turn to monitoring those containers after they are deployed.
In the next article, I discuss ways to integrate security monitoring tools with containers, and run some sample tests.
Did I miss anything or get something wrong? I want to hear from you on what has worked, what what hasn't, or what other threats I should consider.