Container Security: The 4C’s Of Cloud-Native Security – Part 2

Securing the container images of your cloud-native application building blocks addresses one of the 4C's in cloud-native security. If you haven't heard about the 4C's of cloud-native security yet or want a quick refresher, you should read my corresponding introduction post.

In this article:

This article will dive into different aspects of container security and walk through creating a secure container image. To do so, we will cover the following topics:

  • Author secure Dockerfiles according to best practices checked by dockle
  • Scanning and eliminating vulnerabilities in the inner- and outer-loop using container vulnerability scanning

That said, let’s briefly talk about the example application. We will containerize a simple web API build with Go for demonstration purposes. The API itself consists of three exposed HTTP endpoints:

  • POST: /echo – Responds the entire payload sent to the endpoint as the body of an HTTP 200 response
  • GET: /healthz/ready – Responds with an HTTP 200
  • GET: /healthz/alive – Responds with an HTTP 200

Nothing fancy because it does not matter what happens inside the app as we’re dealing with container security for the sake of this article. Although I use Go here, you can use all concepts and tools no matter which programming language or framework you are using. You can find the source code of the application in the cloud-native-security repository (See container-security subfolder).

Containerizing the Sample application

Containerizing existing applications is super easy with Visual Studio Code (VSCode) and its Docker extension. If you have not yet installed the Docker extension, go ahead and install it now! To install the Docker extension for VSCode, you can search for Docker (published by Microsoft itself) using the extension browser (CMD-or-Ctrl+Shift+X) you execute code --install-extension ms-azuretools.vscode-docker in the terminal of your choice.

If you haven’t cloned the sample repository, it is now the time to do that:

				
					# move to the directory where you keep your repos
cd repos

# clone the repo
git clone git@github.com:thinktecture-labs/cloud-native-security.git

# move into the cloned repo
cd cloud-native-security/container-security

# fire up VSCode
code .
				
			

The Docker extension adds several commands to your VSCode installation. Execute the Docker: Add Docker Files to Workspace command now. This will guide you through the process of generating a tailored Dockerfile. When asked by VSCode, provide the following answers:

  • Application Platform: Go
  • Application Port: 3000
  • Include optional Docker Compose files? No

By providing the answers above, VSCode can generate the following Dockerfile for you:

				
					#build stage
FROM golang:alpine AS builder
RUN apk add --no-cache git
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go build -o /go/bin/app -v ./...

#final stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /go/bin/app /app
ENTRYPOINT /app
LABEL Name=containersecurity Version=0.0.1
EXPOSE 3000
				
			

Git it a shot and build a new container image using docker build . -t container-security:0.0.1, after a couple of seconds (first execution may take a little bit longer because base images must be pulled), you should see the docker build command is finished successfully.

Check Dockerfile with dockle

Having our container-security:0.0.1 image in place (you can check the list of all your container images using docker images), we can check if the Dockerfile meets best practices. To get our Dockerfile checked, we will use dockle. If you have not installed dockle on your machine, check the detailed installation instructions here on GitHub.

To get our image checked, invoke dockle container-security:0.0.1 and you should see the following incidents being displayed:

				
					WARN    - CIS-DI-0001: Create a user for the container
      * Last user should not be root
INFO    - CIS-DI-0005: Enable Content trust for Docker
      * export DOCKER_CONTENT_TRUST=1 before docker pull/build
INFO    - CIS-DI-0006: Add HEALTHCHECK instruction to the container image
        * not found HEALTHCHECK statement
				
			

For the scope of this article, we will fix CIS-DI-0001 and CIS-DI-0006. We will not enable content trust for Docker (DCT) in this article because not all managed services offerings for running containerized workloads support DCT as of today. This also applies for Azure Kubernetes Service (See corresponding discussion on GitHub).

CIS-DI-0001: Last user should not be root

The Dockerfile generated by VSCode does not add a custom, non-privileged user. This means our container will run as root, which should be prevented is possible. We can fix CIS-DI-0001 by adding a new user and switching to the user context with the USER command. So let’s quickly update our Dockerfile, add a dedicated user called bob, and switch in his context:

				
					#build stage
FROM golang:alpine AS builder
RUN apk add --no-cache git
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go build -o /go/bin/app -v ./...

#final stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /go/bin/app /app

# non-root
RUN adduser -D bob && chown -R bob /app
USER bob
# end-non-root

ENTRYPOINT /app
LABEL Name=containersecurity Version=0.0.2
EXPOSE 3000
				
			

With those changes in place, let’s create a new image from the Dockerfile and 0.0.2 tag. When docker has finished the build, we can again invoke dockle to check the image:

				
					docker build . -t container-security:0.0.2
# ...
# => => naming to docker.io/library/container-security:0.0.2  

dockle container-security:0.0.2

INFO  - CIS-DI-0005: Enable Content trust for Docker
    * export DOCKER_CONTENT_TRUST=1 before docker pull/build
INFO  - CIS-DI-0006: Add HEALTHCHECK instruction to the container image
    * not found HEALTHCHECK statement
				
			

As dockle reports, we’ve successfully optimized our Dockerfile and CIS-DI-0001 is gone.

CIS-DI-0006: Add health check instruction to the container image

To fix CIS-DI-0006, we have to provide instructions for Docker on how to perform health checks for this container. Luckily, our sample API exposes several HTTP endpoints. For continuously checking the health of the API, we want Docker to issue an HTTP request to /healthz/alive.

Consult the official Dockerfile reference here, to see which options and arguments can be used in combination with HEALTHCHECK.

Remember that health check commands are executed inside the container. Because our base image is tiny, we’ve to add a package to issue an HTTP request. For demonstration purposes, let’s take curl. Let’s update the Dockerfile to add curl and configure our custom health check:

				
					#build stage
FROM golang:alpine AS builder
RUN apk add --no-cache git
WORKDIR /go/src/app
COPY . .

RUN go get -d -v ./...
RUN go build -o /go/bin/app -v ./...

#final stage
FROM alpine:latest
## also install curl for healthcheck
RUN apk --no-cache add ca-certificates curl

COPY --from=builder /go/bin/app /app

# non-root
RUN adduser -D bob && chown -R bob /app
USER bob
# end-non-root

# healthcheck
HEALTHCHECK CMD curl --fail http://localhost:3000/healthz/alive || exit 1
# end healthcheck

ENTRYPOINT /app
LABEL Name=containersecurity Version=0.0.3
EXPOSE 3000
				
			

Although we used curl in this example, keep in mind that curl will impact the overall size of your container image, and the overall attack surface of your container image increases because of the added software. Sometimes it is wiser to write a small (self-contained) CLI for Docker health checks. Although this can be achieved with any programming language, I would prefer writing it in Rust because of robustness, safety, and the final size of the distributable.

Inner-loop vulnerability scanning with Snyk

Now that our Dockerfile meets best practices (except CIS-DI-0005), we must check our container images for known vulnerabilities. Vulnerability scanning checks all software inside the container (Software bill of materials SBOM) for known vulnerabilities. This means that all its dependencies and even Alpine Linux are scanned on top of our executable.

Plenty of tools are available for scanning container images in the inner loop. I decided to demonstrate vulnerability scanning using Snyk (snyk CLI). I’ve chosen Snyk because it is also integrated with Docker Desktop, which most developers use these days on Windows and macOS. If you want to use the standalone snyk CLI or scan your container images for vulnerabilities directly with docker scan, a Snyk account is required. For further information, check out the corresponding parts of the Snyk documentation. Once installed, you have to authenticate using snyk auth.

To scan our container image with snyk invoke sync container test container-security:0.0.3. Snyk finish the scan and report 0 vulnerabilities

				
					Testing container-security:0.0.3...

Organization:   thorsten-hans
Package manager:  apk
Project name:   docker-image|container-security
Docker image:   container-security:0.0.3
Platform:     linux/amd64
Base image:    alpine:3.15.0
Licenses:     enabled

✔ Tested 19 dependencies for known issues, no vulnerable paths found.

According to our scan, you are currently using the most secure version of the selected base image.
				
			

Awesome!

However, there may be situations where your vulnerability scanner will report a list of discovered vulnerabilities. If this happens, enough information is included to understand which dependency introduces the vulnerability, and links to detailed information about how to remediate it if a fix for that particular vulnerability is available:

				
					✗ Medium severity vulnerability found in glibc/libc-bin
 Description: NULL Pointer Dereference
 Info: https://snyk.io/vuln/SNYK-UBUNTU2004-GLIBC-1564900
 Introduced through: glibc/libc-bin@2.31-0ubuntu9.2, meta-common-packages@meta
 From: glibc/libc-bin@2.31-0ubuntu9.2
 From: meta-common-packages@meta > glibc/libc6@2.31-0ubuntu9.2
 Image layer: Introduced by your base image (ubuntu:20.04)

				
			

Outer-loop vulnerability scanning with Azure Container Registry and Azure Defender for Containers

Although we successfully scanned our image for vulnerabilities in the inner-loop using Snyk, we should always enforce vulnerability scanning in the outer-loop too. Popular container registries such as Azure Container Registry (ACR) provide seamless integration with vulnerability scanners. ACR is no exception here. By enabling Azure Defender for Containers, your container images will automatically scanned when pushed to your ACR instance.

Microsoft does not have its own vulnerability scanner. Instead, Azure Defender for Containers leverages the vulnerability scanner created and provided by Qualys. When writing this article, customers using Azure Defender for Containers are charged $0.29 per image scan.

To enable Azure Defender for Containers, Azure Defender for Cloud must be activated for the desired Azure subscription: You can enable Azure Defender for Cloud by navigating to Azure Defender for Cloud | Environment settings. From the list of subscriptions, select the desired Azure subscription and ensure Azure Defender for Cloud is activated.

From the list of features, ensure Azure Defender for Containers is set to ON as shown in the following figure:

Once you’ve activated Azure Defender for Containers, Azure will scan container images in the following situations:

  • new image / tag get pushed to ACR
  • new images get imported to ACR

On top of that, all container images that are pulled from ACR during the last 30 days will be scanned once a week from Azure Defender for Containers for free.

If the scanner detects vulnerabilities, all information about the vulnerabilities of an image will be aggregated and displayed in Azure Defender for Cloud, along with recommendations on how to remediate and further background information about each vulnerability.

Having a list of all malicious container images with all discovered vulnerabilities in one place is super helpful. You can use Azure Defender for Cloud to drill into every container image its tags and see when vulnerabilities were introduced or remediated.

Recap

Containers are perhaps the most popular distribution format of this time. Managed Kubernetes offerings such as Azure Kubernetes Service (AKS) and others drove container adoption to an even higher level in the past years. Every developer must have a common understanding of what containers are, how they work, and how they should be authored with security in mind.

In addition to authoring robust and secure Dockerfiles, development teams regularly have to check their custom container images for vulnerabilities. Fortunately, there are plenty of tools available to integrate our software development process seamlessly.

More articles about Cloud Native
Free
Newsletter

Current articles, screencasts and interviews by our experts

Don’t miss any content on Angular, .NET Core, Blazor, Azure, and Kubernetes and sign up for our free monthly dev newsletter.

EN Newsletter Anmeldung (#7)
Related Articles
AI
sg
One of the more pragmatic ways to get going on the current AI hype, and to get some value out of it, is by leveraging semantic search. This is, in itself, a relatively simple concept: You have a bunch of documents and want to find the correct one based on a given query. The semantic part now allows you to find the correct document based on the meaning of its contents, in contrast to simply finding words or parts of words in it like we usually do with lexical search. In our last projects, we gathered some experience with search bots, and with this article, I'd love to share our insights with you.
17.05.2024
Angular
SL-rund
If you previously wanted to integrate view transitions into your Angular application, this was only possible in a very cumbersome way that needed a lot of detailed knowledge about Angular internals. Now, Angular 17 introduced a feature to integrate the View Transition API with the router. In this two-part series, we will look at how to leverage the feature for route transitions and how we could use it for single-page animations.
15.04.2024
.NET
KP-round
.NET 8 brings Native AOT to ASP.NET Core, but many frameworks and libraries rely on unbound reflection internally and thus cannot support this scenario yet. This is true for ORMs, too: EF Core and Dapper will only bring full support for Native AOT in later releases. In this post, we will implement a database access layer with Sessions using the Humble Object pattern to get a similar developer experience. We will use Npgsql as a plain ADO.NET provider targeting PostgreSQL.
15.11.2023