Skip to content

aa8y/dave

Repository files navigation

Dave

CI codecov

Dave is a tool which is intended to help with Docker image authoring. It tries to fill the gaps Docker Hub has around building images, which are:

  • Lack of support for build arguments.
  • The only way to build multiple tags being the usage of a different Dockerfile for each tag which goes against the DRY philosophy.
  • No testing infrastructure so that tags are published only after passing (a) test(s).

Dave is an acronym for Docker Authoring made Very Easy. Actually that's not completely true since I came up with the name first and then the acronym. And if you're wondering where the name came from, here's a hint.

NOTE: It is still work in progress. So suffice to many things might change. Hopefully not too many though.

Features

Dave performs its operations by using metadata in a YAML serialized manifest file. The format is explained later. The following operations are supported.

  • all: Executes build, test, structure-test and push in order.
  • build: Builds one or more images using the given Docker build command template and its arguments.
  • push: Pushes a local image using the given Docker push command template and its arguments.
  • structure-test: Runs container-structure-test against a built image using configs declared in the manifest. Requires the container-structure-test binary on PATH.
  • test: Tests a Docker image by invoking a certain command on the image. A non-zero exit status code fails the test.

All templating is done using Mustache.

Usage

The command-line utility requires at least one command to be passed.

dave <command1> [<command2> ...] [OPTIONS]

The options that it accepts are:

  • --context or -c: Accepts a string denoting the location of the Dockerfile. If just the context is passed, the command(s) would be invoked on all tags pertaining to the context.
  • --tags or -t: Accepts a list tags separated by spaces. Requires the context to be passed as well. If the context is not passed, the tags would be ignored.
  • --manifest or -m: Accepts a path to the manifest file. Defaults to manifest.yml in the current directory.

If no other parameters are passed, the command(s) would be executed for all contexts and all tags in the manifest.

Manifest File

The manifest file follows a trickle down format. What this means that all the values in the parent node trickle down but can be overridden by the child node. Here's a sample format.

parameters:
  repository: aa8y/spark
  hadoopVersion: 2.7.4
templates:
  push: docker push {{{repository}}}:{{tag}}
  test: docker run --rm -it {{{repository}}}:{{tag}} spark-shell --version
contexts:
  stable:
    tagKeys:
      - sparkVersion
    templates:
      build: >
        docker build -t {{{repository}}}:{{tag}}
          --build-arg HADOOP_VERSION={{hadoopVersion}}
          --build-arg SPARK_VERSION={{sparkVersion}} {{context}}
    tags:
      '1.6.3':
        hadoopVersion: 2.6.5
      '2.2.0':
      '1.6':
        hadoopVersion: 2.6.5
        sparkVersion: 1.6.3
      '2.2':
        sparkVersion: 2.2.0
      'latest':
        sparkVersion: 2.2.0
  edge:
    parameters:
      scalaVersion: '2.11'
    templates:
      build: >
        docker build -t {{{repository}}}:{{tag}}
          --build-arg HADOOP_VERSION={{hadoopVersion}}
          --build-arg SCALA_VERSION={{scalaVersion}}
          --build-arg SPARK_BRANCH={{branch}} {{context}}
    tags:
      edge-1.6:
        branch: branch-1.6
        hadoopVersion: 2.6.5
        scalaVersion: '2.10'
      edge-2.2:
        branch: branch-2.2
      edge:
        branch: master

Let's go through it part by part. Here's the first part.

parameters:
  repository: aa8y/spark
  hadoopVersion: 2.7.4
templates:
  push: docker push {{{repository}}}:{{tag}}
  test: docker run --rm -it {{{repository}}}:{{tag}} spark-shell --version

This is the first part of the manifest, which refers to the global defaults will get assigned to all contexts and from there to all tags. Supported keys are parameters and templates. parameters contains parameters for the build, pull, push and test templates. And templates contains the Docker build, pull, push and test Mustache command templates. Here, we used the {{{}}} for the repository vs {{}} for the tag as the former contains a / which would otherwise be HTML-encoded. These defaults can be overridden by redefining the keys on any level from where they'll trickle down to the lowest level, i.e. for each tag.

Here's the second part.

contexts:
  stable:
    ...
  edge:
    ...

The second part of the manifest, contexts refers to a Docker context. That basically means the location where the Dockerfile would exist and which would also be the source for the COPY commands in the said Dockerfile. Each context key is automatically assigned as a value to a key called context in the parameters, which can be used in the templates.

Here's the next part.

stable:
  tagKeys:
    - sparkVersion
  templates:
    build: >
      docker build -t {{{repository}}}:{{tag}}
        --build-arg HADOOP_VERSION={{hadoopVersion}}
        --build-arg SPARK_VERSION={{sparkVersion}} {{context}}
  tags:
    '1.6.3':
      hadoopVersion: 2.6.5
    '2.2.0':
    '1.6':
      hadoopVersion: 2.6.5
      sparkVersion: 1.6.3
    '2.2':
      sparkVersion: 2.2.0
    'latest':
      sparkVersion: 2.2.0

tags refers to the tags which would be built. In the aforementioned snippet, we would therefore be building 5 tags for the stable context. The build command for each of these tags would be the one defined in the context templates and the push command would be the one which would trickle down from the global templates discussed before. A tag key gets automatically assigned as a value to the tag key along with any other tag keys defined in tagKeys, if present, and can be used in templates. So, for example, in the above snippet the value of sparkVersion for tag 1.6.3 would be 1.6.3 whereas 2.2 would override the value from its tag-specific parameters.

And here's the last part.

edge:
  parameters:
    scalaVersion: '2.11'
  templates:
    build: >
      docker build -t {{{repository}}}:{{tag}}
        --build-arg HADOOP_VERSION={{hadoopVersion}}
        --build-arg SCALA_VERSION={{scalaVersion}}
        --build-arg SPARK_BRANCH={{branch}} {{context}}
  tags:
    edge-1.6:
      branch: branch-1.6
      hadoopVersion: 2.6.5
      scalaVersion: '2.10'
    edge-2.2:
      branch: branch-2.2
    edge:
      branch: master

In the last part, the only special thing to be seen is a new local global parameter, scalaVersion has been defined. This would then be assigned to each tag and can also be overridden as it is for the edge-1.6 tag.

And while the manifest file can be named anything, the default name assumed is manifest.yml in the current directory. Also, although the sample manifest has keys in lowerCamelCase, lower_snake_case and lower-kebab-case are also supported.

Structure Tests

dave structure-test runs container-structure-test natively against built images, without you having to write a Mustache template for the command. Declare a structureTest block at any level; configs is the only required field.

parameters:
  repository: aa8y/core
templates:
  build: docker build -t {{{repository}}}:{{tag}} {{{context}}}
structureTest:
  configs:
    - script/test/common.yaml
contexts:
  alpine:
    structureTest:
      configs:
        - script/test/alpine.yaml
    tags:
      alpine:
  jdk/8:
    structureTest:
      configs:
        - script/test/jdk-8.yaml
    tags:
      jdk8:

The configs list concatenates as it trickles global → context → tag (unlike every other field, which child-overrides-parent). So in the snippet above, the alpine tag is tested against both common.yaml and alpine.yaml.

The image reference defaults to {{{repository}}}:{{tag}} rendered against the trickled-down parameters. Override it by adding image: '...' inside the structureTest block.

Multiple tags per context

When a context has more than one tag, you can declare configs per tag, or template the path at the context level so it expands per tag. Config paths are Mustache-rendered against the same parameters as build/push templates, so {{tag}}, {{{repository}}}, and any tagKeys you declare all work.

contexts:
  # Per-tag, explicit — use when file names don't follow a pattern.
  multi-explicit:
    tags:
      tag-a:
        structureTest:
          configs:
            - script/test/tag-a.yaml
      tag-b:
        structureTest:
          configs:
            - script/test/tag-b.yaml

  # Templated path at the context level — one line, expands per tag.
  multi-templated:
    structureTest:
      configs:
        - script/test/{{tag}}.yaml
    tags:
      tag-a:
      tag-b:

The two patterns mix freely: a templated default at the context level can be augmented (not replaced) by tag-level configs, since the lists concatenate.

The container-structure-test binary is not bundled with dave — install it separately (brew install container-structure-test on macOS, or download from the release page).

Examples

Here are projects where Dave is being utilized to build, test and push images. See manifest.yml to see how the metadata has been stored.

  • aa8y/docker-scala: A simple Docker project with one Dockerfile (i.e. one context) from which all Docker images are built.

CI Builds

GitHub Actions

Here's an example workflow to use Dave with GitHub Actions:

name: Docker images

on:
  push:
    branches: [main]

env:
  CST_VERSION: 1.22.1

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24
      - run: npm install -g dave
      - name: Install container-structure-test
        run: |
          curl -fsSL -o /tmp/cst \
            "https://github.com/GoogleContainerTools/container-structure-test/releases/download/v${CST_VERSION}/container-structure-test-linux-amd64"
          chmod +x /tmp/cst
          sudo mv /tmp/cst /usr/local/bin/container-structure-test
      - run: dave build
      - run: dave structure-test
      - name: Log in to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      - run: dave push

Drop the CST install + dave structure-test steps if your manifest doesn't declare any structureTest: blocks, and drop the login + dave push steps if you only want to build and test.

Future Work

  • Verify the metadata read from the manifest against a schema. Maybe use JSON Schema?

License

MIT

About

A helper to build, test and push Docker images and template Dockerfiles.

Topics

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors