Skip to main content

Docker Build Secrets

Some Docker builds need to consume sensitive data, for example a token to install a library from a private Python registry. Passing such a value as a --build-arg is a poor security practice: the value is typically persisted in the final image or in a build cache layer.

Use Docker build secrets instead. They expose the value to a single RUN instruction and never write it to the image or the cache.

Configuring a build secret on AIchor​

On AIchor, build-time environment variables are configured under the project's Environment Variables tab, in the Build variables section. They can also be marked as sensitive. See Environment Variables for how to create them.

For the examples below, imagine the build-time environment variable PIP_INSTALL_ARG is set at the project-level to :

https://__token__:<token>@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple

Consuming the secret in a Dockerfile​

Add a RUN instruction that mounts the secret and sources it into the shell environment:

RUN --mount=type=secret,id=_env,dst=/etc/secrets/.env . /etc/secrets/.env \
    && pip install --index-url $PIP_INSTALL_ARG ridl==x.x.x

Breaking that down:

  • --mount=type=secret,id=_env,dst=/etc/secrets/.env is the Docker build secret mount. The id passed by AIchor is always _env. The dst path can be any path you choose.

  • AIchor generates a .env-style file at dst containing every build variable set in the UI, for example:

    PIP_INSTALL_ARG="https://__token__:<token>@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple"
    OTHER_ENV_VAR="Something else"

The file is available only to this RUN instruction and is never persisted in a layer or the build cache.

  • . /etc/secrets/.env is the POSIX way to load the file's variables into the current shell.
  • pip install --index-url $PIP_INSTALL_ARG ridl==x.x.x installs the package using the --index-url provided by the secret.
Cache invalidation

Because the secret is excluded from the build cache, changing a build variable in the UI does not invalidate subsequent builds on its own. If you add a second token for another library, update the pip install command (typically by adding the new package or bumping a pinned version) so Docker reruns the layer. Pinning library versions and bumping them on change is a good habit for this reason.

In a CI context this enables one-time-token strategies without invalidating the Docker cache when library versions are unchanged.

Building the Dockerfile locally​

When building a Dockerfile that uses RUN --mount=type=secret,id=_env outside of AIchor, generate the .env file from your local shell and pass it via --secret:

echo "PIP_INSTALL_ARG=\"$PIP_INSTALL_ARG\"" > /tmp/.env
DOCKER_BUILDKIT=1 docker build --secret id=_env,src=/tmp/.env .

PIP_INSTALL_ARG must already be exported in your shell. DOCKER_BUILDKIT=1 enables BuildKit, which is required for Docker build secrets.

Tip: passing a secret file​

If a step needs the secret as a file (e.g. a service-account JSON), base64-encode it locally:

cat my_secret_file.json | base64

Set the result as a build variable (for example GCP_SA_BASE64), then decode it inside the RUN instruction and clean it up before the layer finishes:

RUN --mount=type=secret,id=_env,dst=/etc/secrets/.env . /etc/secrets/.env \
    && echo "$GCP_SA_BASE64" | base64 -d > /gcp_sa.json \
    && pip install pcb-envpool --extra-index-url https://europe-west4-python.pkg.dev/instadeep/pcb-envpool/simple/ ; \
    ret=$? ; rm /gcp_sa.json ; exit $ret

The trailing rm /gcp_sa.json and exit $ret ensure the decoded file is removed even if pip install fails, while preserving its exit code.