Hey,
The first time I got involved with Docker was three and a half years ago while I was still an undergraduate student.
By that time, the only thing that used to matter to me in regards to Docker was the fact that I was able to run “a minimalistic VM” (which is almost an absurd way of referring to Docker or Linux containers in general).
Remember the time when the blogs were all about “how a container compares to a VM”?
Some years later (having dealt with Docker in production a lot), the most interesting piece for me (and others it seems) became the container image - this immutable thing that carries all of your dependencies - create a little recipe specifying the steps to reach the final desired state and ship that packed filesystem. A “Tarball++".
In regards to Golang, do we need much effort to create a working image? Totally not!
Here in this blog post, we go through:
The file structure of a common Golang project
For almost every Go project, I tend to always structure my Golang application in the same way:
.
├── .editorconfig
├── Dockerfile # By placing the Dockerfile on the root
│ # we can make it very simple - when executing
│ # the `docker build` command from the root
│ # the context of the build will carry all the
│ # relevant Go files and dependencies (assuming
│ # you have a `vendor` directory at least).
├── Makefile
├── vendor
├── lib # A sample Golang package (you'd probably not call
│ ├── foo.go # it `lib`, but something more meaningful).
│ ├── foo_test.go
│ ├── something.go
│ └── something_test.go
├── anotherpackage # Another go package.
│ ├── bar.go
│ └── bar_test.go
├── main.go # The `main` package.
└── VERSION
This way, I have to make minimal changes to this Dockerfile and the Makefile that I use (check the Makefile at Minimal Golang Makefile).
By keeping it simple like that we can build almost any kind of Docker images for all sorts of Golang projects without having to learn weird Makefile syntax or something complex for such a simple task.
An initial approach to a Golang Docker Image
If we take a multi-step approach to reach a very minimal Go Dockerfile, we can start with a path where we pack all the Golang source into it, build the code and then we’re done with it.
# Retrieve the `golang:alpine` image to provide us the
# necessary Golang tooling for building Go binaries.
#
# Here I retrieve the `alpine`-based just for the
# convenience of using a tiny image.
FROM golang:alpine
# Add the `main` file that is really the only Golang
# file under the root directory that matters for the
# build
ADD ./main.go /go/src/github.com/cirocosta/l7/main.go
# Add all the files from the packages that I own
ADD ./lib /go/src/github.com/cirocosta/l7/lib
# Add vendor dependencies (committed or not)
# I typically commit the vendor dependencies as it
# makes the final build more reproducible and less
# dependant on dependency managers.
ADD ./vendor /go/src/github.com/cirocosta/l7/vendor
# 0. Set some shell flags like `-e` to abort the
# execution in case of any failure (useful if we
# have many ';' commands) and also `-x` to print to
# stderr each command already expanded.
# 1. Get into the directory with the golang source code
# 2. Perform the go build with some flags to make our
# build produce a static binary (CGO_ENABLED=0 and
# the `netgo` tag).
# 3. copy the final binary to a suitable location that
# is easy to reference in the next stage
RUN set -ex && \
cd /go/src/github.com/cirocosta/l7 && \
CGO_ENABLED=0 go build \
-tags netgo \
-v -a \
-ldflags '-extldflags "-static"' && \
mv ./l7 /usr/bin/l7
# Set the binary as the entrypoint of the container
ENTRYPOINT [ "l7" ]
While this is an approach that indeed works, it has two drawbacks:
- it makes you have to update the Dockerfile whenever a new package is created, or the filestructure changes, and
- it ends up in an image that contains the entire Go compiler, Make and other tools that are not needed for the consumer of this image.
We can make it better.
Improving the Golang Dockerfile using multi-stage builds
One improvement that we can perform is making use of multi-stage builds.
This tackles the second drawback outlines in the last section.
The whole idea is that you can separate your build process into stages such that each stage marks an entirely new base image which can COPY files from previous stages.
FROM golang:alpine AS builder
ADD ./main.go /go/src/github.com/cirocosta/l7/main.go
ADD ./lib /go/src/github.com/cirocosta/l7/lib
ADD ./vendor /go/src/github.com/cirocosta/l7/vendor
RUN set -ex && \
cd /go/src/github.com/cirocosta/l7 && \
CGO_ENABLED=0 go build \
-tags netgo \
-v -a \
-ldflags '-extldflags "-static"' && \
mv ./l7 /usr/bin/l7
# Create the second stage with the most basic that we need - a
# busybox which contains some tiny utilities like `ls`, `cp`,
# etc. When we do this we'll end up dropping any previous
# stages (defined as `FROM <some_image> as <some_name>`)
# allowing us to start with a fat build image and end up with
# a very small runtime image. Another common option is using
# `alpine` so that the end image also has a package manager.
FROM busybox
# Retrieve the binary from the previous stage
COPY --from=builder /usr/bin/l7 /usr/local/bin/l7
# Set the binary as the entrypoint of the container
ENTRYPOINT [ "l7" ]
Just by adding this extra stage you can see the differences in practice.
At least the size of the Go toolchain will be reduced (that’s at least 30MB).
ps.: it’s not everytime that 30MB makes a difference. I’m sure that early optimization is a problem, but this is such a straightforward optimization that everyone can take it without much thought.
The minimal Golang Dockerfile
The final optimization missing is reducing those lines that add specific directories to the builder stage.
While that’s something useful to be done when you have RUN steps in between (it can let you cache directories and avoid re-running build steps all the time), in this case, it’s not very useful if you only have a handful of dependencies and no big files in the repository.
To have an even simple Dockerfile, we can then replace those adds by a wildcard ADD ./ that simply puts everything we have into the image build context:
FROM golang:alpine AS builder
# Add all the source code (except what's ignored
# under `.dockerignore`) to the build context.
ADD ./ /go/src/github.com/cirocosta/l7/
RUN set -ex && \
cd /go/src/github.com/cirocosta/l7 && \
CGO_ENABLED=0 go build \
-tags netgo \
-v -a \
-ldflags '-extldflags "-static"' && \
mv ./l7 /usr/bin/l7
FROM busybox
# Retrieve the binary from the previous stage
COPY --from=builder /usr/bin/l7 /usr/local/bin/l7
# Set the binary as the entrypoint of the container
ENTRYPOINT [ "l7" ]
Some remarks
You might want to end up with a FROM alpine instead of FROM busybox and then run apk add --update ca-certificates if your binary needs to perform requests to HTTPS endpoints - just using busybox will lead to an image that doesn’t contain root CA certificates which would make HTTPS requests fail.
If you’re interested in learning how to deal with Docker, Golang or just wanting to learn more about software engineering in general, make sure you subscribe to the mailing list!
In case of questions or if you spot something odd, please get in touch with me at cirowrc on Twitter.
Have a good one!
finis