Most of the time I see myself creating a Golang project: be it a one-file single-purpose thing or a more complex project.

To facilitate that process, I try always to keep the structure the same such that there’s no need to put much thought into the process of bootstrapping a new project.

The file structure usually looks like this:

.
├── .editorconfig
├── Dockerfile
├── Makefile
├── VERSION
├── lib
│   ├── foo.go
│   ├── foo_test.go
│   ├── something.go
│   └── something_test.go
└── main.go

Some things to pay attention:

  1. editorconfig is a real deal - I use vim with set shiftwidth=2; set tabstop=2, that is, 2 as the indentation. For go, tab with eight spaces as indentation.

  2. be it a cli tool or a library, I always make use of a main.go in the root, so that’s it’s obvious how to fetch it - go get -u <repo>.

Snippet

Given the file structure described above, below is the Makefile.

In this case, I extracted the Makefile from a test project I started some time ago (a “load-balancer” that makes use of fasthttp: cirocosta/l7).

# I usually keep a `VERSION` file in the root so that anyone
# can clearly check what's the VERSION of `master` or any
# branch at any time by checking the `VERSION` in that git
# revision
VERSION         :=      $(shell cat ./VERSION)
IMAGE_NAME      :=      cirocosta/l7

# As a call to `make` without any arguments leads to the execution
# of the first target found I really prefer to make sure that this
# first one is a non-destructive one that does the most simple 
# desired installation. It's very common to people set it as `all`
# but it could be anything like `a`.
all: install

# Install just performs a normal `go install` which builds the source
# files from the package at `./` (I like to keep a `main.go` in the root
# that imports other subpackages). As I always commit `vendor` to `git`
# a `go install` will typically always work - except if there's an OS
# limitation in the build flags (e.g, a linux-only project).
install:
	go install -v

# keeping `./main.go` with just a `cli` and `./lib/*.go` with actual 
# logic, `tests` usually reside under `./lib` (or some other subdirectories).
# Here we could do something like `find . -name "*" -type d -exec ...` but IMO
# that's unnecessary. Just `cd`ing to what matters to you is fine - no need to
# handle the case of directories that you don't want to execute a command.
test:
	cd ./lib && go test -v

# Just like `test`, formatting what matters. As `main.go` is in the root,
# `go fmt` the root package. Then just `cd` to what matters to you (`vendor`
# doesn't matter).
fmt:
	go fmt
	cd ./lib && go fmt


# This target is only useful if you plan to also create a Docker image at
# the end. 
# I really like publishing a Docker image together with the GitHub release
# because Docker makes it very simple to someone run your binary without
# having to worry about the retrieval of the binary and execution of it
# - docker already provides the necessary boundaries.
image:
	docker build -t cirocosta/l7 .


# This is pretty much an optional thing that I tend to always include.
# Goreleaser is a tool that allows anyone to integrate a binary releasing
# process to their pipelines. Here in this target With just a simple 
# `make release` you can have a `tag` created in GitHub with multiple
# builds if you wish. 
# See more at `gorelease` github repo.
release:
	git tag -a $(VERSION) -m "Release" || true
	git push origin $(VERSION)
	goreleaser --rm-dist

.PHONY: install test fmt release

Save that content in the Makefile file in root directory of the project, create a VERSION file with something like 0.0.1 (semver) and you’re ready to go.

Naturally, vendor directories and all of that is already covered.

Projects with many packages and vendor dependencies

In bigger projects, it might happen that you have a bunch of packages and they’re not as flat as I described here.

Then it becomes cumbersome to have to update the Makefile every time that you need to create a new package, making maintenance tricky.

One way of making this more generic is adapting our Makefile to handle both nesting and multiple packages more intelligently.

The first thing we could try is getting the path to every directory that has a Golang file and either perform a fmt or test there.

Naturally, this doesn’t work very well if you have vendored your dependencies, so you have to have extra care to remove the vendor directory from the search.

One way of achieving that is using a combination of the find and xargs utilities:

# Search for every file that matches the pattern (ends with `.go`)
# that does not match the path `./vendor/*`.
#
# For every file path matched, extract the filename and keep only
# the directory  - `dirname` does this.
#
# As we have multiple files under the same directory, there'll be
# many duplicates. Piping to `uniq` allows us to filter out
# those repetitions.
find . -name "*.go" -not -path "./vendor/*" | \
        xargs -I {} dirname {}  | \
        uniq

Tieing that with our Makefile, we can set two variables: one that contains all the directories that include tests and another that contains all the directories with any Golang file:

GO_SRC_DIRS := $(shell \
	find . -name "*.go" -not -path "./vendor/*" | \
	xargs -I {} dirname {}  | \
	uniq)

GO_TEST_DIRS := $(shell \
	find . -name "*_test.go" -not -path "./vendor/*" | \
	xargs -I {} dirname {}  | \
	uniq)

# Shows the variables we just set.
# By prepending `@` we prevent Make
# from printing the command before the
# stdout of the execution.
show:
	@echo "SRC  = $(GO_SRC_DIRS)"
	@echo "TEST = $(GO_TEST_DIRS)"

Now, testing it:

make show

SRC  = ./lib ./lib/sub ./lib .
TEST = ./lib

With those lists as variables, we can make use of them as dependencies for our new fmt and test targets:

test: $(GO_TEST_DIRS)
	@for dir in $^; do \
		pushd ./$$dir > /dev/null ; \
		go test -v ; \
		popd > /dev/null ; \
	done;

fmt: $(GO_SRC_DIRS)
	@for dir in $^; do \
		pushd ./$$dir > /dev/null ; \
		go fmt ; \
		popd > /dev/null ; \
	done;

As $^ targets all the dependencies, on both targets we’re able to use the selector to iterate over the directories of each.

Note that I’m not using cd to get into the directories. That’s because not knowing how deep in the file structure we’ll go, just stacking the directory changes with pushd is easier as to get back to the original place we just need to popd.

Closing thoughts

I think it’s very handy to keep a standard way of performing basic operations across multiple repositories. In my experience this reduces the friction of moving from one project to another. Having a common flow of how to build, create an image and publish a project using a Makefile has helped me in such area.

Do you think the same? What are your thoughts?

Reach me on Twitter at any time @cirowrc and subscribe to the list if you’re a Golang developer or simply likes software stuff!

finis