Self-hosting git and builds without running a bunch of web services

Infrastructure
git
Docker

My main side project at the moment is this blog. I tinker away with it when I feel like it. I'm not working on it with anyone else, not fielding pull requests, and not writing documentation.

That said, I do still like having it in version control. I like having a Dockerfile to encapsulate dependencies and configuration, and using docker compose as a portable runtime manager. It's probably not everyone's idea of a simple setup, but I like it.

The blog is deployed to a small VPS in Hetzner, running behind nginx.

GitHub, Github Actions, and the GitHub Container Registry have been how I've been handling builds, but it's been slow, and it's started to seem absurd to have all these builds occurring in some random datacenter presumably on the other side of the Atlantic ocean, that I am then pulling back to a server in Europe.

I started fishing around for a locally hosted replacement, coming across Forgejo, Woodpecker CI, and OneDev in addition to more familiar names like GitLab and SourceHut.

At some point it occurred to me that this was a lot of ceremony for something relatively simple. How hard could it be just to have a git remote and hang some builds off it? I'm not trying to launch my own GitHub here.

I want to:

  1. git push to a central repo
  2. When I push, it kicks of a container image build
  3. That image is pushed to an image registry that I can deploy from

This does not need several UIs and databases. In any case git being git, you can always switch out for something more complex later.

#Hosting a git repo

On GitHub, an SSH clone URL looks something like:

git@github.com:duggan/duggan.ie.git

It's an SSH URI just like it would be with scp or rsync. A helpful breakdown from Mike Slinn and Jaditpol on StackOverflow:

    git@github.com:myuser/myrepo.git
    \_/ \________/ \_______________/
     |      |              |
    user   host           path

Since I'm the only user, I don't need to get fancy, and I can just use my own login.

The origin of a git repo is more or less just the contents of the .git directory in a remote location. That's it. You don't even need to run a git server if you're happy enough using ssh for transport.

I put it outside my home path just to keep it a little out of the way, as I won't need to interact with it too often:

sudo mkdir -p /srv/git
sudo chown -R $USER:$USER /srv/git
cd /srv/git
mkdir example.git
cd example.git/
git init --bare

In my case I've replaced my GitHub origin with this one directly in the .git/config of my local checkout, but when I was testing the waters I just added an additional remote:

$ git remote add test ross@buildmachine:/srv/git/duggan.ie.git

#Building an image on push

Git's own hooks system, combined with a Makefile and Docker, is enough to put together a pretty flexible build system for anything you can deploy using a container.

In my newly minted upstream repo, I added a file named post-receive into the hooks directory, i.e., /srv/git/duggan.ie.git/hooks/post-receive (and chmod +x it):

#!/bin/bash

set -e

# Read the push info (branch being updated)
while read oldrev newrev refname; do
    # Only trigger on main branch
    if [[ $refname == "refs/heads/main" ]]; then
        echo "=== Build triggered for main branch ==="

        # Get repo name from current directory
        REPO_NAME=$(basename "$(pwd)" .git)
        WORK_TREE="/tmp/git-build-${REPO_NAME}"
        GIT_DIR=$(pwd)

        # Clean checkout
        rm -rf "$WORK_TREE"
        mkdir -p "$WORK_TREE"
        git --work-tree="$WORK_TREE" --git-dir="$GIT_DIR" checkout -f main

        cd "$WORK_TREE"
        # Check for Makefile and build target
        if [[ -f Makefile ]]; then
            if grep -q "^build:" Makefile; then
                echo "=== Running make build ==="
                make build
                echo "=== Build complete ==="
            else
                echo "Makefile found but no 'build' target, skipping"
            fi
        else
            echo "No Makefile found, skipping build"
        fi

        # Cleanup
        rm -rf "$WORK_TREE"
    fi
done

When I push to the repo, this looks for a Makefile in the root of the repo, checks for a build command and executes it.

My Makefile is super basic, just:

IMAGE=localhost:5000/duggan.ie:latest

build:
	docker build -t $(IMAGE) .
	docker push $(IMAGE)

As a bonus, I could have thrown that post-receive script into a git template so that new repos are automatically created with a copy of the post receive script, but it doesn't seem necessary just yet.

#Hosting an image

At this point the image is built, which will be fine if I just wanted to run the container on the same machine, but I need to deploy this to my little Hetzner VPS.

This is where Tailscale comes in handy, as it lets me have Docker's own container registry running on my build server, which I then pull from on the VPS. It's very straightforward to set up.

On the build server I have a docker-compose.yml file with the registry configured:

 services:
  registry:
    image: registry:2
    container_name: registry
    restart: always
    ports:
      - "5000:5000"
    volumes:
      - ./data:/var/lib/registry
    environment:
      REGISTRY_STORAGE_DELETE_ENABLED: "true"

#Deploying from a private registry

With the registry and VPS both on the Tailscale network, I can allow regular HTTP traffic, which requires a small tweak to Docker on the VPS.

In /etc/docker/daemon.json:

{
  "insecure-registries": ["buildmachine:5000"]
}

With that done, I'm able to update my make deploy command to point at the new registry:

services:
  web:
    image: buildmachine:5000/duggan.ie:latest

This all works much more quickly than before, and since it's running the builds over SSH directly, I'm getting feedback faster than I would refreshing a build log.

There is currently no comments system. If you'd like to share an opinion either with me or about this post, please feel free to do so with me either via email (ross@duggan.ie) on Mastodon (@duggan@mastodon.ie) or even on Hacker News.