Chameth.com - posts like docker-automatic-nginx-proxy, g15-ram-upgrade, migrating-from-github-to-forgejo, why-you-should-be-using-https but not just-a-nod

Building a family of single-user services

2026-08-27T00:00:00Z

Last month I made and wrote about a personal service to track my workouts and record personal bests. Since then, things have escalated. I now have eight different services running in the same “family”:

Name	Purpose
Collegiate Room	Simple weather overview for my current location
German Mall	LLM broker for use by other services
Gloss White	Weight tracking
Olive Crosses	Central service hub
Pompei Band	Workout and personal best tracking
Speed Carp	Spaced repetition system
Tiger Island	Reading history and library tracking
Typical Horse	Location history and analytics

The names are all anagrams of something related to their function. Finding those anagrams brings me an unreasonable amount of joy. Unscrambling them is an exercise left to the reader. On top of those eight, I currently have four other service ideas sat in my todo list. It’s going to be a big family.

A postmortem on the previous hodge-podge

Most of this functionality isn’t new to me. When I started out playing around with LLMs, I created a “personal data aggregator” that just kept getting things bolted on. I said at the time:

My biggest just-for-me project is a web app that I started to help me aggregate film recommendations. It’s since morphed into a general personal data aggregation service: it deals with data from GitHub, Todoist, Letterboxd, TMDB, Healthkit, and others. It also lets me make re-orderable lists, store recipes, and more. Parts of this could definitely be open sourced, and I might carve them out at some point, but it’s mostly a glorious hodge-podge of things specific to me. Having all these services in one place lets me make quick and dirty automations, for example: when I create a Todoist note on my phone or watch, I often forget to set the due date, so it doesn’t show up in the “Today” view. It was literally a few lines of code to plumb things together so any inbox task without a due date gets set to today automatically.

In a twist that nobody could see coming, it turns out that a “glorious hodge-podge” is actually quite hard to maintain and very annoying to work with. It didn’t help that it had a fairly extensive Svelte frontend and I really am not a fan of JavaScript. I ended up using it less and less just because it wasn’t fun to use or work on, and I didn’t want to waste the limited LLM tokens on doing massive refactors or reworks.

I’ve got almost two decades of professional software development experience under my belt, so you’d think I would have learnt the lessons on how to write maintainable software. Hell, my CV even mentions that as something I’m particularly good at. So what went wrong?

There were a few factors. One of the obvious things to blame is that it was written mostly with LLMs. Most of that blame isn’t very honest, though. In the year or so since, LLMs have definitely got better at writing good code; but I was manually reviewing all the code at the time, and telling it quite carefully what to implement and how to do it. So the buck really stops with me.

I think it’s actually more because it was my first larger project using LLMs. It’s a bit like learning a new programming language. I don’t think the first sizable project I’ve written in any language has ever been any good. You can read and learn as much as you want, but you need to actually run into some problems headfirst to get a grip on how the language works. Or at least I do. LLM coding agents are basically the same deal.

What the LLMs actually did was mask several problems. The architecture wasn’t right for how the service developed. It started out as a single-purpose app and gradually got things bolted on, without any major changes. Everything ended up in horizontal slices: a massive database layer with every query used by every part of the app, a massive HTTP layer, and so on. Working on a single feature became a tap dance all over the source tree. This sort of friction is normally quite apparent when you’re writing code by hand, and at some point it hurts enough that you throw in the towel and do some refactoring. But the LLM will happily plod on without exposing that pain, so the project needed a more active kind of architecture management that it just didn’t get.

The other problem they covered up was how brittle and unmaintainable the JavaScript frontend was. I’m not even sure how I ended up with a Svelte frontend in hindsight. It’s my preferred framework, but I still only reach for it as a last resort when there’s no other way to avoid using one. It seems like there were lots of ways to avoid it here, but I seem to have… well… avoided them. Anyway, each feature got its own slew of Svelte components, without much thought to reusability, shared patterns, or anything sensible. Some of them were just thin wrappers around data supplied by Go, others had vast swathes of logic in them, with no real rhyme or reason as to which way that would fall. It’s the same issue as before: a human would look at it, say “WTF?” and suggest tidying it up; instead, the LLM just gets on with what you tell it to do.

There’s also a big non-LLM factor: one of the primary reasons all of this unrelated functionality got tossed into the same app was that I’d previously worked on a few projects with a microservice architecture and really didn’t like it. My conclusion was they only really worked in large environments where you can dedicate people or teams to working on all the boilerplate that happens from maintaining lots of interconnected projects. Dumping everything and the kitchen sink into one badly architected monolith was, perhaps, not the best rebound from that.

Pompei Band and beyond

When I wanted to implement ~~Strava at home~~ my PB tracking app, it seemed fairly natural to build it out as a new service, rather than bolt it on to the slowly decomposing hodge-podge that was giving off a funny smell. Then when I wanted a weather dashboard, why not do it in the same style as Pompei Band? The same serving architecture, the same page layout… I could even come up with a similar weird two-word name!

Then wouldn’t it be nice if the weather dashboard knew where I was, so it could show the local weather? I could just build that functionality in, but it’d also be a good opportunity to carve out the location history feature from my old service and make a new, shinier one. Then basically every time I even thought about going near the old service, I mentally sketched out a replacement service.

Olive Crosses showing links to the other services

At one point I had the thought that it’d be nice if the services could all link to one another, but I didn’t want to have to touch every service whenever I added a new one. So I made a “hub” service, which maintains the One True Catalogue, and provides a bit of JavaScript the others can all embed to add a navigation sidebar.

So after complaining about microservices and JavaScript, it sounds like I have landed on microservices (or at least miniservices) and JavaScript? Sort of yes, but sort of no. Each service is still its own thing and makes sense independent of the others. There are some cross-service dependencies, but they’re very simple and don’t have the same cognitive or boilerplate overhead as a login service that talks to an authentication service that queries a user manager and so on. And as for the JavaScript, it’s almost exclusively used for progressive enhancement and a few limited client-side visual things like maps and graphs. All of the logic lives in Go, which is a lot easier to maintain and test, and all of the pages are simple Go templates.

Having these services built in a way that I actually like has motivated me to play around a lot more with new features:

My workout tracking app pulls photos from my private Immich server that were taken during workouts, and shows them alongside. Now I can see all my obligatory end-of-run selfies in one place.
The location history app can detect trips away from home, and get an LLM to automatically title them. My most recent, very exotic, trip has the title “A fortnight in Milton Keynes and Coventry”. Each trip gets its own little map, and in the future I want to pull in images like with workouts.
My weight tracking app now exports weigh-ins to Garmin, as it doesn’t fully integrate with Apple Health (it will export its own weights, but not import others, like the ones from my Withings scales). Stupid walled gardens, but thankfully I can fix it myself.
My book tracking app automatically cleans up the titles of books from the nonsense provided by Amazon to something more palatable. “Killing Eve: Long Shot: A twist on obsession, betrayal and revenge from Luke Jennings, author of the TV sensation Killing Eve” becomes “Long Shot (Killing Eve #5)”. It does this by passing the title to an LLM with a bunch of rules.
The hub generates me a snippet of Tailscale configuration that defines the minimal grants needed to allow the intra-service dependencies to work.

The LLM integrations were inspired by my friend Shane who recently demonstrated an inventory tracking app he’d made, which used Haiku to process images of boxes and describe the contents. I’ve dabbled with LLM integration before, and always disliked the process of fiddling with the prompts, and debugging when the LLM gave a nonsensical response. So I built German Mall, which makes it a lot easier. It lets me define a “workflow” consisting of a system prompt, initial message, and expected response format. I can browse past runs, tweak the prompt, rerun them to see the change, all without touching the services using them. Quick iteration loops are the best. German Mall also provides markdown documentation for each workflow, so integrating it downstream is simply a case of pointing a coding agent at the right documentation endpoint.

The “tidy book titles” workflow in German Mall

It’s not all fun and games: having a bunch of services all following the same pattern means you need to do cross-cutting maintenance on them now and then. But that’s surprisingly easy with LLMs. I put all the projects in a common folder, and I launch cross-cutting tasks from that folder. Usually I’ll make the change in one project, and then the cross-cutting task is simply “Look at commit XYZ in ./foo, make the same change to the other projects”.

I’ve also ended up with a shared library used by all the projects to cut down on some boilerplate. That’s normally something I’d resist, especially with personal projects, just because the extra overhead of working in a separate project, managing releases of it, and so on, is a bit annoying. But, again, LLMs can do almost all of that grunt work. They effectively reduce the “cost” of maintaining separate services to a negligible level, leaving me to focus on the more fun aspects like coming up with new names.

Monthly Meanderings: April 2026

2026-05-01T00:00:00Z

It’s apparently been a whole month since the last edition of Monthly Meanderings. Not sure when that happened. This month I’ve been to a LAN party, had an MRI, and have been trying unsuccessfully to get my doctor to change a prescription. The latter two are not doing much to help shake that middle-aged feeling I mentioned last month…

Website updates

Another pair of blog posts for this month: The Case of the Unchanging Config about a problem I had with a bind-mounted config in a Docker container, and Migrating from GitHub to Forgejo, a write-up of… well… migrating from GitHub to Forgejo. That one’s been on my to-do list for a while, so it’s nice to finally get it written up.

It’s not visible when you look at the site, but I’ve been doing a lot of refactoring this month. The existing design had a package for database operations, a package for the stylesheet, a package for shortcodes, and so on. As I bolt more and more things on this gets less and less useful: those packages get bigger and bigger, and trying to understand how, say, the music import functionality works means hopping all over the place. Instead, I’m moving towards having each feature contained in its own package. It defines its own stylesheet, database operations, HTTP handlers, and so on. They’re then all wired together at the top layer. There’s still some more to do on that front, but it’s progressing in a good direction, and I’m happier with the code.

The only actual visible change is the addition of a page dedicated to World of Warcraft that automatically imports some data from the game’s API. We’ll come back to that later!

Other projects

I made a trio of new tiny projects: pdsps, a reverse proxy for a Bluesky/ATProto PDS that overrides the age verification response so you don’t need to hand over ID; irc-adze, a plugin for irc-bot that receives and announces adze webhooks; and wow-spec-switch, a super-simple addon for World of Warcraft that lets you switch specialisation with a command.

I’ve also been working on a custom app launcher in the style of ulauncher but without requiring WebKit to render things. I’ve not yet polished it up enough to open source it, but it’s not far off.

Entertainment

Another single film month for me. It was a great one, though:

Project Hail Mary

2026-04-01

A film adaptation of a book I loved that’s actually good. Huh!

Ryan Gosling does great in what is often a solo act. Early on I thought the film was trying a bit too hard to be funny, but at some point it started working and I enjoyed it.

There are lots of parallels to The Martian, Arrival, and so on, but it’s a sufficiently different mash-up of the concepts that it’s not a problem.

I’m still enjoying SNL UK a lot more than I expected to, although I keep forgetting what day it airs. They should put it in the name or something.

The rest of my “entertainment” time has been spent playing World of Warcraft. I played an awful lot about 6-7 years ago, and have dipped in and out occasionally since. I got an MMORPG itch again recently and jumped back in, and I’m enjoying it a lot once again. I’ve been playing a healer properly for the first time (after enjoying healing in Final Fantasy XIV):

Level 90

Female Void Elf

Holy Priest

280 average item level

Methrica-Terenas

Professions

Midnight Alchemy		100/100
Midnight Engineering		92/100
Midnight Cooking		1/100
Midnight Fishing		4/300

View on blizzard.com View on raider.io

Around the web

git-history documentation

The latest release of git added this handy-seeming history subcommand. It allows you to reword a commit, or split a commit up interactively, without having to go through the interactive rebase song and dance.

Git has acquired a bunch of new commands in recent releases, but I tend to ignore them in favour of the old ones I know well. This one will definitely get use from me, though.

Git Koans by Steve Losh

Sticking with the same theme, this series of koans about Git and its various quirks had me laughing. If you’re reading this blog, you’re probably part of the target audience for it.

SQLite prefixes its temp files with etilqs_

I think I’d come across this before, but it’s still a fun anecdote. It reminds me of Daniel Stenberg’s e-mail collection; as the author of curl his name is listed on basically everything with the inevitable influx of people contacting him.

A dot a day keeps the clutter away

I love this approach to tracking how frequently things are used, and the weird sort of archaeological record you get from rotating the colours of the stickers used.

Migrating from GitHub to Forgejo

2026-04-30T00:00:00Z

When Microsoft bought GitHub in 2018 my kneejerk reaction — like so many others — was to start looking for alternatives. For a while I self hosted a Gitea instance but I never totally bought into it: some repositories I still pushed to GitHub, some I pushed to Gitea and they got mirrored, and I ended up causing myself problems when I got the two confused. Part of the problem was that the GitHub UI was faster and cleaner than Gitea’s at the time; using Gitea felt like a chore compared to GitHub. I ended up not maintaining it and eventually binning it and just going back to GitHub.

An all-too familiar unicorn

Fast forward eight years, and GitHub is about what we all imagined when Microsoft bought it. If you take the most pessimistic way of counting, they have zero nines of reliability¹. If you take the most generous, they have a single nine. That’s around 30 minutes of downtime every day. It feels like it must be more than that, given how many times you see the damned unicorn.

Uptime aside, they’ve crowbarred LLMs in all over the place; they don’t have the time to maintain official GitHub actions; the ones they do maintain have Copilot running roughshod all over them²; actions themselves are slow to run³, often have weird transient failures, and there are so many footguns that I’m pretty sure GitHub actions should be an entire category in the CWE top 10.

Speaking of security, while I was drafting this post a remote code execution was reported and the details are… impressive. The actual security issue was a pretty stupid oversight. Those happen. But they end up being able to execute code as a globally shared git user with access to all the other repositories on the node. What? I can’t quite get my head around it. No sandboxing, no containers, it just runs as a git user?

So, yeah, I’m not a fan of GitHub in its current state. A few months back I took the plunge and set up Forgejo, a fork of Gitea where they did radical things like add tests and ensure that it’s community operated not twisted for commercial use. Forgejo also potentially has some fun security problems⁴, but the way I host it mitigates more-or-less all problems. It’s also slightly less shocking when an open source project has lapses in basic security versus a $2 trillion corporation.

It’s taken a bit of doing, but I’m very happy with the setup I have.

The basic setup

I run all my server software using Docker, and Forgejo is no exception. They have decent documentation on how to get it running, and publish rootless images which is nice (and diminishes my urge to make an artisanal version myself). You can set configuration options using environment variables, which is perfect for containers, although the names can end up a bit unwieldy. Like this thing: FORGEJO__repository.signing__SIGNING_NAME=Chris Smith. It gets the job done, but the environment-to-ini mapping is pretty ugly.

I didn’t want to expose Forgejo publicly, as I wanted to avoid having to try and secure it⁵, or dealing with bots scraping it, or users signing up, and so on. I wanted somewhere to store my git repositories, handle CI for me, and mirror them somewhere public for other people to see and interact with. If you’ve read some of my other posts you’ll probably guess where this is heading: Tailscale. Alongside the Forgejo container, I run a Tailscale instance with a serve.json that covers both the web frontend and the SSH listener used for git operations:

{
    "TCP": {
        "22": {
            "TCPForward": "forgejo:2222"
        },
        "443": {
            "HTTPS": true
        }
    },
    "Web": {
        "${TS_CERT_DOMAIN}:443": {
            "Handlers": {
                "/": {
                    "Proxy": "http://forgejo:3000"
                }
            }
        }
    },
    "AllowFunnel": {
        "${TS_CERT_DOMAIN}:443": false
    }
}

This allows me to interact normally with Forgejo from any of my devices or servers (which all run Tailscale), while not exposing it to the Internet at all. It also handles the TLS certificates automatically, which saves me setting up something else to do that.

For CI, I run the Forgejo runner image on the same server, and gave it a dind container to use to actually run the workloads. dind is the cutesy name for Docker-in-Docker, basically a Docker daemon running inside a Docker container. That keeps the CI workload separated nicely from the “production” workload running on the server, which is nice from both a security and a monitoring point of view.

The uncanny valley of actions

Forgejo actions are basically like GitHub actions: they run a series of steps using one or more container images. You define those steps in the exact same clunky YAML format as with GitHub. The big difference between them is that GitHub defaults to using an absolutely massive base image filled with out-of-date preinstalled software, and Forgejo leaves that for the administrator to configure. You could, of course, just use the same image as GitHub, but lugging around a multi-gigabyte image that doesn’t even have recent versions of the software I want isn’t really my style. Instead, I set about using a plain debian image as the base, and then did a face-palm when I tried to run a “normal” action and remembered they’re all written in JavaScript, so expect node to exist.

Accommodating JavaScript isn’t really my style, either, so I did the very sensible thing of writing my own suite of actions in Go. Being written in Go means they can be compiled statically, published as a container, and manage their own very limited dependencies. For example, the checkout action is built into an alpine container with git added, while the dockerbuild action uses the buildah image as a base. Each image has the tools it needs, pinned to a recent version, and nothing more. This is very much not a sensible approach to take for most people, but it made me happy. With six basic actions, I could replace almost all the ad-hoc workflows I had in GitHub⁶.

Other than the base images and my self-inflicted JavaScript machinations, everything works the same as GitHub. But faster. So much faster. Most of my CI workflows finish on Forgejo before GitHub would even have allocated a runner to the job⁷. And I can actually view the logs without them glitching out.

Step aside Dependabot, here comes Renovate

One of the boons and/or banes of hosting code on GitHub is access to Dependabot⁸. It’s a tool that monitors your dependencies, and automatically submits PRs when new versions are available. It also does security alerts, but they’re comically bad for Go at least⁹. I do like having dependencies automatically handled, especially with a cooldown, which is how I had Dependabot configured before moving.

To replace it, I’ve configured Renovate, an open source alternative. I’d seen it used a bunch before: both by projects on GitHub who prefer it over Dependabot, and by people using GitLab and other platforms where Dependabot isn’t. It mostly does the same things, but Renovate has a few very nice extras.

Firstly, it supports transcluding config from another repo. So in each of my many, many projects, I just have this stub of a renovate config:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "local>meta/renovate"
  ]
}

Then the actual config lives in a default.json in the meta/renovate repository. I mirror that repository to GitHub, if you want to have a look. This is an amazing feature. When I enabled cooldowns in my Dependabot config it was a slog to go through all my active repositories and make the same change over and over again. With Renovate I can configure that centrally. As far as I know Dependabot can’t do anything like that, even for enterprises.

Within that central config I use some of the other nice features. It has the ability to auto merge changes. I use this to automatically accept changes from trusted projects, or my own libraries:

    {
      "matchManagers": ["gomod"],
      "matchPackagePatterns": [
        "^github\\.com\\/csmith\\/",
        "^chameth\\.com\\/"
      ],
      "matchUpdateTypes": ["minor", "patch"],
      "automerge": true,
      "minimumReleaseAge": "0 days"
    },
    {
      "matchManagers": ["gomod"],
      "matchPackagePatterns": [
        "^github\\.com\\/csmith\\/",
        "^chameth\\.com\\/"
      ],
      "matchUpdateTypes": ["major"],
      "minimumReleaseAge": "0 days"
    },

This takes any minor or patch release of my libraries, and automerges it with no cooldown. For major version changes it overrides the cooldown but doesn’t automerge. To automerge Dependabot PRs you have to write an action, implement that logic yourself, and hope you’ve successfully avoided all the footguns inherent in that.

Speaking of major version updates, Renovate also has an option to automatically update the imports in Go packages when it’s offering a major update. In the config I have:

"postUpdateOptions": ["gomodTidy", "gomodUpdateImportPaths"],

Which means that every time it does an update it’ll run go mod tidy, and it’ll update the import paths. In Go, v2 of a module has a different import path to v1 (so they can coexist if transient dependencies need different versions), so usually you have to go through and find and replace those paths. Renovate handles that for me now! I still have to address whatever breaking changes are in the new version, but there’s a lot less grunt work.

It can also run custom commands¹⁰, so for example in my legotapas project I have this extra snippet in the renovate.json:

  "postUpgradeTasks": {
    "commands": [
      "sh -c 'rm plate_*.go'",
      "go run ./cmd/generate"
    ],
    "executionMode": "branch",
    "fileFilters": ["*.go"]
  }

I won’t go into the gory details, but this handles automatically regenerating a bunch of files that change every time a certain dependency is updated. Previously I had a separate action to do that on GitHub; just being able to drop commands in the config file is much nicer.

To actually run Renovate, I simply have a workflow in Forgejo on a schedule that uses their official Docker image. It also triggers if I update the config, which is nice when I add a new rule, as it’ll immediately update the open PRs to comply. Overall, Renovate has felt like a pretty big quality-of-life upgrade.

Odd problems

Not everything has been smooth sailing. Sometimes when Renovate automerged a change, it would just… disappear. The PR was closed as merged, but the commit never landed on master. It wasn’t a huge issue as the next Renovate run would just recreate it, but it would then disable the automerge and I’d get confused as to why. It turns out there was a major problem with how I had everything set up. Renovate was telling Forgejo to automerge when the required checks pass, but I had no branch protection rules set up in Forgejo, so as soon as the PR was created it was eligible to merge. As the CI was so fast I hadn’t noticed that the automerges were happening before the CI results were posted.

The disappearing commits were because the repository was getting into a bad state when Renovate tried to automerge multiple changes instantly in the same repo. Forgejo has a bunch of maintenance commands that helped to fix that problem, and I then wrote a little script to enable branch protection rules so the CI would actually run. The automerging has been smooth since.

Another issue I had early on was that merging would fail because the GPG keyring was locked. I had Forgejo configured to sign all the automatic commits, so you can verify what commits came from my infrastructure. Forgejo is the only process using the keyring, so I wasn’t sure why it was becoming locked. It turns out that GnuPG uses the hostname as part of the lockfile, and because Forgejo was running in Docker it was getting random hostnames based on the container ID. Explicitly setting a hostname made GnuPG and by extension Forgejo much happier.

Webhooks and updating

Forgejo has this amazing feature: you can add a webhook that’s called for all your repos. GitHub, for some reason, supports this for organisations but not people. Even though the two are treated the same in a lot of places. It doesn’t seem like it would be hard, and it’s such an obvious thing to want, but… no. I wrote a whole tool to add webhooks for me to get around this. Now I don’t need it, Forgejo does the right thing!

One particularly interesting use of these webhooks has been making my Docker containers auto-update when a new image is pushed. I previously had a webhook receiver for a few projects that would update them automatically, but GitHub package notifications were extremely laggy. Every other webhook fired within a minute or so of the event happening, but package webhooks would sometimes take 30-60+ minutes to fire. I opened a support case with GitHub and it got escalated to engineering and fixed about 10 weeks later¹¹. It was better for a while, and then got worse again. Trying to do continuous deployment with a 30 minute random lag was frustrating.

So now I had reliable webhooks, but the update process was a bit gross. I have a hangup about internet-facing services not having access to the Docker socket. It just feels like too big a security risk. So I had a process that received hooks, and then another that figured out if containers needed to be updated and restarted them. Adding a new container to the system meant recompiling the second process, which was a massive faff.

Fortunately, I was saved from this mess by my friend Greg. He wrote adze, a tool that receives webhooks and uses Docker Compose to update matching containers. He wrote a blog post about it. It’s all one process, but that’s no longer a blocker for me because the webhooks can be delivered privately from my Forgejo instance without exposing adze to the internet. It finds matching containers automatically based on the images they’re using¹², and combined with Forgejo not forcing me to add a webhook to every single repository makes my life so much easier. I write some code, push it to Forgejo, and within a minute or so it’s running on my dev instance.

I have IRC notifications set up for when adze does updates, and I still have some old ones that announce GitHub package webhooks. I keep seeing things like this and chuckling:

15:44:42 <@ircjag> [ADZE] chameth.com: pending updating git.yak-wall.ts.net/public/chameth.com
15:44:45 <@ircjag> [ADZE] chameth.com: success updating git.yak-wall.ts.net/public/chameth.com
17:11:54 <@ircjag> [GHCR] Container pushed to csmith/chameth.com:dev.

I pushed a change at 15:43, Forgejo built it and pushed the image (to both its own registry and to GitHub), then at 15:44 adze updated the container to the new version. An hour and a half later GitHub gets around to delivering its own webhook.

Mirroring and PRs

The final piece I had to figure out was how to actually interact with people. Most of my projects are small and don’t get a massive amount of public contributions, but the option should still be there. And ideally not involve anyone having to figure out how to e-mail patches¹³.

Mirroring outwards is simple: Forgejo supports it natively. Whenever I push to Forgejo, it pushes out to the corresponding repo on GitHub. I’ve also started mirroring some projects to Codeberg, to avoid having GitHub as my single “public” presence. But what happens when I get a pull request? If I merge it in the GitHub web UI, Forgejo will push its own branch over the top and get rid of it¹⁴. I could pull the changes, merge them, and push the result to Forgejo but then I don’t get the benefit of any CI.

I ended up writing a small private tool that deals with this for me. I give it a GitHub PR, and it effectively imports it into Forgejo for me. I created a prs organisation, and it’ll create a fork of the project in that org if it doesn’t exist. Then it pushes the PR contents to a branch, and creates a PR from the prs repo into the main one. It runs as a user which can write to the PRs org, but only submit pull requests to public repositories outside of it.

When the PR is imported, I can review it in the Forgejo UI, and then I can approve the CI workflows for that particular PR. This gives me some extra defence-in-depth in case anyone is trying to abuse one of those action footguns. Once the CI passes, I can merge it as normal in the Forgejo UI, it gets mirrored out to GitHub, and GitHub marks the PR as merged automatically. It’s not the most elegant solution, but the actual workflow is pretty smooth, especially for the low volumes of PRs I get.

With that, I have Forgejo set up to do everything I need from a git host, and I’m no longer hamstrung every time GitHub goes down.

This way of counting isn’t particularly fair, but it’s somewhat realistic. GitHub break out their uptime for things like git operations, pull requests, actions, webhooks and packages. Realistically if any of those independent things are down then my overall workflow doesn’t, well, work. ↩︎
To be clear, I’m not objecting to the use of Copilot here. It’s a tool. But if you use a tool to do something fairly simple and it spews out 35 commits, including at least 5 reverts, then either it’s a terrible tool or you’re using it very wrong. I didn’t hunt down this example, I came across it while trying to debug something and wasted a bunch of time trying to figure out what on earth was going on. ↩︎
You could self-host runners, which may help, except they’ve already threatened to charge you for the privilege of bringing your own runners. ↩︎
If people could stop finding RCEs in git services for a few days so I can stop redrafting this intro, that’d be great. ↩︎
I’m feeling pretty smug about this decision after reading about the security issues. ↩︎
There’s one holdout: my dockerfiles repo which is a very special snowflake with a very complex workflow and I’m still deciding how to handle it. ↩︎
Which is to be expected, given I’ve got a server dedicated to one person, and they’re trying to serve however-many million users at once. That doesn’t make it any less nice, though! ↩︎
I’d link to it but there’s nowhere really good to link to. It’s just been subsumed into the ‘security’ functions of GitHub, along with a bunch of copilot nonsense. ↩︎
The go project maintain a vulnerability database that includes details about which exact symbols are affected, and the govulncheck tool uses this to only report security issues in code that’s actually used. Dependabot doesn’t do that: it constantly alerts about code that’s not imported, so won’t ever be shipped. ↩︎
In the default config it doesn’t allow anything of the sort, of course. You can whitelist individual commands, but because this is a private server I can merrily whitelist .* and do whatever I like. ↩︎
The turnaround time isn’t particularly impressive, but the fact that they kept the support case open, and the (human!) support agent updated me several times is. I don’t have a lot of good things to say about GitHub, but I don’t think I’ve ever had a support experience that good from a company of a similar size. Especially as I wasn’t a paying customer at the time. ↩︎
Why didn’t I do that in the first place? It seems so obvious now… ↩︎
I really like the idea of an e-mail based workflow, but I absolutely hate actually doing it. ↩︎
This is what happened to me a lot when I was half using Gitea. ↩︎

An interesting Tailscale + Docker gotcha

2026-01-29T00:00:00Z

As I’ve written about before, I use Tailscale for a lot of things. I thought I had it set up in a reasonably secure manner, but I recently noticed a problem.

I use Tailscale’s ACLs to limit what each node can access, based on the tags I apply to it. So an app node can’t access anything via Tailscale, while an integration or server node can access things tagged with either app or integration. This is expressed pretty simply in the Tailscale ACL JSON:

{
	"tagOwners": {
		// Servers that can be SSH'd into
		"tag:server": [],
		// Applications that are exposed on tailscale but never connect out
		"tag:app": [],
		// Things which talk to other services over tailscale (connecting to apps etc)
		"tag:integration": [],
	},
	"grants": [
		{
			// Users can access everything
			"src": ["autogroup:member"],
			"dst": ["*"],
			"ip":  ["*"],
		},
		{
			// Servers and integrations can access integrations and apps
			"src": ["tag:server", "tag:integration"],
			"dst": ["tag:integration", "tag:app"],
			"ip":  ["*"],
		},
	],
}

Over the past week I’ve been setting up a private Forgejo instance behind Tailscale, complete with an actions runner that runs things using a docker-in-docker container. I didn’t want the runner knowing anything about Tailscale, so I had it configured to speak to Forgejo direct over HTTP (forgejo:3000) instead of using full Tailscale HTTPS URL that I use when accessing it (http://git.example-net.ts.net/)¹.

Everything was going fine, until I forgot to do that translation… and it worked. My Forgejo action runners could access anything on my tailnet. I run several things on Tailscale that just have authentication turned off, on the basis that only authorised devices can access them. Things like the admin interface for this website. I definitely didn’t intend for any workflow I run on my git server to have access to edit my website!

This wasn’t just limited to Forgejo, either. Any docker container I was running could access the tailnet. In hindsight it’s fairly obvious why: the host is running Tailscale, connected as a node tagged with my server tag. That creates a tailscale0 interface, and automagically sets up iptables rules to route Tailscale traffic over the interface. Docker also automagically sets up iptables rules to bridge traffic, and apparently these two sets of rules interact in such a way that traffic from Docker containers is allowed to route via the tailscale0 interface.

I say it’s fairly obvious in hindsight — there’s no reason why Docker would special case any particular host interface after all — but it still feels pretty surprising. Because both bits of software inject their own iptables rules, I never really had a good mental model for how they interact. The host Tailscale node was a completely separate building block to Docker. It would be a pain to use either of them if they didn’t do these rules, but it’s also one of the reasons I don’t really like “magical” things².

There are lots of ways to fix this, but none of them feel particularly great. You can configure both Tailscale and Docker to not automatically fiddle with iptables and handle the rules yourself, but I really hate dealing with iptables³. As a stopgap I did hold my nose and add some iptables rules to drop traffic to the tailscale0 interface if it originated from the IP ranges that Docker was configured to use:

iptables -I DOCKER-USER -s 192.168.0.0/16 -o tailscale0 -j DROP
iptables -I DOCKER-USER -s 172.17.0.0/12 -o tailscale0 -j DROP

The DOCKER-USER chain is a nice little escape hatch; it comes before the main auto-generated DOCKER chain, and Docker leaves the rules in it alone.

So I added these rules, and felt pretty good about myself, and then… everything started breaking in weird ways. After some debugging I realised the problem was DNS⁴. The server’s DNS resolver is 100.100.100.100, a special Tailscale address. This is so that it can resolve tailnet hostnames via MagicDNS (and so I don’t have to configure my custom DNS servers manually on each device; Tailscale does it for me). My new iptables rules inadvertently dropped all the DNS packets coming from docker containers. D’oh.

Obviously the solution here is to double down and add MORE iptables rules:

iptables -I DOCKER-USER -s 192.168.0.0/16 -d 100.100.100.100 -o tailscale0 -j ACCEPT
iptables -I DOCKER-USER -s 172.17.0.0/12 -d 100.100.100.100 -o tailscale0 -j ACCEPT

This got DNS working again, but it all feels a bit gross. Not least because to actually make these persist you need to use iptables-save, and then you also get all the rules that both Docker and Tailscale have inserted. I came across a script to filter out the Docker ones, but… yuck⁵.

I realised a cleaner idea might just be to lock down what server tagged nodes could access within Tailscale itself. That way I can avoid touching iptables at all. It’s handy to be able to curl services from the host when debugging, but it’s not really necessary. So I removed that access, and… stuff broke again. I’m using my Forgejo instance as a registry for some of the docker images I run, so the docker daemon needs to be able to reach it. I ended up making a new tag for infrastructure, which can be accessed from server devices. This does still allow all the docker containers to reach Forgejo, but I already have it set up with appropriate access controls and public/private repository splits. Forgejo is a service designed to run publicly, so this seems a reasonable trade-off for convenience. I used the tests feature of Tailscale’s ACL config to make sure I’d got the rules right:

{
	"tests": [
		{
			// Servers can only access infrastructure
			"src":   "tag:server",
			"proto": "tcp",
			"allow": [
				"tag:infrastructure:8080",
			],
			"deny": [
				"100.84.16.43:8080",
				"me@example.com:8080",
				"tag:server:8080",
				"tag:app:8080",
				"tag:integration:8080",
			],
		},
	],
}

The whole situation still feels a bit messy. If I ever get around to switching to nftables I might loop back and manually craft some rules for routing traffic, instead of leaving Tailscale and Docker to do their own thing.

I discovered later on that logging into the container registry at forgejo:3000 actually issued a redirect to https://git.example-net.ts.net/ so this was all basically for naught… ↩︎
I didn’t use Tailscale’s MagicDNS for a long time just because the word “magic” put me off. Only when I eventually got around to learning how it worked, and seeing that it wasn’t really that magical under the hood, did I change my mind. ↩︎
nftables seems far better in lots of ways, but I can’t really be bothered migrating. Maybe next time I reimage the server for whatever reason… ↩︎
It’s always DNS… ↩︎
Again, nftables would almost certainly help here. It actually has (gasp) configuration files. But again, I really didn’t want to spend the time migrating. ↩︎

Monthly Meanderings: December 2025

2026-01-01T00:00:00Z

For a while I’ve been idly thinking about a way to get smaller bits of content onto my website without it being too annoying for me, or too hard to consume. Things like interesting links, small project updates, and so on.

I didn’t immediately come up with anything I was happy with, so just sat on it. Ignoring the problem seems to work surprisingly well for things like this. I’d been meaning to redesign the about page for a while, but wasn’t sure what to do with all the interests I wanted to list. After ignoring the problem for a while I came across the interests directory and realised I could just bundle all those things into their own page.

This seems amazingly obvious in hindsight, but I think I needed a bit of a push to accept that a page just about my interests was… well… interesting. Anyway, as the end of December rolled around, I got a similar dose of inspiration: a flurry of monthly round-up posts from various blogs I follow. If I bundle up all the small bits and bobs, it’ll make something vaguely blog post shaped, and writing and consuming those is already a solved problem. Again, pretty obvious in hindsight, and again I think I just needed a nudge to shut up the little voice in my head saying “but those people all have interesting things to say… do you, really?”

So without further ado, welcome to the inaugural post in my new monthly meanderings series. If you have any feedback on the format or contents, please drop me a note with the contact form at the bottom!

Website updates

As already mentioned, I reworked the about page and split out a new interests page. The old about page had a few lists of interests show up in thought bubbles when you hovered over things, which was a cool effect on desktop, but barely worked at all on mobile. The new page gives me a lot more room for waffle, and I do like waffling.

Only a single new blog post this month: exposing game servers over Tailscale. Just a quick how-to I wrote up, having not found it documented usefully anywhere before. I have a bunch of ideas for posts I want to write, but none of them have really sparked joy quite yet, so I’m once again employing the “sit on them for a bit and maybe inspiration will strike” approach.

Other projects

I created a small collection of Docker related projects this month: containuum is a go library that helps with listing containers, and keeping that list up-to-date. On top of that I built centauri-docker-confd which generates a config for centauri (my reverse proxy) based on running Docker containers. I already had a way of doing that, but containuum is designed to be a bit more robust, and I made it so the config can be passed over a network connection instead of having to write it to disk and then signal the proxy to reload.

The next project has a bit of backstory: after reading I got hacked, my server started mining Monero this morning [warning: the article is interesting but a bit LLM-sloppy] I realised that I, too, was running a vulnerable version of Umami. Luckily, as far as I can tell, it wasn’t exploited. But it got me looking into automated vulnerability scanning. I don’t put much stock into it normally, as you get so many false alerts, but this is one case where it would have definitely helped. There didn’t seem to be any good solution for my “middle-of-the-road” approach to containers (lots of containers, but not needing Kubernetes), so I wrote purser. It takes every image used by a running container, runs them through Trivy, and generates a combined report. I’ll add some form of active notifications to it in the future.

Entertainment

This month I’ve been playing a lot of Hogwarts: Legacy, after picking it up for free on Epic Games. I grew up with the Harry Potter books coming out, and while I’m not a super-fan, it still has a special appeal to me. Obviously she-who-must-not-be-named is doing her best to out-evil Voldemort, but I think you can still enjoy the universe despite that. Especially when you don’t have to contribute any money that might go towards her.

Anyway, the game is surprisingly fun. They did a really good job of making it feel like an open world. You want to walk around from place to place to see everything, find all the various collectables, and so on. Compared to Far Cry or Cyberpunk the world is a lot smaller but that means it’s far more densely packed with stuff to do. I initially didn’t vibe with the way combat works: it’s pretty heavily reliant on dodging or shielding against incoming attacks, which reminded me a lot of Clair Obscur: Expedition 33. Fortunately, it’s a lot more forgiving in Hogwarts: Legacy and I got used to it after not too long.

It’s been a relatively sedate month for other forms of entertainment. I saw Fackham Hall and Wicked: For Good at the cinema. The former was a funny romp, vaguely reminiscent of Airplane or The Naked Gun, as you’d expect from something penned by Jimmy Carr. Wicked was… fine? I wasn’t overly enamoured with part one, so it’s not a surprise I didn’t vibe with the follow-up. If you smushed them into a single normal-length feature I think they would have made a really good film, but two films both over two hours long is a bit much for what actually happens.

Around the web

Wikipedia: List of citogenesis incidents

A list of incidents where (often false) information on Wikipedia has ended up becoming its own source. That led to Mark Dominus’ blog post titled Imaginary Albanian eggplant festivals… IN SPACE where he discovers that a mountain on Ceres got named after a made-up festival (and then got it fixed!)

Not relying on Wikipedia as a source was drummed into us at school, but it feels like this sort of thing is only going to get worse and worse now LLMs are on the scene. I assume we’re already at the point where LLM hallucinations are prevalent enough that they’ll find their way into the training data for future iterations.

Simon Collison’s website

I really like the design here. Both the homepage and how well every journal entry is formatted. It makes me feel bad for the many walls of text that adorn this site. The archived v4 design is also wonderful.

My Dinner with Skinner

A 43-minute-long feature film that is “just” the Steamed Hams gag from The Simpsons done in the style of My Dinner with Andre. There’s a (much shorter) behind-the-scenes video that goes over the extreme amount of effort that was put into it, too.

I’m still trying to wrap my head around this. It’s… something.

Delete Spotify? Sure, But Don’t Just Replace it With Another Subscription

A similar take to my own escaping Spotify the hard way, but it avoids getting lost in the technical weeds like I did. (In my defence, they’re very fun weeds to get lost in.) Stephanie is also one of the people that does monthly wrap-ups, and I was pleasantly surprised when the latest one popped up in my RSS reader and I saw my own site linked.