Chameth.com - posts like 10-weeks-with-an-apple-watch-10, docker-automatic-nginx-proxy, simple-backups-restic-hetzner, why-you-should-be-using-https

Monthly Meanderings: July 2026

2026-08-01T00:00:00Z

Another month, another roundup post. This month has been very… exercise-y. Which is, for the record, not normal for me. For the previous year my only source of exercise has basically been walking, but that’s not a great fitness activity for me. When my regular walks entered the 10-15km territory it got more and more annoying to find the time and/or an interesting route.

I also managed to injure my knee while walking through a particularly muddy field, and spent a few months going slightly stir crazy. Coming out of that I started running three times a week, bought a bicycle, and added in some strength exercises along with the ones the physio recommended. If you told me a few months ago that I’d be annoyed at not being able to exercise too much on rest days I’d think you were mad.

Website updates and other projects

Before we talk even more about exercise, let’s do the usual topics I cover in these posts. The only website update this month has been a post about… exercise. Maybe I have a problem. Anyway, in Tracking PBs without the dark patterns I talk about building a slimmed-down, single-player Strava to track my personal bests, without all the nonsense that annoys me.

Other than the PB tracker, I’ve not been working much on personal projects. I’ve not really had any other itches that need scratching, just the usual background noise of dependency updates and other project maintenance.

Entertainment

Behold, the section where I normally talk about the films, TV programmes, and board games that have kept me entertained this month: […]. Yep, not a single one.

I have played a few games, though. A friend and I finally got around to trying out Escape Simulator 2, and it’s just as good as the original. The rooms feel a bit more detailed, and there are a few cool interactions that probably wouldn’t be possible in the old version. There’s also a hard mode for all the rooms which we’ve not yet tried, but sounds like it’ll keep us amused for a bit longer.

I’ve also been playing a bit of World of Warcraft, but it’s now in the awkward stage where I’ve done most things I can reasonably do this season, and there’s not really any point in doing anything else. All the gear will become mostly irrelevant in a few weeks when season two releases, and we’ve hit the major milestone achievements for Mythic+. Feels a bit anticlimactic to get to a fairly good stage and then just… not play for a bit.

Fitness

Given how much exercise I’ve done, it probably deserves its own section in these monthly posts. I have the vague idea of aiming to run a marathon before I turn 40 (i.e., within around 15 months). That’s both scary and yet not completely impossible. I’m going to work up through the distances and see how it goes. A 5k at the start of September, 10k in November, half marathon around Easter next year, then a full marathon just before my birthday in October. That way even if it gets too tough or something goes wrong I’ll have accomplished something.

New hobbies mean new toys. My bike is nothing particularly special: a hybrid from Halfords that’s about one step up from entry-level. But it gets the job done, and has helped me remember how much I enjoy cycling. The infrastructure around here is pretty terrible, though. There are lots of little bits of multi-use paths and a few dedicated bike lanes, but they’re not joined up or signed well in the slightest. In most cases it’s easier to just stay on the road and deal with the traffic. It’s a bit sad that they’ve spent time and money to do something and it’s not very useful.

On the running side, I picked up a Garmin Forerunner 165 watch. I’ve been wearing an Apple Watch for a little over a year now, but using the touchscreen to navigate screens and start/stop activities while running is annoyingly fiddly. The digital crown keeps getting gummed up with sunscreen and requiring cleaning, too. The Forerunner has physical buttons, and is much better at showing the data I want to see in a clear format, and letting me set things up like target pace or distance on the watch itself, instead of via a phone app. Oh, and the battery seems like it’s going to last for a week rather than a day, which is a nice change of pace.

I’m going to need to make a nice widget to show off my monthly runs and bike rides, but for now here’s some artisanal stats:

I went out for 14 runs, totalling 48km, although a significant portion of that was walking for warmup, cooldown or recovery.
My longest actual run was a 20 minute straight run covering 2.5km.
My average running pace is 8min/km, which is a “conversational” pace for me right now.
I went for 9 cycle rides: 2 on rented mountain bikes, then the rest on my own bike. They covered 128km.
When cycling I made 8 coffee shop stops, which works out to exactly one every 16km on average.

Next month I’m hoping to ramp the running up to hit 5k by the very end. I’m mostly following the NHS Couch to 5k plan but I’m tweaking it a little bit as otherwise I’d finish on 3.75km at my current pace. I guess “Couch to 30 minutes of running” isn’t quite as catchy. I made a Couch to 5k page to track my progress and adjustments.

Around the web

Blogging Can Just Be Stating The Obvious

I often look at my own posts and think, “There’s nothing novel, or important, or deep in here at all — is this even worth saying?”

I’ve written before about The Curse of Knowledge and Blogging, and this is a very similar take from Jim Nielsen, but it’s still super interesting to see a similar idea expressed in another way by another person.

I Built a Voice Opening Door to Moria (My Garage)

A YouTube video documenting a really cool project. What grabs me is the amount of disciplines he dabbles in: he’s part blacksmith, part carpenter, part electrician, part programmer, and so on. It’s one of the best examples of maker culture I’ve seen in ages.

There’s one bit in particular that sticks in my mind: he tried to find a rendering of the door online, and realised all the existing ones were gibberish or illegible, so he downloaded a font and worked out the exact inscription himself. It makes my inner perfectionist feel validated.

Linus Torvalds on AI in the Linux kernel

LLMs are very divisive, and there are a lot of unsettled moral and ethical issues. But at the end of the day they’re a tool that’s not going to go away any time soon. There are people on crusades to try and ban LLM tools in projects, boycott any projects that accept contributions, and so on. It’s a bit like militant veganism. I can understand where they’re coming from, but also please stop shouting at me. Anyway, this take from Linus seems pretty spot on to me.

Buttons

An interactive, graphical history of how buttons have been rendered in various operating systems and desktop environments over time. Maybe it’s just my age and nostalgia, but it really does feel like they peaked around the late 90s/early 2000s. I’m not quite sure why we went from elegant skeuomorphic buttons to ugly plain squares with weird accent colours when you click them.

Rotating sandwiches

For some reason I expected this to be static pictures of sandwiches which spun in a circle, but it’s actually recordings of sandwiches rotating on a turntable. It’s oddly mesmerising. I wonder why I’ve never seen a restaurant menu with gifs like that?

Tracking PBs without the dark patterns

2026-07-23T00:00:00Z

I’ve recently been bitten by some kind of strange exercise bug. I’ve gone from being mostly sedentary to struggling to not go out for a cheeky bike ride on a day that’s meant to be a rest day. Maybe this is what a midlife crisis feels like?

Anyway, one of the things I’ve enjoyed about this development is seeing my personal best efforts gradually go up. There’s something incredibly rewarding about doing a run, and then seeing a nice little gold medal because as part of that you ran your fastest 1km ever. Unfortunately the only software I’ve found that does that is Strava, and boy do I not get on with it.

The many shortcomings of Strava

Strava is basically the app for running and cycling. Everyone talks about it. It sounds like they like it. But I really, really don’t. It’s like someone went through a catalogue of user-hostile behaviours and said “we’ll take them all!”.

The problems started in the setup wizard: it touts a load of privacy features, but doesn’t offer any way to enable them. All your activities are public by default, and you have to spelunk around in the settings to turn that off. I wasn’t actually sure if I wanted my activities to be public or not, and ended up flip-flopping around a bit. There is a clunky workflow to bulk change past activities, but it’s arbitrarily time-gated so you can’t use it too often…

The bulk operations are extremely limited, too: you can set the activity visibility, and your heart rate visibility, and that’s it. Want to bulk delete things? You’re out of luck. Bulk rename? No chance. You either do them one-by-one or use some janky third-party script. I think I pressed around 300 little ‘delete’ links in the course of trying to get my activity history imported in a way I liked.

Those problems give a bit of a signal about Strava’s priorities, but I could live with them. Once I’ve stopped faffing with the initial import I’m not really going to want to change the privacy settings or bulk edit things. Alas, the problems didn’t end there.

Look at me! Look at me! GIVE ME ENGAGEMENT!

The most obvious one is that everything is an upsell. Vast swathes of the app are premium-only, and they don’t ever get out of your way: they stick around acting as adverts. We’re all used to freemium apps that try to hawk upgrades, but the sheer amount of screen real estate taken up by these is egregious. I started a premium trial, expecting things to get a lot better, and… they didn’t really. All the premium features are still branded as such, so now instead of “look at what you could have if you paid us” it’s “look at what you’re getting for paying us (don’t even think about cancelling)”. Subscribing also makes your avatar get a premium outline, and there’s no way to disable it. I really hate that. Now I’m an advert myself. I don’t want to be part of their weird dark patterns, thank you very much.

And while we’re talking about dark patterns: there’s so much engagement bait that I physically cringe at it. You get a notification for logging an activity with a vague call to action to check it out. You get a separate notification when you break or come close to one of your PBs, but it doesn’t tell you which; there’s plenty of space in the notification for it, but they want you to hit the app instead. It all just feels so slimy to me. I don’t know whether it doesn’t bother other people as much, or if they’re desensitised to it, or just don’t realise how pleasant things could actually be.

Making a replacement

I realised pretty early on that if I only want automatic PB tracking, I could just write something myself to do it. I already use Health Auto Export to exfiltrate data from Apple Health, including workouts. I can just make a service that receives that data, crunches some numbers, and activates the dopamine receivers in my brain when I set a new record. But I didn’t really want to bother doing that if I could avoid it. I tried a whole slew of different apps and none of them did the thing, at least not without a subscription. And Strava had soured me on fitness subscriptions. I finally resigned myself to making something.

And by “myself” I mean “an LLM”. This is one of the first projects where I’ve not written a single line of code, and I’ve barely even reviewed anything produced by the LLM. I gave it an overall architecture and some sample data, and had it write a plan. I critiqued the plan, had it broken down into a few different phases, and then had it work through them. I occasionally stopped it when I saw it doing something stupid, and gave it a couple of technical steers, but my feedback has mostly been functional. I’m probably not going to be releasing it publicly any time soon for that reason — I don’t want to publish it under my name when I’ve not even looked at the code. I also wouldn’t be comfortable making software this way if other people were going to rely on it, but for non-critical, personal-use-only software I’m happy with the approach.

Anyway, I called it “Pompei Band”. Project names are hard. One of my common techniques is to put a term into an anagram finder; in this case “PB dopamine” anagrams to “Pompei Band”, and it even abbreviates to PB. Weird, but I like it. This is how it shows recent activities on the homepage:

Dashboard entry for a recent run in PB

You can click through to see the standard activity details most tracking apps show: a map, heart rate, pace, gradient, and heart rate zones. You can also click on each effort to see a history of them, which gives a nice overview of progress:

1 mile run PB history

It pretty much does exactly what I want, and has none of the annoying behaviours that Strava leans into. When I first drafted this post I wrote “there’s a few more bits I’d like to add at some point, but I can happily prompt them into existence when I want them”, but in the time I spent procrastinating before taking screenshots I ended up implementing them all. My favourite is this little history widget I added to the homepage, shown here next to Strava’s:

Recent activity calendar in Pompei Band and in Strava

The Strava one just confuses me. I’m not sure what the scale of the circles means. I’m guessing it’s time, which in effect means it just highlights my bike rides which are longer than runs. The colours for the line charts don’t work for me, and I’m not sure what three zig-zag lines are meant to represent. Pompei Band’s version uses distinct, obvious colours for different activities, and scales the brightness independently for each type of activity based on length (so long cycle rides don’t affect the short runs). I also flipped the axes so it looks more like a GitHub activity timeline, because that’s how my brain works. Nothing groundbreaking, but also none of the paper-cuts the Strava version inflicted on me.

Software doesn’t have to be awful

This blog post has been rattling around in my head for a few days. I didn’t want it to just be a litany about Strava being bad, and I didn’t want to sound like an LLM hype piece, which made it tricky to figure out what I was actually trying to say. I think the core of it is: software doesn’t have to be awful.

Everyone’s heard of enshittification, and it’s so prevalent that it’s easy to accept it as a fact of life. That’s just how late-stage capitalism works, right? But we can just make better software. And as LLMs slowly become more and more capable, that “we” grows to a broader and broader group. It’s easy to be negative about that: we’re going to drown in vibe-coded products, much like we’re already drowning in written LLM slop. On the other hand, a future where you can just say “make me single-player Strava without all the dark patterns” and get something that works just how you want it is pretty exciting.

The one thing I can’t decide: if everyone can whip up their own individual software, will the existing platforms actually improve? If Strava didn’t have all the dark patterns and engagement bait I would have just have paid for it. Will they realise that? Or will they double down even harder, try to lock you into their platform, and build moats around user data? Now I’ve said it, it seems pretty obvious they’ll try to do that. But I hope they won’t.

Migrating from GitHub to Forgejo

2026-04-30T00:00:00Z

When Microsoft bought GitHub in 2018 my kneejerk reaction — like so many others — was to start looking for alternatives. For a while I self hosted a Gitea instance but I never totally bought into it: some repositories I still pushed to GitHub, some I pushed to Gitea and they got mirrored, and I ended up causing myself problems when I got the two confused. Part of the problem was that the GitHub UI was faster and cleaner than Gitea’s at the time; using Gitea felt like a chore compared to GitHub. I ended up not maintaining it and eventually binning it and just going back to GitHub.

An all-too familiar unicorn

Fast forward eight years, and GitHub is about what we all imagined when Microsoft bought it. If you take the most pessimistic way of counting, they have zero nines of reliability¹. If you take the most generous, they have a single nine. That’s around 30 minutes of downtime every day. It feels like it must be more than that, given how many times you see the damned unicorn.

Uptime aside, they’ve crowbarred LLMs in all over the place; they don’t have the time to maintain official GitHub actions; the ones they do maintain have Copilot running roughshod all over them²; actions themselves are slow to run³, often have weird transient failures, and there are so many footguns that I’m pretty sure GitHub actions should be an entire category in the CWE top 10.

Speaking of security, while I was drafting this post a remote code execution was reported and the details are… impressive. The actual security issue was a pretty stupid oversight. Those happen. But they end up being able to execute code as a globally shared git user with access to all the other repositories on the node. What? I can’t quite get my head around it. No sandboxing, no containers, it just runs as a git user?

So, yeah, I’m not a fan of GitHub in its current state. A few months back I took the plunge and set up Forgejo, a fork of Gitea where they did radical things like add tests and ensure that it’s community operated not twisted for commercial use. Forgejo also potentially has some fun security problems⁴, but the way I host it mitigates more-or-less all problems. It’s also slightly less shocking when an open source project has lapses in basic security versus a $2 trillion corporation.

It’s taken a bit of doing, but I’m very happy with the setup I have.

The basic setup

I run all my server software using Docker, and Forgejo is no exception. They have decent documentation on how to get it running, and publish rootless images which is nice (and diminishes my urge to make an artisanal version myself). You can set configuration options using environment variables, which is perfect for containers, although the names can end up a bit unwieldy. Like this thing: FORGEJO__repository.signing__SIGNING_NAME=Chris Smith. It gets the job done, but the environment-to-ini mapping is pretty ugly.

I didn’t want to expose Forgejo publicly, as I wanted to avoid having to try and secure it⁵, or dealing with bots scraping it, or users signing up, and so on. I wanted somewhere to store my git repositories, handle CI for me, and mirror them somewhere public for other people to see and interact with. If you’ve read some of my other posts you’ll probably guess where this is heading: Tailscale. Alongside the Forgejo container, I run a Tailscale instance with a serve.json that covers both the web frontend and the SSH listener used for git operations:

{
    "TCP": {
        "22": {
            "TCPForward": "forgejo:2222"
        },
        "443": {
            "HTTPS": true
        }
    },
    "Web": {
        "${TS_CERT_DOMAIN}:443": {
            "Handlers": {
                "/": {
                    "Proxy": "http://forgejo:3000"
                }
            }
        }
    },
    "AllowFunnel": {
        "${TS_CERT_DOMAIN}:443": false
    }
}

This allows me to interact normally with Forgejo from any of my devices or servers (which all run Tailscale), while not exposing it to the Internet at all. It also handles the TLS certificates automatically, which saves me setting up something else to do that.

For CI, I run the Forgejo runner image on the same server, and gave it a dind container to use to actually run the workloads. dind is the cutesy name for Docker-in-Docker, basically a Docker daemon running inside a Docker container. That keeps the CI workload separated nicely from the “production” workload running on the server, which is nice from both a security and a monitoring point of view.

The uncanny valley of actions

Forgejo actions are basically like GitHub actions: they run a series of steps using one or more container images. You define those steps in the exact same clunky YAML format as with GitHub. The big difference between them is that GitHub defaults to using an absolutely massive base image filled with out-of-date preinstalled software, and Forgejo leaves that for the administrator to configure. You could, of course, just use the same image as GitHub, but lugging around a multi-gigabyte image that doesn’t even have recent versions of the software I want isn’t really my style. Instead, I set about using a plain debian image as the base, and then did a face-palm when I tried to run a “normal” action and remembered they’re all written in JavaScript, so expect node to exist.

Accommodating JavaScript isn’t really my style, either, so I did the very sensible thing of writing my own suite of actions in Go. Being written in Go means they can be compiled statically, published as a container, and manage their own very limited dependencies. For example, the checkout action is built into an alpine container with git added, while the dockerbuild action uses the buildah image as a base. Each image has the tools it needs, pinned to a recent version, and nothing more. This is very much not a sensible approach to take for most people, but it made me happy. With six basic actions, I could replace almost all the ad-hoc workflows I had in GitHub⁶.

Other than the base images and my self-inflicted JavaScript machinations, everything works the same as GitHub. But faster. So much faster. Most of my CI workflows finish on Forgejo before GitHub would even have allocated a runner to the job⁷. And I can actually view the logs without them glitching out.

Step aside Dependabot, here comes Renovate

One of the boons and/or banes of hosting code on GitHub is access to Dependabot⁸. It’s a tool that monitors your dependencies, and automatically submits PRs when new versions are available. It also does security alerts, but they’re comically bad for Go at least⁹. I do like having dependencies automatically handled, especially with a cooldown, which is how I had Dependabot configured before moving.

To replace it, I’ve configured Renovate, an open source alternative. I’d seen it used a bunch before: both by projects on GitHub who prefer it over Dependabot, and by people using GitLab and other platforms where Dependabot isn’t. It mostly does the same things, but Renovate has a few very nice extras.

Firstly, it supports transcluding config from another repo. So in each of my many, many projects, I just have this stub of a renovate config:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "local>meta/renovate"
  ]
}

Then the actual config lives in a default.json in the meta/renovate repository. I mirror that repository to GitHub, if you want to have a look. This is an amazing feature. When I enabled cooldowns in my Dependabot config it was a slog to go through all my active repositories and make the same change over and over again. With Renovate I can configure that centrally. As far as I know Dependabot can’t do anything like that, even for enterprises.

Within that central config I use some of the other nice features. It has the ability to auto merge changes. I use this to automatically accept changes from trusted projects, or my own libraries:

    {
      "matchManagers": ["gomod"],
      "matchPackagePatterns": [
        "^github\\.com\\/csmith\\/",
        "^chameth\\.com\\/"
      ],
      "matchUpdateTypes": ["minor", "patch"],
      "automerge": true,
      "minimumReleaseAge": "0 days"
    },
    {
      "matchManagers": ["gomod"],
      "matchPackagePatterns": [
        "^github\\.com\\/csmith\\/",
        "^chameth\\.com\\/"
      ],
      "matchUpdateTypes": ["major"],
      "minimumReleaseAge": "0 days"
    },

This takes any minor or patch release of my libraries, and automerges it with no cooldown. For major version changes it overrides the cooldown but doesn’t automerge. To automerge Dependabot PRs you have to write an action, implement that logic yourself, and hope you’ve successfully avoided all the footguns inherent in that.

Speaking of major version updates, Renovate also has an option to automatically update the imports in Go packages when it’s offering a major update. In the config I have:

"postUpdateOptions": ["gomodTidy", "gomodUpdateImportPaths"],

Which means that every time it does an update it’ll run go mod tidy, and it’ll update the import paths. In Go, v2 of a module has a different import path to v1 (so they can coexist if transient dependencies need different versions), so usually you have to go through and find and replace those paths. Renovate handles that for me now! I still have to address whatever breaking changes are in the new version, but there’s a lot less grunt work.

It can also run custom commands¹⁰, so for example in my legotapas project I have this extra snippet in the renovate.json:

  "postUpgradeTasks": {
    "commands": [
      "sh -c 'rm plate_*.go'",
      "go run ./cmd/generate"
    ],
    "executionMode": "branch",
    "fileFilters": ["*.go"]
  }

I won’t go into the gory details, but this handles automatically regenerating a bunch of files that change every time a certain dependency is updated. Previously I had a separate action to do that on GitHub; just being able to drop commands in the config file is much nicer.

To actually run Renovate, I simply have a workflow in Forgejo on a schedule that uses their official Docker image. It also triggers if I update the config, which is nice when I add a new rule, as it’ll immediately update the open PRs to comply. Overall, Renovate has felt like a pretty big quality-of-life upgrade.

Odd problems

Not everything has been smooth sailing. Sometimes when Renovate automerged a change, it would just… disappear. The PR was closed as merged, but the commit never landed on master. It wasn’t a huge issue as the next Renovate run would just recreate it, but it would then disable the automerge and I’d get confused as to why. It turns out there was a major problem with how I had everything set up. Renovate was telling Forgejo to automerge when the required checks pass, but I had no branch protection rules set up in Forgejo, so as soon as the PR was created it was eligible to merge. As the CI was so fast I hadn’t noticed that the automerges were happening before the CI results were posted.

The disappearing commits were because the repository was getting into a bad state when Renovate tried to automerge multiple changes instantly in the same repo. Forgejo has a bunch of maintenance commands that helped to fix that problem, and I then wrote a little script to enable branch protection rules so the CI would actually run. The automerging has been smooth since.

Another issue I had early on was that merging would fail because the GPG keyring was locked. I had Forgejo configured to sign all the automatic commits, so you can verify what commits came from my infrastructure. Forgejo is the only process using the keyring, so I wasn’t sure why it was becoming locked. It turns out that GnuPG uses the hostname as part of the lockfile, and because Forgejo was running in Docker it was getting random hostnames based on the container ID. Explicitly setting a hostname made GnuPG and by extension Forgejo much happier.

Webhooks and updating

Forgejo has this amazing feature: you can add a webhook that’s called for all your repos. GitHub, for some reason, supports this for organisations but not people. Even though the two are treated the same in a lot of places. It doesn’t seem like it would be hard, and it’s such an obvious thing to want, but… no. I wrote a whole tool to add webhooks for me to get around this. Now I don’t need it, Forgejo does the right thing!

One particularly interesting use of these webhooks has been making my Docker containers auto-update when a new image is pushed. I previously had a webhook receiver for a few projects that would update them automatically, but GitHub package notifications were extremely laggy. Every other webhook fired within a minute or so of the event happening, but package webhooks would sometimes take 30-60+ minutes to fire. I opened a support case with GitHub and it got escalated to engineering and fixed about 10 weeks later¹¹. It was better for a while, and then got worse again. Trying to do continuous deployment with a 30 minute random lag was frustrating.

So now I had reliable webhooks, but the update process was a bit gross. I have a hangup about internet-facing services not having access to the Docker socket. It just feels like too big a security risk. So I had a process that received hooks, and then another that figured out if containers needed to be updated and restarted them. Adding a new container to the system meant recompiling the second process, which was a massive faff.

Fortunately, I was saved from this mess by my friend Greg. He wrote adze, a tool that receives webhooks and uses Docker Compose to update matching containers. He wrote a blog post about it. It’s all one process, but that’s no longer a blocker for me because the webhooks can be delivered privately from my Forgejo instance without exposing adze to the internet. It finds matching containers automatically based on the images they’re using¹², and combined with Forgejo not forcing me to add a webhook to every single repository makes my life so much easier. I write some code, push it to Forgejo, and within a minute or so it’s running on my dev instance.

I have IRC notifications set up for when adze does updates, and I still have some old ones that announce GitHub package webhooks. I keep seeing things like this and chuckling:

15:44:42 <@ircjag> [ADZE] chameth.com: pending updating git.yak-wall.ts.net/public/chameth.com
15:44:45 <@ircjag> [ADZE] chameth.com: success updating git.yak-wall.ts.net/public/chameth.com
17:11:54 <@ircjag> [GHCR] Container pushed to csmith/chameth.com:dev.

I pushed a change at 15:43, Forgejo built it and pushed the image (to both its own registry and to GitHub), then at 15:44 adze updated the container to the new version. An hour and a half later GitHub gets around to delivering its own webhook.

Mirroring and PRs

The final piece I had to figure out was how to actually interact with people. Most of my projects are small and don’t get a massive amount of public contributions, but the option should still be there. And ideally not involve anyone having to figure out how to e-mail patches¹³.

Mirroring outwards is simple: Forgejo supports it natively. Whenever I push to Forgejo, it pushes out to the corresponding repo on GitHub. I’ve also started mirroring some projects to Codeberg, to avoid having GitHub as my single “public” presence. But what happens when I get a pull request? If I merge it in the GitHub web UI, Forgejo will push its own branch over the top and get rid of it¹⁴. I could pull the changes, merge them, and push the result to Forgejo but then I don’t get the benefit of any CI.

I ended up writing a small private tool that deals with this for me. I give it a GitHub PR, and it effectively imports it into Forgejo for me. I created a prs organisation, and it’ll create a fork of the project in that org if it doesn’t exist. Then it pushes the PR contents to a branch, and creates a PR from the prs repo into the main one. It runs as a user which can write to the PRs org, but only submit pull requests to public repositories outside of it.

When the PR is imported, I can review it in the Forgejo UI, and then I can approve the CI workflows for that particular PR. This gives me some extra defence-in-depth in case anyone is trying to abuse one of those action footguns. Once the CI passes, I can merge it as normal in the Forgejo UI, it gets mirrored out to GitHub, and GitHub marks the PR as merged automatically. It’s not the most elegant solution, but the actual workflow is pretty smooth, especially for the low volumes of PRs I get.

With that, I have Forgejo set up to do everything I need from a git host, and I’m no longer hamstrung every time GitHub goes down.

This way of counting isn’t particularly fair, but it’s somewhat realistic. GitHub break out their uptime for things like git operations, pull requests, actions, webhooks and packages. Realistically if any of those independent things are down then my overall workflow doesn’t, well, work. ↩︎
To be clear, I’m not objecting to the use of Copilot here. It’s a tool. But if you use a tool to do something fairly simple and it spews out 35 commits, including at least 5 reverts, then either it’s a terrible tool or you’re using it very wrong. I didn’t hunt down this example, I came across it while trying to debug something and wasted a bunch of time trying to figure out what on earth was going on. ↩︎
You could self-host runners, which may help, except they’ve already threatened to charge you for the privilege of bringing your own runners. ↩︎
If people could stop finding RCEs in git services for a few days so I can stop redrafting this intro, that’d be great. ↩︎
I’m feeling pretty smug about this decision after reading about the security issues. ↩︎
There’s one holdout: my dockerfiles repo which is a very special snowflake with a very complex workflow and I’m still deciding how to handle it. ↩︎
Which is to be expected, given I’ve got a server dedicated to one person, and they’re trying to serve however-many million users at once. That doesn’t make it any less nice, though! ↩︎
I’d link to it but there’s nowhere really good to link to. It’s just been subsumed into the ‘security’ functions of GitHub, along with a bunch of copilot nonsense. ↩︎
The go project maintain a vulnerability database that includes details about which exact symbols are affected, and the govulncheck tool uses this to only report security issues in code that’s actually used. Dependabot doesn’t do that: it constantly alerts about code that’s not imported, so won’t ever be shipped. ↩︎
In the default config it doesn’t allow anything of the sort, of course. You can whitelist individual commands, but because this is a private server I can merrily whitelist .* and do whatever I like. ↩︎
The turnaround time isn’t particularly impressive, but the fact that they kept the support case open, and the (human!) support agent updated me several times is. I don’t have a lot of good things to say about GitHub, but I don’t think I’ve ever had a support experience that good from a company of a similar size. Especially as I wasn’t a paying customer at the time. ↩︎
Why didn’t I do that in the first place? It seems so obvious now… ↩︎
I really like the idea of an e-mail based workflow, but I absolutely hate actually doing it. ↩︎
This is what happened to me a lot when I was half using Gitea. ↩︎

An interesting Tailscale + Docker gotcha

2026-01-29T00:00:00Z

As I’ve written about before, I use Tailscale for a lot of things. I thought I had it set up in a reasonably secure manner, but I recently noticed a problem.

I use Tailscale’s ACLs to limit what each node can access, based on the tags I apply to it. So an app node can’t access anything via Tailscale, while an integration or server node can access things tagged with either app or integration. This is expressed pretty simply in the Tailscale ACL JSON:

{
	"tagOwners": {
		// Servers that can be SSH'd into
		"tag:server": [],
		// Applications that are exposed on tailscale but never connect out
		"tag:app": [],
		// Things which talk to other services over tailscale (connecting to apps etc)
		"tag:integration": [],
	},
	"grants": [
		{
			// Users can access everything
			"src": ["autogroup:member"],
			"dst": ["*"],
			"ip":  ["*"],
		},
		{
			// Servers and integrations can access integrations and apps
			"src": ["tag:server", "tag:integration"],
			"dst": ["tag:integration", "tag:app"],
			"ip":  ["*"],
		},
	],
}

Over the past week I’ve been setting up a private Forgejo instance behind Tailscale, complete with an actions runner that runs things using a docker-in-docker container. I didn’t want the runner knowing anything about Tailscale, so I had it configured to speak to Forgejo direct over HTTP (forgejo:3000) instead of using full Tailscale HTTPS URL that I use when accessing it (http://git.example-net.ts.net/)¹.

Everything was going fine, until I forgot to do that translation… and it worked. My Forgejo action runners could access anything on my tailnet. I run several things on Tailscale that just have authentication turned off, on the basis that only authorised devices can access them. Things like the admin interface for this website. I definitely didn’t intend for any workflow I run on my git server to have access to edit my website!

This wasn’t just limited to Forgejo, either. Any docker container I was running could access the tailnet. In hindsight it’s fairly obvious why: the host is running Tailscale, connected as a node tagged with my server tag. That creates a tailscale0 interface, and automagically sets up iptables rules to route Tailscale traffic over the interface. Docker also automagically sets up iptables rules to bridge traffic, and apparently these two sets of rules interact in such a way that traffic from Docker containers is allowed to route via the tailscale0 interface.

I say it’s fairly obvious in hindsight — there’s no reason why Docker would special case any particular host interface after all — but it still feels pretty surprising. Because both bits of software inject their own iptables rules, I never really had a good mental model for how they interact. The host Tailscale node was a completely separate building block to Docker. It would be a pain to use either of them if they didn’t do these rules, but it’s also one of the reasons I don’t really like “magical” things².

There are lots of ways to fix this, but none of them feel particularly great. You can configure both Tailscale and Docker to not automatically fiddle with iptables and handle the rules yourself, but I really hate dealing with iptables³. As a stopgap I did hold my nose and add some iptables rules to drop traffic to the tailscale0 interface if it originated from the IP ranges that Docker was configured to use:

iptables -I DOCKER-USER -s 192.168.0.0/16 -o tailscale0 -j DROP
iptables -I DOCKER-USER -s 172.17.0.0/12 -o tailscale0 -j DROP

The DOCKER-USER chain is a nice little escape hatch; it comes before the main auto-generated DOCKER chain, and Docker leaves the rules in it alone.

So I added these rules, and felt pretty good about myself, and then… everything started breaking in weird ways. After some debugging I realised the problem was DNS⁴. The server’s DNS resolver is 100.100.100.100, a special Tailscale address. This is so that it can resolve tailnet hostnames via MagicDNS (and so I don’t have to configure my custom DNS servers manually on each device; Tailscale does it for me). My new iptables rules inadvertently dropped all the DNS packets coming from docker containers. D’oh.

Obviously the solution here is to double down and add MORE iptables rules:

iptables -I DOCKER-USER -s 192.168.0.0/16 -d 100.100.100.100 -o tailscale0 -j ACCEPT
iptables -I DOCKER-USER -s 172.17.0.0/12 -d 100.100.100.100 -o tailscale0 -j ACCEPT

This got DNS working again, but it all feels a bit gross. Not least because to actually make these persist you need to use iptables-save, and then you also get all the rules that both Docker and Tailscale have inserted. I came across a script to filter out the Docker ones, but… yuck⁵.

I realised a cleaner idea might just be to lock down what server tagged nodes could access within Tailscale itself. That way I can avoid touching iptables at all. It’s handy to be able to curl services from the host when debugging, but it’s not really necessary. So I removed that access, and… stuff broke again. I’m using my Forgejo instance as a registry for some of the docker images I run, so the docker daemon needs to be able to reach it. I ended up making a new tag for infrastructure, which can be accessed from server devices. This does still allow all the docker containers to reach Forgejo, but I already have it set up with appropriate access controls and public/private repository splits. Forgejo is a service designed to run publicly, so this seems a reasonable trade-off for convenience. I used the tests feature of Tailscale’s ACL config to make sure I’d got the rules right:

{
	"tests": [
		{
			// Servers can only access infrastructure
			"src":   "tag:server",
			"proto": "tcp",
			"allow": [
				"tag:infrastructure:8080",
			],
			"deny": [
				"100.84.16.43:8080",
				"me@example.com:8080",
				"tag:server:8080",
				"tag:app:8080",
				"tag:integration:8080",
			],
		},
	],
}

The whole situation still feels a bit messy. If I ever get around to switching to nftables I might loop back and manually craft some rules for routing traffic, instead of leaving Tailscale and Docker to do their own thing.

I discovered later on that logging into the container registry at forgejo:3000 actually issued a redirect to https://git.example-net.ts.net/ so this was all basically for naught… ↩︎
I didn’t use Tailscale’s MagicDNS for a long time just because the word “magic” put me off. Only when I eventually got around to learning how it worked, and seeing that it wasn’t really that magical under the hood, did I change my mind. ↩︎
nftables seems far better in lots of ways, but I can’t really be bothered migrating. Maybe next time I reimage the server for whatever reason… ↩︎
It’s always DNS… ↩︎
Again, nftables would almost certainly help here. It actually has (gasp) configuration files. But again, I really didn’t want to spend the time migrating. ↩︎

Securing all the things with 1Password

2025-11-26T00:00:00Z

For many years I’ve been a keen user of Bitwarden. Recently I’ve had a lot of small paper-cut problems. The browser extension was redesigned and just doesn’t quite work how I expect any more. The prompt to save new login info misfired more than it worked. The mobile app stopped background refreshing properly. No one issue was enough to make me want to leave Bitwarden, but it definitely wasn’t sparking joy. Then there were rumblings about a price increase. I don’t think they’ve announced anything, but the numbers I saw were a jump from $10/year to $24/year. A nearly 2.5x price increase for a tool that was actively getting worse over time did not seem like a good deal to me¹.

I spent a while looking at alternatives, but there were very few that met my needs. Nordpass is highly praised but doesn’t support saving TOTP codes on personal plans². All the nice open-source/self-hosted solutions fall down when you want reliable synchronisation to an iOS device (which is fair enough). Proton Pass looked like it had all the features, but the pricing was asinine. Different pages showed different prices, the prices were introductory rates that jumped up on renewal, but that was hidden away in small print, etc. Yuck. The prices are also pretty high. More or less the same as 1Password, which I’d been discounting due to the cost, until now…

After eliminating basically every other option, I took a serious look at 1Password. It seemed to tick all the right boxes, I’d often heard good things about it, and it wouldn’t actually be much more expensive than Bitwarden if they went ahead with the rumoured price increase. I started a trial to give it a test.

Initial impressions

1Password is — as you’d expect for something that originated as a Mac app — very pretty. But it’s also highly functional. The looks don’t get in the way, they’re nice extras on top. When it’s locked, you get a large version of the 1Password logo, which is a front-on view of the barrel of a lock, next to the password input. When you authenticate, the barrel turns as though you’d put a key in and rotated it, and then the background splits in two and opens like a vault door. Does a password manager need to do that? No. Does it make me a little bit happy every time I see it? Yes. Look at it, it’s amazing!

The design philosophy carries through the rest of the app. Despite using Bitwarden daily, I often clicked on the wrong thing when I wanted to do anything other than autofill; in about a month of using 1Password I don’t think I’ve ever misclicked or been confused about the UI. The browser extension just feels snappier than Bitwarden, too. I’m not sure if it’s actually faster or if the design just makes it feel that way.

Functionality-wise, 1Password does everything Bitwarden did for me but slightly better. Bitwarden kicked you to its web interface for some operations (like checking compromised passwords); 1Password has that all built into the app. Instead of editing entries in a fiddly little browser popup, 1Password just summons forth the desktop app and you edit it there. It does a good job of detecting QR codes for TOTPs and offering to save them. I think the only small complaint I have is that the browser extension doesn’t have an autofill keybind. You can rig that up via the desktop app, but it feels like 1Password leans towards you using the UI it adds on login forms. To be fair, that UI works really well, and is very useful when you have multiple accounts on a site. My muscle memory for autofilling is strong, though.

Diving deeper

At this point I’d decided to switch fully to 1Password, so I started digging into what else it could do. The “Watchtower” feature combines a bunch of reporting about passwords (weak, breached, reused, and so on), but also lists sites where 1Password knows you can use passkeys or 2FA and don’t have them set up. I’d previously disabled passkey support in Bitwarden because it just appeared one day and got in the way of me trying to use a YubiKey, but decided to give them a go with 1Password. “Passkeys” is basically just a marketing term for FIDO2’s WebAuthn with automatic discovery bolted on top. The big problem is how you synchronise or export them. It’s all well and good letting your iPhone create a passkey when the Amazon app arbitrarily prompts it to, but what happens when you log in on a different device, or swap to Android, or whatever? Having 1Password deal with them fixes that³.

The passkey flow is very smooth: 1Password pops up an account selector, with obvious options to create a new account or pass the request through to another device like a YubiKey. It’s smart enough to guess the right account most times, even when you’ve got 17 different Google accounts for $reasons. Using them is equally simple: it just shows a popup and you tap to confirm you want to use the passkey. The Watchtower functionality made it really easy to go through and add passkeys everywhere that supports them. The thing I really appreciate is that they make the data that powers it public. It’s obviously partially a marketing thing for them, but it’s also a useful resource they could have kept proprietary.

Passkeys are a bit basic, though. 1Password has many more fun offerings. There’s an entire developer section with various goodies. At first I thought “that’s nice but I won’t use it”, but now I’m using basically everything it offers. The first thing to draw me in was the CLI. It’s pretty straightforward, but like the browser extensions it can nicely integrate with the desktop app so you unlock them all at once, deal with auth prompts in the GUI, and so forth. I have a couple of command-line tools that need passwords (e.g. for dealing with encrypted backups); now instead of prompting for the passwords, they shell out to op read and I authorise it via a popup in the 1Password app. Simple, but a nice quality of life improvement.

Next up there’s a beta feature called “Environments”. This lets you define groups of environment variables within 1Password, and have them exported automatically to certain destinations. At the minute it supports exporting to the AWS SecretsManager, and local .env files. When making a local file it actually makes a named pipe, so the credentials aren’t just sitting on disk (and won’t end up accidentally committed to anything). When the 1Password vault is locked, anything trying to read from the file will cause an authentication popup, much like with the CLI tool. I used to have the mentality that if something could read arbitrary files on disk, it was basically “game over” from a security point of view. Recent events made me realise there’s some value in defence-in-depth there, though, and 1Password provides a nice solution.

Fingerprints + Linux = Sadness

One feature of 1Password had me feeling a bit left out. Instead of entering your vault password to unlock it every time, you can have it invoke the system authentication library. On an iPhone this uses Face ID, on a Mac it can invoke Touch ID, and on Linux it uses Polkit. In the default configuration, Polkit just asks for your local user password, which is not much of an improvement. A fingerprint scanner seemed like the obvious solution here: it’s both a lot more convenient than typing a long password, and provides some assurance that it’s actually me doing the action⁴. Unfortunately, fingerprint readers on Linux seem to be a bit of a sorry affair.

Don’t get me wrong: there’s support. libfprint has a big list of supported devices, but I found it almost impossible to actually find a device I could buy that had one of the listed sensors. Most of them seem to be exclusively built into laptops. A lot of the cheaper devices just use whatever sensor they can get their hands on, so it’s pot luck if you receive a version that will work with libfprint or one that won’t. There’s also not a lot of information beyond that published by libfprint. I guess it makes sense: external fingerprint readers just aren’t a thing most people care about. While that makes me feel all special, it’s somewhat prohibitive to getting an actual working solution.

I’d given up on finding an answer to this when I stumbled upon an article by Scott Laird. He uses a YubiKey Bio to do it, via pam_u2f. The Bio isn’t actually a fingerprint reader: it just has an on-board reader to secure the credentials. The OS doesn’t get involved in the process. But because it can provide a fingerprint-secured U2F key, and pam_u2f can require that key to perform system operations, you basically get the same result. Maybe it’s actually even better than using a “real” fingerprint reader, as the data is secured on the hardware device…

The YubiKey Bio was a bit more pricey than I really wanted, coming in at around £90, but it felt like a worthy investment to add a bit of security and avoid typing a password dozens of times a day. The setup was simple: the official app that you use to enrol fingerprints is packaged in the Arch User Repository, and it worked perfectly. I already had pam_u2f set up as I used a (non-Bio) YubiKey to gate access to sudo, so I just generated a new key on the Bio, updated my config to use that, and added it to the Polkit policy. Now when I need to unlock 1Password or authenticate something, it prompts me to scan my fingerprint. Perfect!

Biometric all the things \o/

One of the other things Scott mentioned in that article was SSH keys. I’d seen 1Password supports acting as an SSH agent, but didn’t really see why I’d want it to. I used a resident key on my old YubiKey, so I didn’t have much to gain from a security point of view. Scott’s article made me realise I could have some big convenience gains, though. Currently every time I need to use the key, I enter a PIN and then physically touch it. I don’t mind this for the most part, but it turns out git-lfs does something like four different requests every time you do a git operation, none of which reuse the same SSH connection for some reason. That gets painful when each request involves a PIN and a touch. I also liked the idea of using pam_rssh for sudo authentication, but didn’t want to have to do the PIN-and-touch dance every time I used it.

You can probably see where this is going: I generated an SSH key in 1Password, and then… spent literal hours disabling all the other SSH agents that were running on my system, and trying to figure out where SSH_AUTH_SOCK was being populated. I didn’t manage to figure that out in the end⁵, and ended up just overriding it to point at 1Password’s agent socket. I added the new key to GitHub, and did the usual ssh git@github.com test, and it worked perfectly. 1Password prompts to allow access to the key, showing the process requesting access, and authorising it requires a fingerprint read:

can haz ssh key?

I’m not sure why Kitty doesn’t get an icon in the prompt; other applications do. You can expand the details section to see the key fingerprint being requested, as well as the exact process name and PID requesting the key. With the proof of concept done, I generated a couple more keys: one for normal SSH, and one for gaining root access using pam_rssh. Using different keys ensures I get prompted for them separately, and means a malicious app couldn’t sneakily ask for a Git key but actually pivot to SSH into a server. Not sure that’s ever likely to be a problem, but it’s basically no additional work for a bit more defence-in-depth, so why not?

I also took a moment to configure pam_u2f for both lightdm and xfce4-screensaver, so I can now log in and unlock my PC with my fingerprint. It’s not quite as smooth as Touch ID on a Mac, but it’s pretty close.

As an aside: this is not the first time I’ve been in this situation. Both Spotify and Todoist have focused on features I don’t want or need (including, but not limited to, pretty much everything branded “AI”), and then hiked the price up afterwards. It’s not quite standard enshittification, but it’s clear that our incentives no longer align properly. ↩︎
Or didn’t, at the time I looked. It now says “All plans” on the website but the docs only exist for business accounts, and there doesn’t seem to be any announcement explaining the change. ↩︎
Having Bitwarden deal with them would also have fixed that, but it got in my way instead of helping me, so… ↩︎
or it’s not me, but I’ll be busy dealing with the presumably-undesired removal of one of my digits, so probably won’t care about whatever is happening on the computer. ↩︎
Although I’m pretty sure the answer starts and ends with “systemd, somehow”. ↩︎