Chameth.com - posts like docker-proxying-redux, securing-all-the-things-with-1password but not why-you-should-be-using-https

Building a family of single-user services

2026-08-27T00:00:00Z

Last month I made and wrote about a personal service to track my workouts and record personal bests. Since then, things have escalated. I now have eight different services running in the same “family”:

Name	Purpose
Collegiate Room	Simple weather overview for my current location
German Mall	LLM broker for use by other services
Gloss White	Weight tracking
Olive Crosses	Central service hub
Pompei Band	Workout and personal best tracking
Speed Carp	Spaced repetition system
Tiger Island	Reading history and library tracking
Typical Horse	Location history and analytics

The names are all anagrams of something related to their function. Finding those anagrams brings me an unreasonable amount of joy. Unscrambling them is an exercise left to the reader. On top of those eight, I currently have four other service ideas sat in my todo list. It’s going to be a big family.

A postmortem on the previous hodge-podge

Most of this functionality isn’t new to me. When I started out playing around with LLMs, I created a “personal data aggregator” that just kept getting things bolted on. I said at the time:

My biggest just-for-me project is a web app that I started to help me aggregate film recommendations. It’s since morphed into a general personal data aggregation service: it deals with data from GitHub, Todoist, Letterboxd, TMDB, Healthkit, and others. It also lets me make re-orderable lists, store recipes, and more. Parts of this could definitely be open sourced, and I might carve them out at some point, but it’s mostly a glorious hodge-podge of things specific to me. Having all these services in one place lets me make quick and dirty automations, for example: when I create a Todoist note on my phone or watch, I often forget to set the due date, so it doesn’t show up in the “Today” view. It was literally a few lines of code to plumb things together so any inbox task without a due date gets set to today automatically.

In a twist that nobody could see coming, it turns out that a “glorious hodge-podge” is actually quite hard to maintain and very annoying to work with. It didn’t help that it had a fairly extensive Svelte frontend and I really am not a fan of JavaScript. I ended up using it less and less just because it wasn’t fun to use or work on, and I didn’t want to waste the limited LLM tokens on doing massive refactors or reworks.

I’ve got almost two decades of professional software development experience under my belt, so you’d think I would have learnt the lessons on how to write maintainable software. Hell, my CV even mentions that as something I’m particularly good at. So what went wrong?

There were a few factors. One of the obvious things to blame is that it was written mostly with LLMs. Most of that blame isn’t very honest, though. In the year or so since, LLMs have definitely got better at writing good code; but I was manually reviewing all the code at the time, and telling it quite carefully what to implement and how to do it. So the buck really stops with me.

I think it’s actually more because it was my first larger project using LLMs. It’s a bit like learning a new programming language. I don’t think the first sizable project I’ve written in any language has ever been any good. You can read and learn as much as you want, but you need to actually run into some problems headfirst to get a grip on how the language works. Or at least I do. LLM coding agents are basically the same deal.

What the LLMs actually did was mask several problems. The architecture wasn’t right for how the service developed. It started out as a single-purpose app and gradually got things bolted on, without any major changes. Everything ended up in horizontal slices: a massive database layer with every query used by every part of the app, a massive HTTP layer, and so on. Working on a single feature became a tap dance all over the source tree. This sort of friction is normally quite apparent when you’re writing code by hand, and at some point it hurts enough that you throw in the towel and do some refactoring. But the LLM will happily plod on without exposing that pain, so the project needed a more active kind of architecture management that it just didn’t get.

The other problem they covered up was how brittle and unmaintainable the JavaScript frontend was. I’m not even sure how I ended up with a Svelte frontend in hindsight. It’s my preferred framework, but I still only reach for it as a last resort when there’s no other way to avoid using one. It seems like there were lots of ways to avoid it here, but I seem to have… well… avoided them. Anyway, each feature got its own slew of Svelte components, without much thought to reusability, shared patterns, or anything sensible. Some of them were just thin wrappers around data supplied by Go, others had vast swathes of logic in them, with no real rhyme or reason as to which way that would fall. It’s the same issue as before: a human would look at it, say “WTF?” and suggest tidying it up; instead, the LLM just gets on with what you tell it to do.

There’s also a big non-LLM factor: one of the primary reasons all of this unrelated functionality got tossed into the same app was that I’d previously worked on a few projects with a microservice architecture and really didn’t like it. My conclusion was they only really worked in large environments where you can dedicate people or teams to working on all the boilerplate that happens from maintaining lots of interconnected projects. Dumping everything and the kitchen sink into one badly architected monolith was, perhaps, not the best rebound from that.

Pompei Band and beyond

When I wanted to implement ~~Strava at home~~ my PB tracking app, it seemed fairly natural to build it out as a new service, rather than bolt it on to the slowly decomposing hodge-podge that was giving off a funny smell. Then when I wanted a weather dashboard, why not do it in the same style as Pompei Band? The same serving architecture, the same page layout… I could even come up with a similar weird two-word name!

Then wouldn’t it be nice if the weather dashboard knew where I was, so it could show the local weather? I could just build that functionality in, but it’d also be a good opportunity to carve out the location history feature from my old service and make a new, shinier one. Then basically every time I even thought about going near the old service, I mentally sketched out a replacement service.

Olive Crosses showing links to the other services

At one point I had the thought that it’d be nice if the services could all link to one another, but I didn’t want to have to touch every service whenever I added a new one. So I made a “hub” service, which maintains the One True Catalogue, and provides a bit of JavaScript the others can all embed to add a navigation sidebar.

So after complaining about microservices and JavaScript, it sounds like I have landed on microservices (or at least miniservices) and JavaScript? Sort of yes, but sort of no. Each service is still its own thing and makes sense independent of the others. There are some cross-service dependencies, but they’re very simple and don’t have the same cognitive or boilerplate overhead as a login service that talks to an authentication service that queries a user manager and so on. And as for the JavaScript, it’s almost exclusively used for progressive enhancement and a few limited client-side visual things like maps and graphs. All of the logic lives in Go, which is a lot easier to maintain and test, and all of the pages are simple Go templates.

Having these services built in a way that I actually like has motivated me to play around a lot more with new features:

My workout tracking app pulls photos from my private Immich server that were taken during workouts, and shows them alongside. Now I can see all my obligatory end-of-run selfies in one place.
The location history app can detect trips away from home, and get an LLM to automatically title them. My most recent, very exotic, trip has the title “A fortnight in Milton Keynes and Coventry”. Each trip gets its own little map, and in the future I want to pull in images like with workouts.
My weight tracking app now exports weigh-ins to Garmin, as it doesn’t fully integrate with Apple Health (it will export its own weights, but not import others, like the ones from my Withings scales). Stupid walled gardens, but thankfully I can fix it myself.
My book tracking app automatically cleans up the titles of books from the nonsense provided by Amazon to something more palatable. “Killing Eve: Long Shot: A twist on obsession, betrayal and revenge from Luke Jennings, author of the TV sensation Killing Eve” becomes “Long Shot (Killing Eve #5)”. It does this by passing the title to an LLM with a bunch of rules.
The hub generates me a snippet of Tailscale configuration that defines the minimal grants needed to allow the intra-service dependencies to work.

The LLM integrations were inspired by my friend Shane who recently demonstrated an inventory tracking app he’d made, which used Haiku to process images of boxes and describe the contents. I’ve dabbled with LLM integration before, and always disliked the process of fiddling with the prompts, and debugging when the LLM gave a nonsensical response. So I built German Mall, which makes it a lot easier. It lets me define a “workflow” consisting of a system prompt, initial message, and expected response format. I can browse past runs, tweak the prompt, rerun them to see the change, all without touching the services using them. Quick iteration loops are the best. German Mall also provides markdown documentation for each workflow, so integrating it downstream is simply a case of pointing a coding agent at the right documentation endpoint.

The “tidy book titles” workflow in German Mall

It’s not all fun and games: having a bunch of services all following the same pattern means you need to do cross-cutting maintenance on them now and then. But that’s surprisingly easy with LLMs. I put all the projects in a common folder, and I launch cross-cutting tasks from that folder. Usually I’ll make the change in one project, and then the cross-cutting task is simply “Look at commit XYZ in ./foo, make the same change to the other projects”.

I’ve also ended up with a shared library used by all the projects to cut down on some boilerplate. That’s normally something I’d resist, especially with personal projects, just because the extra overhead of working in a separate project, managing releases of it, and so on, is a bit annoying. But, again, LLMs can do almost all of that grunt work. They effectively reduce the “cost” of maintaining separate services to a negligible level, leaving me to focus on the more fun aspects like coming up with new names.

The Case of the Unchanging Config

2026-04-09T00:00:00Z

Last week I was attempting to make it so I could share pictures on IRC directly from my client. This sounds simple, but it involves a bouncer¹ that proxies the request to a standalone image hosting service that I had to modify to be compatible. At one point my testing loop was:

Make a change to the hosting service
Commit it
Tag a new release
Wait for it to build
Update the version I’m running on my server
Reconfigure the bouncer to pass the new parameter or change the URL or whatever
Try uploading a photo from my phone
Realise I’ve overlooked something and go back to step one

I could have set up a local copy of everything and tested it sensibly. I should have set up a local copy of everything and tested it sensibly. But it seemed like such a trivial change, and setting up the whole environment seemed like such a pain. After the third or so iteration of failure I was pretty annoyed with myself, computers, and basically everything.

My biggest annoyance was that my bouncer would not pick up the new URL from the config when I changed it. It’s meant to reload the config when it receives a SIGHUP, and it claimed to in the logs, but I could clearly see it was still hitting the old URL. Restarting the bouncer to update the config is a pain, as it disconnects me from all the IRC networks, and has to reconnect to them all, reauthenticate, etc. It also mildly spams everyone who shares a channel with me².

When I finally got everything working I had a look at the bouncer source, and thought I’d spotted the issue. I raised a bug report, ending in this remark:

It looks like the config is reloaded properly, but the handler for uploads is created once at startup and has its own copy of the uploader, so effectively snapshots the config to whatever it is at startup:

fileUploadHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    cfg := srv.Config()
    h := fileupload.Handler{
        Uploader:    cfg.FileUploader,
        DB:          db,
        Auth:        cfg.Auth,
        HTTPOrigins: cfg.HTTPOrigins,
    }
    h.ServeHTTP(w, r)
})

I’m not quite sure what I thought I saw there, but that’s not right. I want to call it a hallucination, but that feels like a very overloaded word these days. I guess I was just seeing what I wanted to see, and that was a good excuse to stop investigating. Still, even if I misidentified the cause, the bug was still present, right? … Right?

The Science of Deduction

When the developer got back and said he couldn’t reproduce my issue, I went back to my install and immediately reproduced it. With a calmer head, I figured it was probably something with my particular set up. First thing to check: am I actually running the version I think I am?

Unfortunately the bouncer doesn’t actually seem to expose the version anywhere that I can see. It’s not in the logs, it doesn’t have a -version flag, and none of the IRC-based status commands seem to include it. But I know it’s a Go app, and I know Go embeds the version information. go version -m <binary> will dump it all out, but the binary is inside a Docker image, and the Docker image is one of my nice, minimal, artisanal ones so doesn’t ship a go binary. No problem, docker compose cp bouncer:/bnc ./bnc yoinks the binary out of the container, and then dumping the version shows that, yes, I am running the version I thought I was. Hmm.

The next tool I reached for in my toolbox was strace. Maybe it’s not actually reading the file for some reason? I immediately executed strace -p <pid> -e trace=openat,open,read,pread64 -f without having to look any part of that up. Yep. Definitely. Then I edited the config, HUP’d the bouncer, and saw that it was… reading the config file. As it’s meant to. By default strace truncates strings to 32 bytes, so I couldn’t actually see the line I’d changed. Some more definitely-not-RTFMing later, and rerunning it with an extra -s 65536 let me see the full config. Surprise! The config hadn’t changed!

To confirm my findings, I used docker compose cp again, this time yoinking the config file from inside the container. The inside config file was definitely different to the outside config file. What? My hypothesis at this point was “something something Docker nonsense”. I mount the config as read-only, and was wondering if that meant that Docker was doing something weird instead of just bind mounting it. A quick trip to /proc/<pid>/mounts showed that it was, in fact, not doing anything weird, and was just bind mounting it.

If the file is bind mounted, then surely it’s the same file? I ran stat on the file on the host, noted the inode number, then pondered how to actually run stat on the file inside the container, given the aforementioned awkwardly minimal image. The solution was easy: access it via /proc/<pid>/root/. I could’ve saved myself a bunch of docker compose cp if I’d thought about that earlier. Oh well. The inode of that file was different. What?

The answer was DNS. It’s always DNS. Oh, sorry, force of habit. I meant the answer was vim. I was editing the config in vim, and when it saves files, by default, it writes the new content to a temporary file and does an atomic rename. That’s normally a good thing: it prevents corruption if the write fails midway through³. In this case, though, that means the new file has a new inode. Bind mounting a file binds to the inode, so the container just keeps clutching onto the original config from when it was started, blissfully unaware that the party has relocated down the street.

This problem is likely to happen whenever you bind mount a file into a container. When I mentioned this to a friend, he immediately responded “oh yeah, never do that”, and went on to describe the horrible hacks he’s had to add to Ansible to sidestep the issue. The nicer solution is to just bind mount an entire directory if you can, as then it doesn’t matter what happens to the files within it. I really like having the config files sat alongside the Docker compose files, though; having to create a directory just to work around some bind mount weirdness upsets me.

Now I knew what the problem was, I found there was an issue raised against Docker twelve years ago⁴. The first response was “that’s expected”, and I fully agree with the author: “respectfully, that might be expected by you, but it was not expected by me”!⁵ As a result of the issue they added a nice note to the docs:

Note: Many tools used to edit files including vi and sed --in-place may result in an inode change. Since Docker v1.1.0, this will produce an error such as “sed: cannot rename ./sedKdJ9Dy: Device or resource busy”. In the case where you want to edit the mounted file, it is often easiest to instead mount the parent directory.

But in the intervening twelve years, both the note and the functionality described have gone missing. Ho-hum.

I found it interesting how I’ve only just hit this problem, given how long I’ve used Docker. But I realised that almost everything I run I’m happy to just restart. Cattle, not pets, and so on. My IRC bouncer is one of the few exceptions to that. The only other thing I regularly hot reloaded was Centauri, my reverse proxy, but that had a whole config directory mounted because it was shared between containers, so nicely sidestepped the foot-gun.

So lesson learnt: check for weird bind mount issues before raising issues about config hot reloading. It’ll join the esteemed company of “maybe the drive is full and causing completely unrelated problems?”, “perhaps everything is dog slow because the kernel ran out of entropy?”, and “did systemd sneakily take over that functionality while you weren’t looking?” in the troubleshooting checklist.

An IRC bouncer is basically an always-on proxy. It connects to the IRC networks for you, then your clients connect to your bouncer. The bouncer can then send incoming messages to all your different clients, cache them when you’re offline and replay them later, and lots of other nice things people take for granted in their chat apps these days. ↩︎
Join/part/quit spam is part of IRC, and clients have ways of handling it, but I still feel bad about doing it excessively. ↩︎
You can disable this in vim by setting backupcopy=yes. Good luck redoing all the debugging if you ever accidentally remove that from your vimrc, though! ↩︎
Which is weird, because Docker can’t possibly be that old. That would make me much older than I’m prepared to accept. ↩︎
If you read the thread they didn’t actually mean to sound so dismissive, but it’s still pretty funny. “Yes, it’s expected that the foot-gun causes your foot to hurt. Duh. What did you expect?” ↩︎

An interesting Tailscale + Docker gotcha

2026-01-29T00:00:00Z

As I’ve written about before, I use Tailscale for a lot of things. I thought I had it set up in a reasonably secure manner, but I recently noticed a problem.

I use Tailscale’s ACLs to limit what each node can access, based on the tags I apply to it. So an app node can’t access anything via Tailscale, while an integration or server node can access things tagged with either app or integration. This is expressed pretty simply in the Tailscale ACL JSON:

{
	"tagOwners": {
		// Servers that can be SSH'd into
		"tag:server": [],
		// Applications that are exposed on tailscale but never connect out
		"tag:app": [],
		// Things which talk to other services over tailscale (connecting to apps etc)
		"tag:integration": [],
	},
	"grants": [
		{
			// Users can access everything
			"src": ["autogroup:member"],
			"dst": ["*"],
			"ip":  ["*"],
		},
		{
			// Servers and integrations can access integrations and apps
			"src": ["tag:server", "tag:integration"],
			"dst": ["tag:integration", "tag:app"],
			"ip":  ["*"],
		},
	],
}

Over the past week I’ve been setting up a private Forgejo instance behind Tailscale, complete with an actions runner that runs things using a docker-in-docker container. I didn’t want the runner knowing anything about Tailscale, so I had it configured to speak to Forgejo direct over HTTP (forgejo:3000) instead of using full Tailscale HTTPS URL that I use when accessing it (http://git.example-net.ts.net/)¹.

Everything was going fine, until I forgot to do that translation… and it worked. My Forgejo action runners could access anything on my tailnet. I run several things on Tailscale that just have authentication turned off, on the basis that only authorised devices can access them. Things like the admin interface for this website. I definitely didn’t intend for any workflow I run on my git server to have access to edit my website!

This wasn’t just limited to Forgejo, either. Any docker container I was running could access the tailnet. In hindsight it’s fairly obvious why: the host is running Tailscale, connected as a node tagged with my server tag. That creates a tailscale0 interface, and automagically sets up iptables rules to route Tailscale traffic over the interface. Docker also automagically sets up iptables rules to bridge traffic, and apparently these two sets of rules interact in such a way that traffic from Docker containers is allowed to route via the tailscale0 interface.

I say it’s fairly obvious in hindsight — there’s no reason why Docker would special case any particular host interface after all — but it still feels pretty surprising. Because both bits of software inject their own iptables rules, I never really had a good mental model for how they interact. The host Tailscale node was a completely separate building block to Docker. It would be a pain to use either of them if they didn’t do these rules, but it’s also one of the reasons I don’t really like “magical” things².

There are lots of ways to fix this, but none of them feel particularly great. You can configure both Tailscale and Docker to not automatically fiddle with iptables and handle the rules yourself, but I really hate dealing with iptables³. As a stopgap I did hold my nose and add some iptables rules to drop traffic to the tailscale0 interface if it originated from the IP ranges that Docker was configured to use:

iptables -I DOCKER-USER -s 192.168.0.0/16 -o tailscale0 -j DROP
iptables -I DOCKER-USER -s 172.17.0.0/12 -o tailscale0 -j DROP

The DOCKER-USER chain is a nice little escape hatch; it comes before the main auto-generated DOCKER chain, and Docker leaves the rules in it alone.

So I added these rules, and felt pretty good about myself, and then… everything started breaking in weird ways. After some debugging I realised the problem was DNS⁴. The server’s DNS resolver is 100.100.100.100, a special Tailscale address. This is so that it can resolve tailnet hostnames via MagicDNS (and so I don’t have to configure my custom DNS servers manually on each device; Tailscale does it for me). My new iptables rules inadvertently dropped all the DNS packets coming from docker containers. D’oh.

Obviously the solution here is to double down and add MORE iptables rules:

iptables -I DOCKER-USER -s 192.168.0.0/16 -d 100.100.100.100 -o tailscale0 -j ACCEPT
iptables -I DOCKER-USER -s 172.17.0.0/12 -d 100.100.100.100 -o tailscale0 -j ACCEPT

This got DNS working again, but it all feels a bit gross. Not least because to actually make these persist you need to use iptables-save, and then you also get all the rules that both Docker and Tailscale have inserted. I came across a script to filter out the Docker ones, but… yuck⁵.

I realised a cleaner idea might just be to lock down what server tagged nodes could access within Tailscale itself. That way I can avoid touching iptables at all. It’s handy to be able to curl services from the host when debugging, but it’s not really necessary. So I removed that access, and… stuff broke again. I’m using my Forgejo instance as a registry for some of the docker images I run, so the docker daemon needs to be able to reach it. I ended up making a new tag for infrastructure, which can be accessed from server devices. This does still allow all the docker containers to reach Forgejo, but I already have it set up with appropriate access controls and public/private repository splits. Forgejo is a service designed to run publicly, so this seems a reasonable trade-off for convenience. I used the tests feature of Tailscale’s ACL config to make sure I’d got the rules right:

{
	"tests": [
		{
			// Servers can only access infrastructure
			"src":   "tag:server",
			"proto": "tcp",
			"allow": [
				"tag:infrastructure:8080",
			],
			"deny": [
				"100.84.16.43:8080",
				"me@example.com:8080",
				"tag:server:8080",
				"tag:app:8080",
				"tag:integration:8080",
			],
		},
	],
}

The whole situation still feels a bit messy. If I ever get around to switching to nftables I might loop back and manually craft some rules for routing traffic, instead of leaving Tailscale and Docker to do their own thing.

I discovered later on that logging into the container registry at forgejo:3000 actually issued a redirect to https://git.example-net.ts.net/ so this was all basically for naught… ↩︎
I didn’t use Tailscale’s MagicDNS for a long time just because the word “magic” put me off. Only when I eventually got around to learning how it worked, and seeing that it wasn’t really that magical under the hood, did I change my mind. ↩︎
nftables seems far better in lots of ways, but I can’t really be bothered migrating. Maybe next time I reimage the server for whatever reason… ↩︎
It’s always DNS… ↩︎
Again, nftables would almost certainly help here. It actually has (gasp) configuration files. But again, I really didn’t want to spend the time migrating. ↩︎

Securing all the things with 1Password

2025-11-26T00:00:00Z

For many years I’ve been a keen user of Bitwarden. Recently I’ve had a lot of small paper-cut problems. The browser extension was redesigned and just doesn’t quite work how I expect any more. The prompt to save new login info misfired more than it worked. The mobile app stopped background refreshing properly. No one issue was enough to make me want to leave Bitwarden, but it definitely wasn’t sparking joy. Then there were rumblings about a price increase. I don’t think they’ve announced anything, but the numbers I saw were a jump from $10/year to $24/year. A nearly 2.5x price increase for a tool that was actively getting worse over time did not seem like a good deal to me¹.

I spent a while looking at alternatives, but there were very few that met my needs. Nordpass is highly praised but doesn’t support saving TOTP codes on personal plans². All the nice open-source/self-hosted solutions fall down when you want reliable synchronisation to an iOS device (which is fair enough). Proton Pass looked like it had all the features, but the pricing was asinine. Different pages showed different prices, the prices were introductory rates that jumped up on renewal, but that was hidden away in small print, etc. Yuck. The prices are also pretty high. More or less the same as 1Password, which I’d been discounting due to the cost, until now…

After eliminating basically every other option, I took a serious look at 1Password. It seemed to tick all the right boxes, I’d often heard good things about it, and it wouldn’t actually be much more expensive than Bitwarden if they went ahead with the rumoured price increase. I started a trial to give it a test.

Initial impressions

1Password is — as you’d expect for something that originated as a Mac app — very pretty. But it’s also highly functional. The looks don’t get in the way, they’re nice extras on top. When it’s locked, you get a large version of the 1Password logo, which is a front-on view of the barrel of a lock, next to the password input. When you authenticate, the barrel turns as though you’d put a key in and rotated it, and then the background splits in two and opens like a vault door. Does a password manager need to do that? No. Does it make me a little bit happy every time I see it? Yes. Look at it, it’s amazing!

The design philosophy carries through the rest of the app. Despite using Bitwarden daily, I often clicked on the wrong thing when I wanted to do anything other than autofill; in about a month of using 1Password I don’t think I’ve ever misclicked or been confused about the UI. The browser extension just feels snappier than Bitwarden, too. I’m not sure if it’s actually faster or if the design just makes it feel that way.

Functionality-wise, 1Password does everything Bitwarden did for me but slightly better. Bitwarden kicked you to its web interface for some operations (like checking compromised passwords); 1Password has that all built into the app. Instead of editing entries in a fiddly little browser popup, 1Password just summons forth the desktop app and you edit it there. It does a good job of detecting QR codes for TOTPs and offering to save them. I think the only small complaint I have is that the browser extension doesn’t have an autofill keybind. You can rig that up via the desktop app, but it feels like 1Password leans towards you using the UI it adds on login forms. To be fair, that UI works really well, and is very useful when you have multiple accounts on a site. My muscle memory for autofilling is strong, though.

Diving deeper

At this point I’d decided to switch fully to 1Password, so I started digging into what else it could do. The “Watchtower” feature combines a bunch of reporting about passwords (weak, breached, reused, and so on), but also lists sites where 1Password knows you can use passkeys or 2FA and don’t have them set up. I’d previously disabled passkey support in Bitwarden because it just appeared one day and got in the way of me trying to use a YubiKey, but decided to give them a go with 1Password. “Passkeys” is basically just a marketing term for FIDO2’s WebAuthn with automatic discovery bolted on top. The big problem is how you synchronise or export them. It’s all well and good letting your iPhone create a passkey when the Amazon app arbitrarily prompts it to, but what happens when you log in on a different device, or swap to Android, or whatever? Having 1Password deal with them fixes that³.

The passkey flow is very smooth: 1Password pops up an account selector, with obvious options to create a new account or pass the request through to another device like a YubiKey. It’s smart enough to guess the right account most times, even when you’ve got 17 different Google accounts for $reasons. Using them is equally simple: it just shows a popup and you tap to confirm you want to use the passkey. The Watchtower functionality made it really easy to go through and add passkeys everywhere that supports them. The thing I really appreciate is that they make the data that powers it public. It’s obviously partially a marketing thing for them, but it’s also a useful resource they could have kept proprietary.

Passkeys are a bit basic, though. 1Password has many more fun offerings. There’s an entire developer section with various goodies. At first I thought “that’s nice but I won’t use it”, but now I’m using basically everything it offers. The first thing to draw me in was the CLI. It’s pretty straightforward, but like the browser extensions it can nicely integrate with the desktop app so you unlock them all at once, deal with auth prompts in the GUI, and so forth. I have a couple of command-line tools that need passwords (e.g. for dealing with encrypted backups); now instead of prompting for the passwords, they shell out to op read and I authorise it via a popup in the 1Password app. Simple, but a nice quality of life improvement.

Next up there’s a beta feature called “Environments”. This lets you define groups of environment variables within 1Password, and have them exported automatically to certain destinations. At the minute it supports exporting to the AWS SecretsManager, and local .env files. When making a local file it actually makes a named pipe, so the credentials aren’t just sitting on disk (and won’t end up accidentally committed to anything). When the 1Password vault is locked, anything trying to read from the file will cause an authentication popup, much like with the CLI tool. I used to have the mentality that if something could read arbitrary files on disk, it was basically “game over” from a security point of view. Recent events made me realise there’s some value in defence-in-depth there, though, and 1Password provides a nice solution.

Fingerprints + Linux = Sadness

One feature of 1Password had me feeling a bit left out. Instead of entering your vault password to unlock it every time, you can have it invoke the system authentication library. On an iPhone this uses Face ID, on a Mac it can invoke Touch ID, and on Linux it uses Polkit. In the default configuration, Polkit just asks for your local user password, which is not much of an improvement. A fingerprint scanner seemed like the obvious solution here: it’s both a lot more convenient than typing a long password, and provides some assurance that it’s actually me doing the action⁴. Unfortunately, fingerprint readers on Linux seem to be a bit of a sorry affair.

Don’t get me wrong: there’s support. libfprint has a big list of supported devices, but I found it almost impossible to actually find a device I could buy that had one of the listed sensors. Most of them seem to be exclusively built into laptops. A lot of the cheaper devices just use whatever sensor they can get their hands on, so it’s pot luck if you receive a version that will work with libfprint or one that won’t. There’s also not a lot of information beyond that published by libfprint. I guess it makes sense: external fingerprint readers just aren’t a thing most people care about. While that makes me feel all special, it’s somewhat prohibitive to getting an actual working solution.

I’d given up on finding an answer to this when I stumbled upon an article by Scott Laird. He uses a YubiKey Bio to do it, via pam_u2f. The Bio isn’t actually a fingerprint reader: it just has an on-board reader to secure the credentials. The OS doesn’t get involved in the process. But because it can provide a fingerprint-secured U2F key, and pam_u2f can require that key to perform system operations, you basically get the same result. Maybe it’s actually even better than using a “real” fingerprint reader, as the data is secured on the hardware device…

The YubiKey Bio was a bit more pricey than I really wanted, coming in at around £90, but it felt like a worthy investment to add a bit of security and avoid typing a password dozens of times a day. The setup was simple: the official app that you use to enrol fingerprints is packaged in the Arch User Repository, and it worked perfectly. I already had pam_u2f set up as I used a (non-Bio) YubiKey to gate access to sudo, so I just generated a new key on the Bio, updated my config to use that, and added it to the Polkit policy. Now when I need to unlock 1Password or authenticate something, it prompts me to scan my fingerprint. Perfect!

Biometric all the things \o/

One of the other things Scott mentioned in that article was SSH keys. I’d seen 1Password supports acting as an SSH agent, but didn’t really see why I’d want it to. I used a resident key on my old YubiKey, so I didn’t have much to gain from a security point of view. Scott’s article made me realise I could have some big convenience gains, though. Currently every time I need to use the key, I enter a PIN and then physically touch it. I don’t mind this for the most part, but it turns out git-lfs does something like four different requests every time you do a git operation, none of which reuse the same SSH connection for some reason. That gets painful when each request involves a PIN and a touch. I also liked the idea of using pam_rssh for sudo authentication, but didn’t want to have to do the PIN-and-touch dance every time I used it.

You can probably see where this is going: I generated an SSH key in 1Password, and then… spent literal hours disabling all the other SSH agents that were running on my system, and trying to figure out where SSH_AUTH_SOCK was being populated. I didn’t manage to figure that out in the end⁵, and ended up just overriding it to point at 1Password’s agent socket. I added the new key to GitHub, and did the usual ssh git@github.com test, and it worked perfectly. 1Password prompts to allow access to the key, showing the process requesting access, and authorising it requires a fingerprint read:

can haz ssh key?

I’m not sure why Kitty doesn’t get an icon in the prompt; other applications do. You can expand the details section to see the key fingerprint being requested, as well as the exact process name and PID requesting the key. With the proof of concept done, I generated a couple more keys: one for normal SSH, and one for gaining root access using pam_rssh. Using different keys ensures I get prompted for them separately, and means a malicious app couldn’t sneakily ask for a Git key but actually pivot to SSH into a server. Not sure that’s ever likely to be a problem, but it’s basically no additional work for a bit more defence-in-depth, so why not?

I also took a moment to configure pam_u2f for both lightdm and xfce4-screensaver, so I can now log in and unlock my PC with my fingerprint. It’s not quite as smooth as Touch ID on a Mac, but it’s pretty close.

As an aside: this is not the first time I’ve been in this situation. Both Spotify and Todoist have focused on features I don’t want or need (including, but not limited to, pretty much everything branded “AI”), and then hiked the price up afterwards. It’s not quite standard enshittification, but it’s clear that our incentives no longer align properly. ↩︎
Or didn’t, at the time I looked. It now says “All plans” on the website but the docs only exist for business accounts, and there doesn’t seem to be any announcement explaining the change. ↩︎
Having Bitwarden deal with them would also have fixed that, but it got in my way instead of helping me, so… ↩︎
or it’s not me, but I’ll be busy dealing with the presumably-undesired removal of one of my digits, so probably won’t care about whatever is happening on the computer. ↩︎
Although I’m pretty sure the answer starts and ends with “systemd, somehow”. ↩︎

10 Weeks with an Apple Watch 10

2025-09-09T00:00:00Z

My watch. Yes, I am available for wrist modelling opportunities.

Around ten weeks ago¹ I picked up an Apple Watch 10, and have been wearing it almost constantly since. It’s not my first Apple Watch — I had a Series 5 for a bit back in 2020 — but it’s the first time I’ve actually stuck with it. Ten weeks seems like an apt time to reflect on it.

Firstly, why did I even bother? Well, for a couple of years I’d been wearing a Xiaomi Smart Band 7, mainly to monitor my sleep stats and set alarms that won’t wake up everyone else nearby. Its battery life was fantastic — with notifications and other things turned off, I got about a month of use between charges — but actually using it felt like trying to order food via the medium of interpretive dance.

My biggest gripe was the screen lock. If I didn’t have the screen locked then I’d periodically trigger it during the night when I moved around and it came in contact with my chest or leg. With the lock enabled you had to deliberately swipe up from bottom to top to enable interaction, but it didn’t work reliably. When I wanted to adjust an alarm, I’d be stood swiping repeatedly trying to get it to respond. When you finally get it unlocked, the whole interface is just fiddly.

The other issue was the data quality. There were some nights when I’d been woken up, sometimes even getting up and moving around, and it just didn’t show it in the data. If it can’t even get whether I’m asleep right, can I trust anything else it says?

I spent a while researching the best devices for sleep tracking. The Oura ring came highly recommended, but it was expensive and required a subscription to do anything useful. No thanks! The Apple Watch was consistently rated pretty well, and I reasoned I could pick up a refurbished older unit. I’ve been using an iPhone as my daily driver for a while, so it’d fit right into my begrudged walled garden.

The Series 10 has a significant advantage, though: it charges much quicker than all the previous generations. On a 30-minute charge, the Series 10 can go from 0 to 60%; the 9 can only make it to 40%, and my old 5 a measly 30%. Shorter charge times means I’m far less likely to leave it on charge and wander off without it. In some ways the daily charging is more convenient than monthly: the wireless charger sits on my desk, and I plop the watch on it for a little while in the evening; I don’t need to dig out the weird pogo-pin connector that has vanished sometime in the last four weeks, then carefully arrange it so it stays attached.

How a watch maybe saved my life

One of the big features of the Apple Watch, like many other wearable devices, is health and fitness tracking. I didn’t think much about this, beyond the sleep data I wanted, at first. I’ve never had a particularly good relationship with exercise², but I do like some good statistics. I started going for walks more often to get more data and see the graphs of VO2 max and HR recovery gradually inch up. That wasn’t the most profound effect on my health, though…

The recent versions of the Apple Watch have a feature that monitors for sleep apnea, a disorder where you don’t breathe properly during your sleep. I knew I wasn’t sleeping great — that’s why I was paying attention to sleep tracking data — but was still a bit surprised to get a notification from Apple Health after wearing the watch for 30 days. It gives you a graph to print and take to your doctor. So I diligently booked an appointment and a few weeks later went to see my GP.

The appointment went about as you’d expect: talking about referral to a sleep centre for a study, and so on. Towards the end, the doctor took my blood pressure (sleep apnea can be caused by, and can cause high blood pressure, in a lovely little vicious cycle). I don’t think either of us expected anything; it was just one of those standard checks for a related problem. After taking the reading, the doctor looked at me and said, “I can’t let you leave with a BP like this”. Yikes!

Blood pressure readings are split into stages: normal is under 120 mmHg³ over 80 mmHg⁴, stage 1 is up to 140/90, stage 2 is up to 180/120, and above is simply called “crisis”. Guess where I was? Also, fun fact: depending on how exactly you count and attribute things, hypertension comes out as the single largest cause of death in humans. It doesn’t kill you outright, but contributes to strokes, coronary artery disease, heart failure, and lots of other lovely things you don’t want on your CV.

After doing a few more readings, it settled down to just under the “crisis” stage and into the “holy crap, start treatment immediately” stage instead. I won’t labour on much more about this, but things are definitely moving in the right direction now⁵.

While the Apple Watch didn’t literally save my life, it triggered the chain of events that led to this diagnosis and treatment. Who knows what would have happened had it remained undetected? Probably nothing good. Also, go check your blood pressure! It’s simple to do and simple to treat if there is an issue, but so many people are walking around with hypertension and not even realising.

Building gates in the walled garden

Even though the watch is arguably a life-saver, not everything is rosy. It’s an Apple product, so you’re firmly locked in a walled garden, jealously guarded by people wearing black turtlenecks. Fortunately, there are a few ways to make it less painful.

All the health and fitness data is stored in Apple Health. You can export data as a big messy file, but it’s a very manual process to do so and the data format is gnarly. Luckily, there’s an app for that! Health Auto Export can, well, automatically export health data. It does what it says on the tin. It can send the data to Home Assistant, over MQTT, or dump it in some cloud file services, but I just have it post it all to a REST endpoint on a service I wrote. Then I dump it all in a database and can do whatever I want with it!⁶

Another tool that’s more useful on the watch itself is Apple Shortcuts. This is their no-code “if-this-then-that”-type thing. You can make automations or shortcuts that run a number of tasks. I have a whole slew of them I access via a complication⁷ on my watch face: one which prompts for input and adds an item to my to-do list (swiping to write letters is surprisingly not horrible), one which lets me select from a bunch of pre-written ones (“charge kindle”, “take laundry out in 1 hour”, type things), one to log my weight into Apple Health, one which can open and close the blinds in my room, and so on. It’s a surprisingly robust and easy-to-use system and offers just enough freedom that I’m not constantly grating on the edge of the walled garden.

Shortcuts being able to initiate arbitrary web requests is the real killer feature for me. Anything I can’t do on the watch itself, I can just farm off to a web server and connect it up with a shortcut. No need to learn Swift or pay Apple for the privilege of being a developer! For a lot of things, like controlling the blinds or adding to-do items, I already had a HTTP endpoint available and exposed over Tailscale⁸. Adding it to the watch was just a case of entering the right things in the Shortcuts app.

Daily nitty-gritty

There are lots of other little bits and pieces that come up when using the watch daily. I don’t think I can bundle them up into a pleasing narrative arc, so instead please enjoy some disjointed paragraphs of observations.

The Apple Watch has a lot of nagging abilities. It can notify you about your fitness “rings”, prompt you to stand up every hour, count how many seconds you wash your hands for, etc. I think I’d dislike these just on general principle, but the way it does them is so condescending it’s painful. I think there’s probably a cultural divide issue here, but there is no way in British English to say “Great job! You washed your hands for 30 seconds!” without it sounding like you’re being amazingly sarcastic or like you’re talking to a young child. So I turned all of that nonsense off. It’s meant to be a tool not a wannabe life coach.

You can access Maps directly on the watch and even do navigation. It works really well. The navigation mode has some nice haptic feedback: it does a short pulse as you’re approaching a turn, and then a long pulse at the actual turn. I like it a lot more than having to dig out my phone or have the directions read out. You get one pulse, glance down and see where you need to go, then it reminds you a little later when it’s time to do it. It’s a delightful user experience.

Watchfaces aren’t quite so delightful. There’s a limited number of built-in ones, and some are “exclusive” to the Ultra Series, and you can’t use them on a peasant watch like a Series 10. Annoyingly, there’s not one that does exactly what I want: a plain analogue clock with an inset date and four complication slots around the outside. Instead, I have to use one of the slots to show the date. There are third-party watchfaces, but they have issues. Firstly, there’s no actual API for making watchfaces⁹, so what they do is bodge it horribly by using a photo background that has fake widgets on it. On top of that they’re almost universally subscription-based. Again, no thanks¹⁰.

The issue I had with the Smart Band triggering when I was sleeping is solved trivially on an Apple Watch, by virtue of it having a physical button in the crown. When you put it in sleep mode, you have to double-press the crown to unlock it before it’ll do anything else. It hasn’t misfired once while I’ve been using it.

Sleep data; spot when I swapped devices!

Finally, a special mention for the gesture controls. If you raise the watch it reliably wakes up (switching from a low refresh rate, dimmed screen to an active, brighter one); you can then double-tap your index finger and thumb together and it will scroll down or page through whatever you’re looking at. The killer feature for this is navigating recipes: you can advance to the next step in a recipe while your hands are covered in flour. It’s also handy for reading notifications: when one pops up, you can double-tap to expand it, then, when it gets to the bottom, it’ll outline the default button (often “Dismiss”) and you can double-tap again to click it.

The verdict

I normally don’t like writing an actual labelled conclusion, but it feels like one is needed here! Overall, I’m happy with the watch. The daily charging doesn’t bother me, the data gathered seems reliable, the health monitoring has obviously paid dividends already, and the walled garden isn’t too chafing. It’s a straight upgrade over my old Smart Band, and I think it was worth the cost.

The original reason for getting the watch was better sleep tracking, though, so how well did it do? I’m much happier with the data: it seems to more accurately represent when I was awake in the night, and overall the sleep phases just seem to make more sense. You can see in the graph that the old data switched frequently between phases, and they didn’t quite line up for some reason; towards the right when the Apple Watch is supplying the data instead there’s a much more consistent pattern of sleep phases that repeat over the course of the night.

I’m not going to advocate that you go out and buy one, though. I know my requirements and usage aren’t typical, and I’ve also not got experience with any recent Android Wear alternatives or the new version of the Pebble watch that’s coming soon. You should definitely get your blood pressure checked, though!

OK, it’s more like 14 now, it’s taken a while for this post to make its way from my brain into text. ↩︎
Exercise for the sake of exercising just seems so overwhelmingly tedious and boring to me. And other types of exercise generally require social interaction, coordinating with people, and so on. ↩︎
Who decided to use “millimetres of mercury displaced” as a unit? You can’t just put random chemical symbols in units! That’s not how this works! ↩︎
Blood pressure readings have two parts: systolic (the pressure when the heart is beating) and diastolic (the pressure between those beats). They’re generally presented with the systolic reading on top and the diastolic reading below, and read as “X over Y”. Now you know what some of the random numbers they shout in medical shows mean! Yay learning! ↩︎
No thanks to how much salt is in everything. I’m pretty sure I’ve had at least twice the recommended daily amount of salt in a single serving before. Don’t even get me started on the things that are “low salt” but are still full of sodium from other sources. I don’t have a problem with ionic compounds, I have a problem with sodium! ↩︎
This mainly looks like drawing graphs that are slightly different to the graphs in the Apple Health graphs, for reasons I’m not sure I can explain. Making graphs is fun, OK? ↩︎
Complications are basically just home screen widgets, but with a fancy name because they’re on a watch. ↩︎
Tailscale actually causes me some problems here: everything works fine when the watch is connected to my phone, as the phone handles the Tailscale part, but if I’m not carrying my phone the watch will try to connect over WiFi directly and doesn’t understand anything about Tailscale. It happens infrequently enough that I’ll just live with it; it’s not much worse than having no signal on a phone. ↩︎
Yay walled gardens… ↩︎
I don’t object to subscribing to things in general, but it has to be something that’s worth the ongoing cost and offers something in return for the subscription. A watchface doesn’t need enough ongoing maintenance to justify subscribing to it, it’s just a cash grab. ↩︎