Chameth.com - posts like artisanal-docker-images, debugging-beyond-the-debugger, g15-ram-upgrade, infinite-avatars but not why-you-should-be-using-https

Building a family of single-user services

2026-08-27T00:00:00Z

Last month I made and wrote about a personal service to track my workouts and record personal bests. Since then, things have escalated. I now have eight different services running in the same “family”:

Name	Purpose
Collegiate Room	Simple weather overview for my current location
German Mall	LLM broker for use by other services
Gloss White	Weight tracking
Olive Crosses	Central service hub
Pompei Band	Workout and personal best tracking
Speed Carp	Spaced repetition system
Tiger Island	Reading history and library tracking
Typical Horse	Location history and analytics

The names are all anagrams of something related to their function. Finding those anagrams brings me an unreasonable amount of joy. Unscrambling them is an exercise left to the reader. On top of those eight, I currently have four other service ideas sat in my todo list. It’s going to be a big family.

A postmortem on the previous hodge-podge

Most of this functionality isn’t new to me. When I started out playing around with LLMs, I created a “personal data aggregator” that just kept getting things bolted on. I said at the time:

My biggest just-for-me project is a web app that I started to help me aggregate film recommendations. It’s since morphed into a general personal data aggregation service: it deals with data from GitHub, Todoist, Letterboxd, TMDB, Healthkit, and others. It also lets me make re-orderable lists, store recipes, and more. Parts of this could definitely be open sourced, and I might carve them out at some point, but it’s mostly a glorious hodge-podge of things specific to me. Having all these services in one place lets me make quick and dirty automations, for example: when I create a Todoist note on my phone or watch, I often forget to set the due date, so it doesn’t show up in the “Today” view. It was literally a few lines of code to plumb things together so any inbox task without a due date gets set to today automatically.

In a twist that nobody could see coming, it turns out that a “glorious hodge-podge” is actually quite hard to maintain and very annoying to work with. It didn’t help that it had a fairly extensive Svelte frontend and I really am not a fan of JavaScript. I ended up using it less and less just because it wasn’t fun to use or work on, and I didn’t want to waste the limited LLM tokens on doing massive refactors or reworks.

I’ve got almost two decades of professional software development experience under my belt, so you’d think I would have learnt the lessons on how to write maintainable software. Hell, my CV even mentions that as something I’m particularly good at. So what went wrong?

There were a few factors. One of the obvious things to blame is that it was written mostly with LLMs. Most of that blame isn’t very honest, though. In the year or so since, LLMs have definitely got better at writing good code; but I was manually reviewing all the code at the time, and telling it quite carefully what to implement and how to do it. So the buck really stops with me.

I think it’s actually more because it was my first larger project using LLMs. It’s a bit like learning a new programming language. I don’t think the first sizable project I’ve written in any language has ever been any good. You can read and learn as much as you want, but you need to actually run into some problems headfirst to get a grip on how the language works. Or at least I do. LLM coding agents are basically the same deal.

What the LLMs actually did was mask several problems. The architecture wasn’t right for how the service developed. It started out as a single-purpose app and gradually got things bolted on, without any major changes. Everything ended up in horizontal slices: a massive database layer with every query used by every part of the app, a massive HTTP layer, and so on. Working on a single feature became a tap dance all over the source tree. This sort of friction is normally quite apparent when you’re writing code by hand, and at some point it hurts enough that you throw in the towel and do some refactoring. But the LLM will happily plod on without exposing that pain, so the project needed a more active kind of architecture management that it just didn’t get.

The other problem they covered up was how brittle and unmaintainable the JavaScript frontend was. I’m not even sure how I ended up with a Svelte frontend in hindsight. It’s my preferred framework, but I still only reach for it as a last resort when there’s no other way to avoid using one. It seems like there were lots of ways to avoid it here, but I seem to have… well… avoided them. Anyway, each feature got its own slew of Svelte components, without much thought to reusability, shared patterns, or anything sensible. Some of them were just thin wrappers around data supplied by Go, others had vast swathes of logic in them, with no real rhyme or reason as to which way that would fall. It’s the same issue as before: a human would look at it, say “WTF?” and suggest tidying it up; instead, the LLM just gets on with what you tell it to do.

There’s also a big non-LLM factor: one of the primary reasons all of this unrelated functionality got tossed into the same app was that I’d previously worked on a few projects with a microservice architecture and really didn’t like it. My conclusion was they only really worked in large environments where you can dedicate people or teams to working on all the boilerplate that happens from maintaining lots of interconnected projects. Dumping everything and the kitchen sink into one badly architected monolith was, perhaps, not the best rebound from that.

Pompei Band and beyond

When I wanted to implement ~~Strava at home~~ my PB tracking app, it seemed fairly natural to build it out as a new service, rather than bolt it on to the slowly decomposing hodge-podge that was giving off a funny smell. Then when I wanted a weather dashboard, why not do it in the same style as Pompei Band? The same serving architecture, the same page layout… I could even come up with a similar weird two-word name!

Then wouldn’t it be nice if the weather dashboard knew where I was, so it could show the local weather? I could just build that functionality in, but it’d also be a good opportunity to carve out the location history feature from my old service and make a new, shinier one. Then basically every time I even thought about going near the old service, I mentally sketched out a replacement service.

Olive Crosses showing links to the other services

At one point I had the thought that it’d be nice if the services could all link to one another, but I didn’t want to have to touch every service whenever I added a new one. So I made a “hub” service, which maintains the One True Catalogue, and provides a bit of JavaScript the others can all embed to add a navigation sidebar.

So after complaining about microservices and JavaScript, it sounds like I have landed on microservices (or at least miniservices) and JavaScript? Sort of yes, but sort of no. Each service is still its own thing and makes sense independent of the others. There are some cross-service dependencies, but they’re very simple and don’t have the same cognitive or boilerplate overhead as a login service that talks to an authentication service that queries a user manager and so on. And as for the JavaScript, it’s almost exclusively used for progressive enhancement and a few limited client-side visual things like maps and graphs. All of the logic lives in Go, which is a lot easier to maintain and test, and all of the pages are simple Go templates.

Having these services built in a way that I actually like has motivated me to play around a lot more with new features:

My workout tracking app pulls photos from my private Immich server that were taken during workouts, and shows them alongside. Now I can see all my obligatory end-of-run selfies in one place.
The location history app can detect trips away from home, and get an LLM to automatically title them. My most recent, very exotic, trip has the title “A fortnight in Milton Keynes and Coventry”. Each trip gets its own little map, and in the future I want to pull in images like with workouts.
My weight tracking app now exports weigh-ins to Garmin, as it doesn’t fully integrate with Apple Health (it will export its own weights, but not import others, like the ones from my Withings scales). Stupid walled gardens, but thankfully I can fix it myself.
My book tracking app automatically cleans up the titles of books from the nonsense provided by Amazon to something more palatable. “Killing Eve: Long Shot: A twist on obsession, betrayal and revenge from Luke Jennings, author of the TV sensation Killing Eve” becomes “Long Shot (Killing Eve #5)”. It does this by passing the title to an LLM with a bunch of rules.
The hub generates me a snippet of Tailscale configuration that defines the minimal grants needed to allow the intra-service dependencies to work.

The LLM integrations were inspired by my friend Shane who recently demonstrated an inventory tracking app he’d made, which used Haiku to process images of boxes and describe the contents. I’ve dabbled with LLM integration before, and always disliked the process of fiddling with the prompts, and debugging when the LLM gave a nonsensical response. So I built German Mall, which makes it a lot easier. It lets me define a “workflow” consisting of a system prompt, initial message, and expected response format. I can browse past runs, tweak the prompt, rerun them to see the change, all without touching the services using them. Quick iteration loops are the best. German Mall also provides markdown documentation for each workflow, so integrating it downstream is simply a case of pointing a coding agent at the right documentation endpoint.

The “tidy book titles” workflow in German Mall

It’s not all fun and games: having a bunch of services all following the same pattern means you need to do cross-cutting maintenance on them now and then. But that’s surprisingly easy with LLMs. I put all the projects in a common folder, and I launch cross-cutting tasks from that folder. Usually I’ll make the change in one project, and then the cross-cutting task is simply “Look at commit XYZ in ./foo, make the same change to the other projects”.

I’ve also ended up with a shared library used by all the projects to cut down on some boilerplate. That’s normally something I’d resist, especially with personal projects, just because the extra overhead of working in a separate project, managing releases of it, and so on, is a bit annoying. But, again, LLMs can do almost all of that grunt work. They effectively reduce the “cost” of maintaining separate services to a negligible level, leaving me to focus on the more fun aspects like coming up with new names.

Migrating from GitHub to Forgejo

2026-04-30T00:00:00Z

When Microsoft bought GitHub in 2018 my kneejerk reaction — like so many others — was to start looking for alternatives. For a while I self hosted a Gitea instance but I never totally bought into it: some repositories I still pushed to GitHub, some I pushed to Gitea and they got mirrored, and I ended up causing myself problems when I got the two confused. Part of the problem was that the GitHub UI was faster and cleaner than Gitea’s at the time; using Gitea felt like a chore compared to GitHub. I ended up not maintaining it and eventually binning it and just going back to GitHub.

An all-too familiar unicorn

Fast forward eight years, and GitHub is about what we all imagined when Microsoft bought it. If you take the most pessimistic way of counting, they have zero nines of reliability¹. If you take the most generous, they have a single nine. That’s around 30 minutes of downtime every day. It feels like it must be more than that, given how many times you see the damned unicorn.

Uptime aside, they’ve crowbarred LLMs in all over the place; they don’t have the time to maintain official GitHub actions; the ones they do maintain have Copilot running roughshod all over them²; actions themselves are slow to run³, often have weird transient failures, and there are so many footguns that I’m pretty sure GitHub actions should be an entire category in the CWE top 10.

Speaking of security, while I was drafting this post a remote code execution was reported and the details are… impressive. The actual security issue was a pretty stupid oversight. Those happen. But they end up being able to execute code as a globally shared git user with access to all the other repositories on the node. What? I can’t quite get my head around it. No sandboxing, no containers, it just runs as a git user?

So, yeah, I’m not a fan of GitHub in its current state. A few months back I took the plunge and set up Forgejo, a fork of Gitea where they did radical things like add tests and ensure that it’s community operated not twisted for commercial use. Forgejo also potentially has some fun security problems⁴, but the way I host it mitigates more-or-less all problems. It’s also slightly less shocking when an open source project has lapses in basic security versus a $2 trillion corporation.

It’s taken a bit of doing, but I’m very happy with the setup I have.

The basic setup

I run all my server software using Docker, and Forgejo is no exception. They have decent documentation on how to get it running, and publish rootless images which is nice (and diminishes my urge to make an artisanal version myself). You can set configuration options using environment variables, which is perfect for containers, although the names can end up a bit unwieldy. Like this thing: FORGEJO__repository.signing__SIGNING_NAME=Chris Smith. It gets the job done, but the environment-to-ini mapping is pretty ugly.

I didn’t want to expose Forgejo publicly, as I wanted to avoid having to try and secure it⁵, or dealing with bots scraping it, or users signing up, and so on. I wanted somewhere to store my git repositories, handle CI for me, and mirror them somewhere public for other people to see and interact with. If you’ve read some of my other posts you’ll probably guess where this is heading: Tailscale. Alongside the Forgejo container, I run a Tailscale instance with a serve.json that covers both the web frontend and the SSH listener used for git operations:

{
    "TCP": {
        "22": {
            "TCPForward": "forgejo:2222"
        },
        "443": {
            "HTTPS": true
        }
    },
    "Web": {
        "${TS_CERT_DOMAIN}:443": {
            "Handlers": {
                "/": {
                    "Proxy": "http://forgejo:3000"
                }
            }
        }
    },
    "AllowFunnel": {
        "${TS_CERT_DOMAIN}:443": false
    }
}

This allows me to interact normally with Forgejo from any of my devices or servers (which all run Tailscale), while not exposing it to the Internet at all. It also handles the TLS certificates automatically, which saves me setting up something else to do that.

For CI, I run the Forgejo runner image on the same server, and gave it a dind container to use to actually run the workloads. dind is the cutesy name for Docker-in-Docker, basically a Docker daemon running inside a Docker container. That keeps the CI workload separated nicely from the “production” workload running on the server, which is nice from both a security and a monitoring point of view.

The uncanny valley of actions

Forgejo actions are basically like GitHub actions: they run a series of steps using one or more container images. You define those steps in the exact same clunky YAML format as with GitHub. The big difference between them is that GitHub defaults to using an absolutely massive base image filled with out-of-date preinstalled software, and Forgejo leaves that for the administrator to configure. You could, of course, just use the same image as GitHub, but lugging around a multi-gigabyte image that doesn’t even have recent versions of the software I want isn’t really my style. Instead, I set about using a plain debian image as the base, and then did a face-palm when I tried to run a “normal” action and remembered they’re all written in JavaScript, so expect node to exist.

Accommodating JavaScript isn’t really my style, either, so I did the very sensible thing of writing my own suite of actions in Go. Being written in Go means they can be compiled statically, published as a container, and manage their own very limited dependencies. For example, the checkout action is built into an alpine container with git added, while the dockerbuild action uses the buildah image as a base. Each image has the tools it needs, pinned to a recent version, and nothing more. This is very much not a sensible approach to take for most people, but it made me happy. With six basic actions, I could replace almost all the ad-hoc workflows I had in GitHub⁶.

Other than the base images and my self-inflicted JavaScript machinations, everything works the same as GitHub. But faster. So much faster. Most of my CI workflows finish on Forgejo before GitHub would even have allocated a runner to the job⁷. And I can actually view the logs without them glitching out.

Step aside Dependabot, here comes Renovate

One of the boons and/or banes of hosting code on GitHub is access to Dependabot⁸. It’s a tool that monitors your dependencies, and automatically submits PRs when new versions are available. It also does security alerts, but they’re comically bad for Go at least⁹. I do like having dependencies automatically handled, especially with a cooldown, which is how I had Dependabot configured before moving.

To replace it, I’ve configured Renovate, an open source alternative. I’d seen it used a bunch before: both by projects on GitHub who prefer it over Dependabot, and by people using GitLab and other platforms where Dependabot isn’t. It mostly does the same things, but Renovate has a few very nice extras.

Firstly, it supports transcluding config from another repo. So in each of my many, many projects, I just have this stub of a renovate config:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "local>meta/renovate"
  ]
}

Then the actual config lives in a default.json in the meta/renovate repository. I mirror that repository to GitHub, if you want to have a look. This is an amazing feature. When I enabled cooldowns in my Dependabot config it was a slog to go through all my active repositories and make the same change over and over again. With Renovate I can configure that centrally. As far as I know Dependabot can’t do anything like that, even for enterprises.

Within that central config I use some of the other nice features. It has the ability to auto merge changes. I use this to automatically accept changes from trusted projects, or my own libraries:

    {
      "matchManagers": ["gomod"],
      "matchPackagePatterns": [
        "^github\\.com\\/csmith\\/",
        "^chameth\\.com\\/"
      ],
      "matchUpdateTypes": ["minor", "patch"],
      "automerge": true,
      "minimumReleaseAge": "0 days"
    },
    {
      "matchManagers": ["gomod"],
      "matchPackagePatterns": [
        "^github\\.com\\/csmith\\/",
        "^chameth\\.com\\/"
      ],
      "matchUpdateTypes": ["major"],
      "minimumReleaseAge": "0 days"
    },

This takes any minor or patch release of my libraries, and automerges it with no cooldown. For major version changes it overrides the cooldown but doesn’t automerge. To automerge Dependabot PRs you have to write an action, implement that logic yourself, and hope you’ve successfully avoided all the footguns inherent in that.

Speaking of major version updates, Renovate also has an option to automatically update the imports in Go packages when it’s offering a major update. In the config I have:

"postUpdateOptions": ["gomodTidy", "gomodUpdateImportPaths"],

Which means that every time it does an update it’ll run go mod tidy, and it’ll update the import paths. In Go, v2 of a module has a different import path to v1 (so they can coexist if transient dependencies need different versions), so usually you have to go through and find and replace those paths. Renovate handles that for me now! I still have to address whatever breaking changes are in the new version, but there’s a lot less grunt work.

It can also run custom commands¹⁰, so for example in my legotapas project I have this extra snippet in the renovate.json:

  "postUpgradeTasks": {
    "commands": [
      "sh -c 'rm plate_*.go'",
      "go run ./cmd/generate"
    ],
    "executionMode": "branch",
    "fileFilters": ["*.go"]
  }

I won’t go into the gory details, but this handles automatically regenerating a bunch of files that change every time a certain dependency is updated. Previously I had a separate action to do that on GitHub; just being able to drop commands in the config file is much nicer.

To actually run Renovate, I simply have a workflow in Forgejo on a schedule that uses their official Docker image. It also triggers if I update the config, which is nice when I add a new rule, as it’ll immediately update the open PRs to comply. Overall, Renovate has felt like a pretty big quality-of-life upgrade.

Odd problems

Not everything has been smooth sailing. Sometimes when Renovate automerged a change, it would just… disappear. The PR was closed as merged, but the commit never landed on master. It wasn’t a huge issue as the next Renovate run would just recreate it, but it would then disable the automerge and I’d get confused as to why. It turns out there was a major problem with how I had everything set up. Renovate was telling Forgejo to automerge when the required checks pass, but I had no branch protection rules set up in Forgejo, so as soon as the PR was created it was eligible to merge. As the CI was so fast I hadn’t noticed that the automerges were happening before the CI results were posted.

The disappearing commits were because the repository was getting into a bad state when Renovate tried to automerge multiple changes instantly in the same repo. Forgejo has a bunch of maintenance commands that helped to fix that problem, and I then wrote a little script to enable branch protection rules so the CI would actually run. The automerging has been smooth since.

Another issue I had early on was that merging would fail because the GPG keyring was locked. I had Forgejo configured to sign all the automatic commits, so you can verify what commits came from my infrastructure. Forgejo is the only process using the keyring, so I wasn’t sure why it was becoming locked. It turns out that GnuPG uses the hostname as part of the lockfile, and because Forgejo was running in Docker it was getting random hostnames based on the container ID. Explicitly setting a hostname made GnuPG and by extension Forgejo much happier.

Webhooks and updating

Forgejo has this amazing feature: you can add a webhook that’s called for all your repos. GitHub, for some reason, supports this for organisations but not people. Even though the two are treated the same in a lot of places. It doesn’t seem like it would be hard, and it’s such an obvious thing to want, but… no. I wrote a whole tool to add webhooks for me to get around this. Now I don’t need it, Forgejo does the right thing!

One particularly interesting use of these webhooks has been making my Docker containers auto-update when a new image is pushed. I previously had a webhook receiver for a few projects that would update them automatically, but GitHub package notifications were extremely laggy. Every other webhook fired within a minute or so of the event happening, but package webhooks would sometimes take 30-60+ minutes to fire. I opened a support case with GitHub and it got escalated to engineering and fixed about 10 weeks later¹¹. It was better for a while, and then got worse again. Trying to do continuous deployment with a 30 minute random lag was frustrating.

So now I had reliable webhooks, but the update process was a bit gross. I have a hangup about internet-facing services not having access to the Docker socket. It just feels like too big a security risk. So I had a process that received hooks, and then another that figured out if containers needed to be updated and restarted them. Adding a new container to the system meant recompiling the second process, which was a massive faff.

Fortunately, I was saved from this mess by my friend Greg. He wrote adze, a tool that receives webhooks and uses Docker Compose to update matching containers. He wrote a blog post about it. It’s all one process, but that’s no longer a blocker for me because the webhooks can be delivered privately from my Forgejo instance without exposing adze to the internet. It finds matching containers automatically based on the images they’re using¹², and combined with Forgejo not forcing me to add a webhook to every single repository makes my life so much easier. I write some code, push it to Forgejo, and within a minute or so it’s running on my dev instance.

I have IRC notifications set up for when adze does updates, and I still have some old ones that announce GitHub package webhooks. I keep seeing things like this and chuckling:

15:44:42 <@ircjag> [ADZE] chameth.com: pending updating git.yak-wall.ts.net/public/chameth.com
15:44:45 <@ircjag> [ADZE] chameth.com: success updating git.yak-wall.ts.net/public/chameth.com
17:11:54 <@ircjag> [GHCR] Container pushed to csmith/chameth.com:dev.

I pushed a change at 15:43, Forgejo built it and pushed the image (to both its own registry and to GitHub), then at 15:44 adze updated the container to the new version. An hour and a half later GitHub gets around to delivering its own webhook.

Mirroring and PRs

The final piece I had to figure out was how to actually interact with people. Most of my projects are small and don’t get a massive amount of public contributions, but the option should still be there. And ideally not involve anyone having to figure out how to e-mail patches¹³.

Mirroring outwards is simple: Forgejo supports it natively. Whenever I push to Forgejo, it pushes out to the corresponding repo on GitHub. I’ve also started mirroring some projects to Codeberg, to avoid having GitHub as my single “public” presence. But what happens when I get a pull request? If I merge it in the GitHub web UI, Forgejo will push its own branch over the top and get rid of it¹⁴. I could pull the changes, merge them, and push the result to Forgejo but then I don’t get the benefit of any CI.

I ended up writing a small private tool that deals with this for me. I give it a GitHub PR, and it effectively imports it into Forgejo for me. I created a prs organisation, and it’ll create a fork of the project in that org if it doesn’t exist. Then it pushes the PR contents to a branch, and creates a PR from the prs repo into the main one. It runs as a user which can write to the PRs org, but only submit pull requests to public repositories outside of it.

When the PR is imported, I can review it in the Forgejo UI, and then I can approve the CI workflows for that particular PR. This gives me some extra defence-in-depth in case anyone is trying to abuse one of those action footguns. Once the CI passes, I can merge it as normal in the Forgejo UI, it gets mirrored out to GitHub, and GitHub marks the PR as merged automatically. It’s not the most elegant solution, but the actual workflow is pretty smooth, especially for the low volumes of PRs I get.

With that, I have Forgejo set up to do everything I need from a git host, and I’m no longer hamstrung every time GitHub goes down.

This way of counting isn’t particularly fair, but it’s somewhat realistic. GitHub break out their uptime for things like git operations, pull requests, actions, webhooks and packages. Realistically if any of those independent things are down then my overall workflow doesn’t, well, work. ↩︎
To be clear, I’m not objecting to the use of Copilot here. It’s a tool. But if you use a tool to do something fairly simple and it spews out 35 commits, including at least 5 reverts, then either it’s a terrible tool or you’re using it very wrong. I didn’t hunt down this example, I came across it while trying to debug something and wasted a bunch of time trying to figure out what on earth was going on. ↩︎
You could self-host runners, which may help, except they’ve already threatened to charge you for the privilege of bringing your own runners. ↩︎
If people could stop finding RCEs in git services for a few days so I can stop redrafting this intro, that’d be great. ↩︎
I’m feeling pretty smug about this decision after reading about the security issues. ↩︎
There’s one holdout: my dockerfiles repo which is a very special snowflake with a very complex workflow and I’m still deciding how to handle it. ↩︎
Which is to be expected, given I’ve got a server dedicated to one person, and they’re trying to serve however-many million users at once. That doesn’t make it any less nice, though! ↩︎
I’d link to it but there’s nowhere really good to link to. It’s just been subsumed into the ‘security’ functions of GitHub, along with a bunch of copilot nonsense. ↩︎
The go project maintain a vulnerability database that includes details about which exact symbols are affected, and the govulncheck tool uses this to only report security issues in code that’s actually used. Dependabot doesn’t do that: it constantly alerts about code that’s not imported, so won’t ever be shipped. ↩︎
In the default config it doesn’t allow anything of the sort, of course. You can whitelist individual commands, but because this is a private server I can merrily whitelist .* and do whatever I like. ↩︎
The turnaround time isn’t particularly impressive, but the fact that they kept the support case open, and the (human!) support agent updated me several times is. I don’t have a lot of good things to say about GitHub, but I don’t think I’ve ever had a support experience that good from a company of a similar size. Especially as I wasn’t a paying customer at the time. ↩︎
Why didn’t I do that in the first place? It seems so obvious now… ↩︎
I really like the idea of an e-mail based workflow, but I absolutely hate actually doing it. ↩︎
This is what happened to me a lot when I was half using Gitea. ↩︎

Making a font of my handwriting

2025-08-08T00:00:00Z

Recently I’ve been on a small campaign to try to make my personal website more… personal. Little ways to make it obvious it’s mine and personal, not just another piece of the boring corporate dystopia that is most of the web these days. I don’t quite want to fully regress to the Geocities era and fill the screen with animated under construction GIFs, but I do want to capture some of that vibe.

I’d added some bits and pieces along those lines: floating images in articles now look like they’re stuck to the page with sellotape, related post links have a wavy border that animates when you hover over them, and so on. Next, I wanted to change the heading fonts from a monospace font to something cursive, to resemble handwriting. Less terminal output, more handwritten letter. I couldn’t find one I liked, though. So why not make my own? It can’t be that hard, right?

Failing to do it myself

I set out to try to make the font myself using open source tools. After doing a bit of research, it seemed like the general approach was to create vectors of each character and then import them into a font editor. That seems to mean either Adobe Illustrator and FontLab (if you have too much money) or Inkscape and FontForge (if you like open source). I fall firmly into the latter category, so I grabbed my graphics tablet and opened Inkscape.

I wrote out my first three letters: capital A, B and C. Saved them in Inkscape, and attempted to import them into FontForge. Then I remembered one crucial thing that had slipped my mind: I absolutely loathe using FontForge. It’s a bit like when you open an old version of GIMP and get a bunch of weird looking windows floating all over the place; it feels like you’re fighting against the tool to do even the most basic operations. The difference is I have cause to edit images a lot more than I edit fonts, and GIMP has actually significantly improved their UI over the years.

Here are the rough steps I went through with FontForge:

Launch Font Forge. It shows a weird bit of art in one window, and an open file dialog in another.
I don’t want to open a file, so I close that dialog. The program exits.
Relaunch Font Forge, and realise that within the “Open Font” dialog is a “New” button. Click it.
Get to the standard font-editing UI. Right-click on the “A” looking for a way to import an SVG. Don’t see one.
Click around a bit, exploring the menus. Everything feels a bit off. You can’t open one menu then hover over the next to see its content, like basically every UI toolkit in existence. I think FontForge has eschewed QT and GTK in favour of doing things itself.
Find the “Import” option in the File menu. Hope it’s for a single glyph not the whole font.
A file picker opens. Again it’s all a bit off from normal desktop conventions. Try to resize it, and just get blank grey space at the bottom.
Type the absolute path I want to go to in the text field.
Get a dialog saying “Not a bdf file /home/chris/etc”. Press OK.
Get a dialog saying “Could not find a bitmap font in”. Press OK.
Press Ctrl+L to see if that lets me enter a path. Click everything in the dialog to try to find a way to enter a path. Get annoyed. Give up. Click through folder-by-folder to get to where I want to be.
Get to the folder and don’t see any files. Change the format to “SVG”. Double-click the newly-visible SVG file.
Get a dialog saying “You must select a glyph before you can import an image into it”. Press OK.
The import dialog goes away, having not imported.
Select the glyph in the main tool area, then repeat the File→Import dance.
It’s actually there now! Open the glyph in the editor and see it’s a complete mess of Bézier curves. I can’t click what I want without accidentally moving a handle for an adjacent curve.
Rage-quit.

I’m sure FontForge is less anger inducing once you’re used to it. And you definitely could use it to build a font like this if you had much more patience than me. I’d had enough of death-by-a-thousand-paper-cuts though.

I briefly tried Inkscape’s built-in support for making an SVG font. It annoyed me a lot less, but it’s fiddly: it seemed like each glyph had to be a single path, so you had to convert the glyphs to paths, then merge them correctly. If you merge them incorrectly then the wrong bits of your letters end up filled (like the inside of the ‘B’). Path manipulation is getting towards the limit of my knowledge of vector editing, and it took a bit of trial and error for each letter that had more than a single stroke. I didn’t fancy doing that for every letter.

I’m usually a big advocate of open source, but this was one of those painful times when it feels like it just falls short. Clunky, painful UI and processes where commercial tools just let you get on with your work.

You can exchange money for goods and services

When I’d been looking for open source tutorials, I found many mentions of a closed source, hosted tool: Calligraphr. It has a free version with limitations (no ligatures, no variations, 75 glyphs per font), and a pro version for £8/month. I’d normally balk at the idea of a subscription for this, but they have the perfect answer: you can make a one-time payment, and your account automatically downgrades back to free after a month. It’s not a hidden option, either, it’s the most prominent button on the upgrade page. That made me happy to give them £8 to play around with the service for a month.

Calligraphr works by having you print templates, write out the letters, then scan them in. It does some magical processing to extract the glyphs, provides tools to tidy them up, align them, etc, and then produces a TTF file for you. You can see some of my completed templates here:

Most of the templates I used for the font

Calligraphr has a nice UI to generate the templates, allowing you to select which glyphs to include. I added the “minimal English”, “basic punctuation” and “Ligatures” sets. That gave me four pages to fill out, and I did them all twice. That let me filter out versions that didn’t work well, and have variants for some letters so the font wasn’t too repetitive. Later on, I went back and added some custom ligatures based on blog post titles that didn’t look quite right: “Re”, “To”, “ers”, “ey”, “hy”, “ra”, “re” and “ty”. Ligatures like this help it look more natural: when we write we don’t just stamp out identical letters regardless of their surroundings, instead they will connect to their neighbours, or overlap slightly, or even share a stroke.

I filled these templates in with a Sharpie, as I wanted a fairly informal, scrap-booky look, and it would also give good solid shapes that should be easy to pick out of the template. I scanned them with the “Scan Document” function on my iPhone, and uploaded the PDFs to Calligraphr.

Iterating and tweaking

The Calligraphr UI allows you to preview the font, but I found it a lot more useful to just download a copy and use it on a local copy of my website. That let me test it with real text, and see how it’d look at the different font sizes I use on the site.

The first version was not great. Despite the guidelines on the template, I apparently wasn’t good at sticking to them. Some letters were floating way off the baseline, and some were sunken below. When those opposites met it looked terrible. Fortunately Calligraphr has a pretty easy tool to slide each letter up and down, and scale it up or down if needed, and you can see it next to other letters as you do it. It took a little bit of time to go through all the variants of all the letters, but the next version looked a lot better.

Another tweak I ended up doing was reducing the spacing between letters. The defaults Calligraphr uses are probably good for a blocky font, but I wanted to put the letters close together to give it more of a joined-up look. Again, this is an easy tool to use, you just drag the sides in or out as desired. While these tweaking steps were probably as fiddly as some of the Inkscape steps I refused to do earlier, they’re a lot more rewarding as you see things improving with each one. It’s a lot easier for me to commit time and effort to improving something that’s already working reasonably, than put that time and energy into an unknown.

Later, I noticed that occasionally there would be a huge gap in a title. Not “the kerning is slightly off” but “there’s enough room to park a bus”. It took me a while to figure out what was happening: a couple of glyphs hadn’t been isolated perfectly and had picked up a few pixels from the template lines at the edge of their boxes. That meant the glyph had a width that covered the actual written glyph, a big gap, and then the rogue marks. At first, I fixed this by just adjusting the width, but that left the little pixels floating awkwardly down-sentence. The proper fix was to use the editing tool and simply delete them, and then Calligraphr snapped the width back to what it should be.

These iterations took a while to do, but I just dipped in and out occasionally over the course of a week, so it didn’t actually feel like too much work. I quite enjoy the process of refining things, too.

Result and a surprise

If you’re viewing this post on my website¹, you can see the font in the headers, captions, and a few other places. Here’s how it compares to my actual handwriting:

My handwriting vs my handwriting font

It’s not close enough to forge documents, but I think it definitely gets across my style, and that’s exactly what I wanted. It’s surprisingly legible even at smaller font sizes — I think the weight of the Sharpie helps here — and at £8 and a bit of manual work was a lot more economical than spending days wresting with open source tools.

A few weeks after I put the finishing touches on the font, I got an e-mail from Calligraphr. As my account had lapsed back to the free version, I was no longer eligible for the “server-side backup” feature. So what did they do? They e-mailed me an exported copy! It’s a JSON file with the properties of each glyph and a base64 encoded image. Not only can I re-upload this to Calligraphr if I resubscribe, I can probably hook something up to edit it should I ever need to. I’m blown away by how pro-user Calligraphr’s business practices are. They’re up-front about pricing, don’t try to get you stuck on an auto-renewing subscription, and automatically export your data. It’s like a breath of fresh air compared to the barrage of dark patterns that other websites foist on us. If you want to make this kind of font, I’d definitely recommend them just because of how nice they are.

And I haven’t changed everything since writing this post… ↩︎

Coming around on LLMs

2025-05-28T00:00:00Z

Claude says hi.

For a long time I’ve been a sceptic of LLMs and how they’re being used and marketed. I tried ChatGPT when it first launched, and was totally underwhelmed. Don’t get me wrong: I find the technology damn impressive, but I just couldn’t see any use for it.

Recently I’ve seen more and more comments along the lines of “people who criticise LLMs haven’t used the latest models”, and a good number of developers that I respect have said they use coding models in some capacity. So it seemed like it was time to give them another shake.

The first decision to make was which model to try. OpenAI are no longer the only player in the game, every tech company of a certain size is now also somehow an AI company¹. I looked a bit at some benchmarks, and then mostly ignored them and went with the only company that I didn’t outright hate: Anthropic, and their model Claude.

Initial impressions

The latest Claude models do feel a lot more “capable” than the earlier ChatGPT versions I remember, but they also still have a lot of the same problems. At their heart, they’re still text-prediction models, and still seem to be trained to predict text that will please the user rather than be factually accurate or useful.

One of the improvements that people kept mentioning was the ability for models to access the web directly. I’ve seen it kick in a bit naturally, and asked for it explicitly sometimes, and it’s… nothing special? Are people just really bad at searching the web? Maybe they should try Kagi? I feel like about 80% of the time I could have found the information just as quickly myself, about 10% of the time it searched and then hallucinated an answer, and the remaining 10% it found something quicker than I otherwise would have. Those aren’t great results, especially when you consider how much cheaper and simpler just searching the web is.

There is another way to trigger web searches in Claude: using “research mode”. When you enable it, it splits off into different models: a “lead researcher” to come up with a plan, and then some minions that execute it. It ends up doing hundreds of web queries in the span of a few seconds. Suddenly I understand why things like Anubis need to exist. I’ve not used the “please launch a DoS attack” button since.

What did impress me, though, was its ability to churn out reasonable-ish code. It can hack together a bash script as well as I can, and do it far faster than I’d be able to. Sometimes they even work. That made me wonder what it would be like doing actual coding with it. Anthropic have a CLI tool called claude-code, so I paid them lots of money and gave it a spin.

Coding with Claude

The first thing I notice about claude-code is that I really like the interface. It’s basically an input box in a terminal. It doesn’t force me to use a certain IDE or do things in a certain way. By default it asks before making any changes, showing you a side-by-side diff of what it’s doing and allowing you to provide feedback. If I had to design a way to interact with a coding agent from scratch, I can’t think of many things I’d improve.

A simple example of a claude-code session

The power in claude-code versus just using the web UI is that it can use tools. It can query git, run grep commands, even use sed if it wants to change something in lots of files at once. It’s very good at figuring out its way around a codebase, even without any explicit instructions. You can see in the screenshot that with a little prompting it managed to get the permalink to this post; I didn’t tell it where the posts were stored, or how to work out the latest, I just told it when it was wrong. If I’d run /init before it would have probably picked up on the fact that my posts have custom permalinks, and noted it in CLAUDE.md.

But how good is it at actually writing code? It’s like having a keen but not particularly thorough Junior Engineer at your beck and call. If you give it a clearly defined task and guidance on how to implement it (and maybe some feedback as it suggests changes), it’s more than capable of doing it. If you don’t give it enough guidance it tends to go more off the rails. I tried having it generate a simple application from scratch with minimal technical guidance and no review of what it was doing, and it made such a mess of it I decided it was quicker to throw it away and start again by hand.

The man behind the curtain

Even with sufficient guidance, at times it’s really obvious that it’s an LLM generating pleasing-token-strings and not something that genuinely understands what it’s doing. It will spit out code like this:

if err != nil {
    if err == sql.ErrNoRows {
       return nil, err
    }
    return nil, err
}

Why’s that check for sql.ErrNoRows there? It’s entirely pointless. There was no instruction to check for it, none of the existing code checked for it, but I assume it comes up quite a bit in the training data. But it didn’t understand why, so it put the check in, and returned the exact same thing as if it hadn’t.

It also occasionally tries to “cheat” or solve problems the wrong way. It sometimes feels a bit like you’re asking for wishes from a Monkey Paw. “Stop the unit tests failing”, you’ll say; Claude will respond with a request to delete the failing test. The man behind the curtain isn’t particularly well hidden, and the training data and pattern matching often shows through.

This lack of understanding also makes it very hard to get Claude to use comments in a sensible way. It absolutely loves doing nonsense like this:

// render form
function renderForm() {
    // ...
}

Even with explicit instructions to avoid useless comments, or comments that only explain what the code does (not why it does it), or other prompts. Again, I’m chalking it up to a mixture of training data that does that, and lack of any actual understanding about why a human developer may want a comment to exist. Of course, there are plenty of flesh-and-blood devs out there that also don’t comment effectively, so maybe I shouldn’t give Claude too much of a hard time on this.

… and yet …

So with all those problems, it sounds like it’s just not worth it, right? Well, not quite. I feel like programming mostly consists of two distinct tasks: thinking how to implement things; actually implementing them; and then debugging why you’re off-by-one somewhere. Letting an LLM do the thinking is a big no-go for me: it’s evidently not good at it, and frankly that’s one of the things I most like about programming. Having let it loose on small projects, I shudder to think what the codebases of all the “vibe coded” projects that are popping up are like.

For the second stage, though, it’s actually quite nice. If I’ve thought through how I want something to be implemented, I can feed those steps to Claude and have it churn out the otherwise not-too-interesting code. This requires far less time and concentration on my part than writing code, to the extent that I can be thinking about the next feature, or doing something else at the same time. I don’t feel like I’m going to end up deskilling myself this way, as I’ve already formed the idea of what I want to code, I’m just using the LLM to spit it out faster than I can type it.

A lot of the most boring bits of coding like implementing CRUD-y operations can be summarised as “Look at this file/function. Do the same thing but slightly differently elsewhere”. Claude is great at this, especially when explicitly prompted like that. You’re basically playing to its pattern-matching strengths, rather than asking it to come up with anything novel. Most of my prompts tend to be prefixed with “Look at @some_file.go and @other_file.go.” to cue up the patterns I want it to use, and I find this works well.

As for debugging, it’s a mixed bag. It’s sometimes amazingly insightful, and sometimes just runs around in circles trying the wrong things over and over. It’s worth asking the question, but I definitely wouldn’t rely on it over my own abilities.

The future

I’m probably going to carry on using claude-code, at least for personal projects. I’ve got so much done that I just wouldn’t have been bothered to do if I was doing it all by hand. I very much enjoy being in the more “diffuse thinking” mindset, planning how things are going to work, rather than being stuck in the mines digging out SQL queries. After all, who wouldn’t want an over-eager assistant to work on all their hobby projects?

Work is a slightly different matter: a private project or even an open source project that disclaims any liability is different to something I’m being paid to deliver, and bear responsibility for fixing if it’s not done correctly. I’m not saying I won’t use it at all, but if I do it’ll be much more constrained than I would in personal projects.

As for non-code usages: I’m not sold. My sceptic hat is still firmly in place. I don’t think chat is a particularly good interface for many things, and hallucinations are still a big problem despite what people say. I hate the tide of AI slop that’s taking over the Internet, and how LLM-powered chat agents are being forced into every random product. You’re definitely not going to be seeing any AI-generated blog posts from me!

One topic I’ve not gone into here is the ethical concerns about using LLMs. They obviously exist, and I do have thoughts, but that’s a topic for another day.

I’m surprised it’s not gone more mainstream: why’s there no Tesco Value LLM model, yet? ↩︎

Upgrading the RAM in a Dell G15 laptop

2023-07-29T00:00:00Z

The Dell G15

I currently use a Dell G15 laptop for work. It has served me well for a little over a year, but recently it has been struggling a little with my day-to-day workload. It came with 32GB of RAM — the highest possible specification at the time¹ — but that is apparently no longer enough for me.

For a recent project, I was working on a Rust library used in an Android app. That meant running the usual glut of Android tools (Android Studio, an emulator and at least one Gradle daemon) alongside a normal IDE (IntelliJ IDEA). Throw in a web browser and a couple of electron apps, and I often managed to use all 32GB.

When you start swapping memory out to an encrypted disk — even an SSD — it doesn’t make for great performance. At first, I tried to work around this by enabling the Linux out-of-memory (OOM) killer, but it turns out that it’s not too good with Electron apps: it will kill the large browser process, but then the small Electron wrapper will just respawn it.

Can it be upgraded or not?

The obvious solution to not having enough RAM is to add more RAM. A quick look in the manual showed this might not be possible, though. The manual includes the following “Memory specifications” table:

Description	Values
Memory slots	Two SODIMM slots
Memory type	DDR5
Memory speed	4800
Maximum memory configuration	32GB
Minimum memory configuration	8GB
Memory size per slot	8GB or 16GB
Memory configurations supported	8 GB, 1 x 8 GB, DDR5, 4800 MHz 16 GB, 1 x 16 GB, DDR5, 4800 MHz 16 GB, 2 x 8 GB, DDR5, 4800 MHz, dual-channel 32 GB, 2 x 16 GB, DDR5, 4800 MHz, dual-channel

That unambiguously says that an upgrade from 32GB is not possible. I gave up.

Later though, I was complaining about memory issues to a friend, and he pointed out a Dell forum thread where a couple of people claim to have successfully installed dual-channel 32GB modules. Since the alternative was getting an entire new PC after only a year, I decided to give it a go.

The upgrade attempt

I ordered a pair of Crucial 32GB DDR5-4800 SODIMMs, and after they turned up dismantled the laptop. The G15 comes apart pretty normally: there are uncovered screws on the bottom holding the lower part of the case on. With those removed and some gentle prying, it pops off, and you get access to the battery, GPU and motherboard.

The first thing I saw was this:

The G15 motherboard adamantly proclaiming that it doesn’t want 32GB SODIMMs

Not one but two labels that indicate it will only accept 8GB or 16GB modules. Oh well, what’s the worst that can happen?

Oops?

I dutifully installed the new modules, reconnected the battery and put the case back together. I pressed the power button, and… nothing. None of the usual garish lights that immediately come on, no screen output, just a dead laptop. After reading some more of the user manual, I found that there is a tiny status LED on the side next to the ethernet port. Forcing the laptop to power off and back on again, the status LED blinked a distress code at me: 2 amber blinks, 4 white blinks. The manual says that is — unsurprisingly — a memory fault.

I figured at this point that the manual and labels on the motherboard were probably right. I took the laptop apart again, reinstalled the original 2x16GB modules, reassembled it, and pressed the power button. It didn’t boot. I don’t spend a lot of time fiddling inside computers, but I’ve done it enough that I’m reasonably confident I can’t entirely break a computer while swapping some RAM modules. I took to Googling², and found an interesting article that said Dell laptops don’t like to boot after RAM changes unless you clear the CMOS by popping out the battery for 15 minutes.

I opened the laptop up, and looked around for the CMOS battery. There wasn’t one. Turns out they don’t exist any more. I left the main battery disconnected for a while to see if it would help, and it didn’t.

Unexpected success

I started to get worried: if I couldn’t fix this, I wouldn’t be able to work until I got a new PC, and that wasn’t really in my budget at the minute. I sat reading old forum threads and help guides³, none of which were actually useful. Out of nowhere, though, the laptop booted up.

Naturally, I immediately shut the laptop down again, opened it up, and switched back to the new RAM modules. Then I turned it on again and sat waiting. After about 15 minutes of it looking totally dead, it turned on and showed a BIOS warning about the hardware configuration being changed. It then booted perfectly normally, and all 64GB of RAM was visible and usable.

My theory is that the forum threads were right: Dell laptops are funny about RAM upgrades. But somehow in removing the physical CMOS battery, they’ve kept the same “you have to wait 15 minutes” behaviour just without any indication that’s what’s happening. Regardless, I now have enough RAM even for the greediest of IDEs and Electron apps.

Bizarrely, the maximum spec has decreased to 16GB since then. ↩︎
In the generic sense. I use Kagi these days. ↩︎
On my phone because, y’know, the laptop was busted. ↩︎