Chameth.com - posts like coming-around-on-llms, travel-toolkit, why-you-should-be-using-https but not debugging-beyond-the-debugger

Monthly Meanderings: August 2026

2026-09-01T00:00:00Z

Another month, another edition of “What’s Chris been up to?”. Fun fact: I very nearly published this post with a “[intro]” placeholder here instead of this paragraph, despite proofreading it multiple times. “Yep, that’s where the intro will go, all good”. Now instead we have this weirdly meta introduction.

Website updates and other projects

A few website updates this month, and one of them neatly addresses the “other projects” bit too. First, I’ve added a new fitness page which has some stats and information about my cycling and running. I’ve also added some fitness-related widgets to support that page and these monthly summaries, and a new music one you can see a bit further down.

I’ve also added what I think is quite a fun new feature: a post feed builder that lets you build RSS feeds containing only posts that are semantically similar (or dissimilar) to other posts. So if you want to only hear about my fitness journey you can pin a few of those posts, or if you want to exclude posts that talk about LLMs you can exclude a few of those. It’s basically super-charged fuzzy tags, but it means I don’t have to actually tag things.

I blogged about building a family of single-user services which is where most of my coding energy went this month. Those projects were almost entirely authored by LLMs, and I’m not hating the process. The models have got a lot better at coding, and I enjoyed getting to focus on the bigger picture aspects rather than getting bogged down in details. Rest assured, all the writing on my website is still 100% me. I am the load-bearing seam, here.

Entertainment

Back in March I said ‘I can’t do “what I listened to last month” stats until next month’, and then for the next four months completely forgot about it. Well, not any more! Now I come equipped with some stats about my monthly music habits:

	Album	Artist	Unique tracks played	Total tracks played
1	Fallen	Evanescence	8	40
2	Love, Drugs & Misery	Eva Under Fire	12	32
3	NEON VOID	Sumo Cyco	6	23
4	The Strange Case of…	Halestorm	11	21
5	Coming in Hot	DIAMANTE	9	20
6	Everest	Halestorm	7	20
7	Glory Days	Little Mix	5	20
8	From Zero	Linkin Park	9	19
9	KPop Demon Hunters (Soundtrack from the Netflix Film)	KPop Demon Hunters Cast	7	19
10	No Need to Argue	The Cranberries	6	19

I suspect I won’t show this every month, as it’s probably going to be quite same-y. I, shockingly, tend to play music that I like more than that which I don’t. I’ll usually either put on my “favourites” playlist, or my “daily mix” playlist which biases higher-rated tracks over lower ones. So effectively this just becomes a noisy view over how highly I’ve rated the tracks in an album. Maybe I can control for that, but then I’m not sure what it’d actually be showing… “Here are all the albums I played this month that I don’t usually play because I don’t like them as much as the other albums I’m not showing any more”? It’s a bit wordy. Back to the drawing board with that idea, I think.

I was away for a few weeks this month, including a few days at a small LAN event. Being at a computer gaming event meant that we, naturally, played some board games. Apparently I hit an h-index of 23 from these plays. That’s 23 games I’ve played at least 23 times each. I’ve been at 22 since December last year: obviously the higher the number goes, the harder it is to reach. I also go on and off games (and the whole hobby at times), which doesn’t help on that front. Anyway, here’s ~~Wonderwall~~ the plays for this month:

6 plays
3 plays
3 plays
1 play
1 play

No films this month, but my World of Warcraft habit has picked back up with the launch of the new season. They’ve made Mistweaver Monk, my previous specialisation, a lot less fun to play, so I’m focusing on one of my many alts: a Restoration Shaman:

Level 90

Female Dwarf

Restoration Shaman

309 average item level

Hirameth-Terenas

Professions

Midnight Blacksmithing		41/100
Midnight Inscription		72/100
Midnight Fishing		35/300

Mythic+

Altar of Fangs	+10	23:10	329
Den of Nalorakk	+10	27:36	325
Kings' Rest	+12	28:57	370
Murder Row	+12	28:37	371
Ruby Life Pools	+11	24:28	340
Temple of Sethraliss	+10	30:57	321
The Blinding Vale	+10	24:38	327
Voidscar Arena	+10	22:37	329
Total M+ rating	2711

View on blizzard.com View on raider.io

I don’t quite like it as much as Mistweaver in season 1, but it’s a lot nicer than Mistweaver in season 2. You can actually make health bars move when you press your healing buttons, rather than ignoring half of them because they’re not worth using.

Fitness

My running kick is still going. I’ve “officially” finished the couch-to-5k programme I was doing, and I actually hit 5k on the last day of the programme. I didn’t actually mean to do that: I was saving the 5k run for this week, but by the time I noticed I’d run “too far” it was close enough that I just kept going. I have an actual 5k race coming up this month, which is exciting. Once that’s over I’m planning to work my way up to 10k. I’ve added a few widgets to keep track of some stats:

activities

6h 5m

total time

103.6km

total distance

42.3km

longest distance

🏆️ New PB 1km 2:17 1:52
🏆️ New PB 2km 5:34 4:13
🏆️ New PB 5km 14:51 12:32
🏆️ New PB 10km 36:00 27:21
🏆️ New PB 20km 1:43:33 1:13:24
🏆️ New PB 30km no previous 1:54:25
🏆️ New PB 40km no previous 2:55:52

activities

6h 16m

total time

43.1km

total distance

5km

longest distance

🏆️ New PB 800m 5:56 6:02
🏆️ New PB 1km 7:35 7:36
🏆️ New PB 1 mile 12:28 12:16
🏆️ New PB 2km 15:35 15:16
🏆️ New PB 3km 24:32 23:05
🏆️ New PB 5km no previous 38:40

As well as the running, I did a couple of long (for me) cycle rides, clocking in at around 40 km. It’s a fun loop, but both times I did it I started flagging towards the end. The second time I made sure to eat and drink plenty along the way, and my legs still raised objections about halfway round. My next goal is 50 km which would let me get to the next town over and back but the route has a lot more elevation to go with its increased length, so it’s probably going to be quite a bit more taxing. I’m not sure when I’ll attempt that, as the weather is starting to turn. I was previously bemused when people mentioned “cycling seasons” and switching to indoor trainers. Then I wanted to go out when it was raining, and suddenly I understood. Running in the rain: surprisingly pleasant; cycling in the rain: no thank you.

Around the web

Hacker News In Uncompromised Detail

When I first read the title my gut reaction was “I know everything I need to know about Hacker News, right?”. The title is accurate though. There’s so much interesting stuff, presented really cleanly. It’s a good read.

A proposal for new Netflix content rating icons

Another dry sounding title, another fun article. It starts with a treatise on rating systems and ends with a lovely satirical list of rating ideas.

Find Git commits by commit message text

I like to think I’m relatively well versed in how to use Git. I can do an interactive rebase and everything! This tip from Ham was new to me, though, and is incredibly useful.

Novice Marathon Training Plan

Hal Higdon has a whole bunch of training plans. He literally wrote the book on marathon training. I’ll hopefully be following this plan next year, assuming I manage the 5k, 10k and half-marathon training! This would be the perfect plan for someone who’s run a half marathon a while ago, and is tempted to join me in my “marathon before I’m 40” goal…

Building a family of single-user services

2026-08-27T00:00:00Z

Last month I made and wrote about a personal service to track my workouts and record personal bests. Since then, things have escalated. I now have eight different services running in the same “family”:

Name	Purpose
Collegiate Room	Simple weather overview for my current location
German Mall	LLM broker for use by other services
Gloss White	Weight tracking
Olive Crosses	Central service hub
Pompei Band	Workout and personal best tracking
Speed Carp	Spaced repetition system
Tiger Island	Reading history and library tracking
Typical Horse	Location history and analytics

The names are all anagrams of something related to their function. Finding those anagrams brings me an unreasonable amount of joy. Unscrambling them is an exercise left to the reader. On top of those eight, I currently have four other service ideas sat in my todo list. It’s going to be a big family.

A postmortem on the previous hodge-podge

Most of this functionality isn’t new to me. When I started out playing around with LLMs, I created a “personal data aggregator” that just kept getting things bolted on. I said at the time:

My biggest just-for-me project is a web app that I started to help me aggregate film recommendations. It’s since morphed into a general personal data aggregation service: it deals with data from GitHub, Todoist, Letterboxd, TMDB, Healthkit, and others. It also lets me make re-orderable lists, store recipes, and more. Parts of this could definitely be open sourced, and I might carve them out at some point, but it’s mostly a glorious hodge-podge of things specific to me. Having all these services in one place lets me make quick and dirty automations, for example: when I create a Todoist note on my phone or watch, I often forget to set the due date, so it doesn’t show up in the “Today” view. It was literally a few lines of code to plumb things together so any inbox task without a due date gets set to today automatically.

In a twist that nobody could see coming, it turns out that a “glorious hodge-podge” is actually quite hard to maintain and very annoying to work with. It didn’t help that it had a fairly extensive Svelte frontend and I really am not a fan of JavaScript. I ended up using it less and less just because it wasn’t fun to use or work on, and I didn’t want to waste the limited LLM tokens on doing massive refactors or reworks.

I’ve got almost two decades of professional software development experience under my belt, so you’d think I would have learnt the lessons on how to write maintainable software. Hell, my CV even mentions that as something I’m particularly good at. So what went wrong?

There were a few factors. One of the obvious things to blame is that it was written mostly with LLMs. Most of that blame isn’t very honest, though. In the year or so since, LLMs have definitely got better at writing good code; but I was manually reviewing all the code at the time, and telling it quite carefully what to implement and how to do it. So the buck really stops with me.

I think it’s actually more because it was my first larger project using LLMs. It’s a bit like learning a new programming language. I don’t think the first sizable project I’ve written in any language has ever been any good. You can read and learn as much as you want, but you need to actually run into some problems headfirst to get a grip on how the language works. Or at least I do. LLM coding agents are basically the same deal.

What the LLMs actually did was mask several problems. The architecture wasn’t right for how the service developed. It started out as a single-purpose app and gradually got things bolted on, without any major changes. Everything ended up in horizontal slices: a massive database layer with every query used by every part of the app, a massive HTTP layer, and so on. Working on a single feature became a tap dance all over the source tree. This sort of friction is normally quite apparent when you’re writing code by hand, and at some point it hurts enough that you throw in the towel and do some refactoring. But the LLM will happily plod on without exposing that pain, so the project needed a more active kind of architecture management that it just didn’t get.

The other problem they covered up was how brittle and unmaintainable the JavaScript frontend was. I’m not even sure how I ended up with a Svelte frontend in hindsight. It’s my preferred framework, but I still only reach for it as a last resort when there’s no other way to avoid using one. It seems like there were lots of ways to avoid it here, but I seem to have… well… avoided them. Anyway, each feature got its own slew of Svelte components, without much thought to reusability, shared patterns, or anything sensible. Some of them were just thin wrappers around data supplied by Go, others had vast swathes of logic in them, with no real rhyme or reason as to which way that would fall. It’s the same issue as before: a human would look at it, say “WTF?” and suggest tidying it up; instead, the LLM just gets on with what you tell it to do.

There’s also a big non-LLM factor: one of the primary reasons all of this unrelated functionality got tossed into the same app was that I’d previously worked on a few projects with a microservice architecture and really didn’t like it. My conclusion was they only really worked in large environments where you can dedicate people or teams to working on all the boilerplate that happens from maintaining lots of interconnected projects. Dumping everything and the kitchen sink into one badly architected monolith was, perhaps, not the best rebound from that.

Pompei Band and beyond

When I wanted to implement ~~Strava at home~~ my PB tracking app, it seemed fairly natural to build it out as a new service, rather than bolt it on to the slowly decomposing hodge-podge that was giving off a funny smell. Then when I wanted a weather dashboard, why not do it in the same style as Pompei Band? The same serving architecture, the same page layout… I could even come up with a similar weird two-word name!

Then wouldn’t it be nice if the weather dashboard knew where I was, so it could show the local weather? I could just build that functionality in, but it’d also be a good opportunity to carve out the location history feature from my old service and make a new, shinier one. Then basically every time I even thought about going near the old service, I mentally sketched out a replacement service.

Olive Crosses showing links to the other services

At one point I had the thought that it’d be nice if the services could all link to one another, but I didn’t want to have to touch every service whenever I added a new one. So I made a “hub” service, which maintains the One True Catalogue, and provides a bit of JavaScript the others can all embed to add a navigation sidebar.

So after complaining about microservices and JavaScript, it sounds like I have landed on microservices (or at least miniservices) and JavaScript? Sort of yes, but sort of no. Each service is still its own thing and makes sense independent of the others. There are some cross-service dependencies, but they’re very simple and don’t have the same cognitive or boilerplate overhead as a login service that talks to an authentication service that queries a user manager and so on. And as for the JavaScript, it’s almost exclusively used for progressive enhancement and a few limited client-side visual things like maps and graphs. All of the logic lives in Go, which is a lot easier to maintain and test, and all of the pages are simple Go templates.

Having these services built in a way that I actually like has motivated me to play around a lot more with new features:

My workout tracking app pulls photos from my private Immich server that were taken during workouts, and shows them alongside. Now I can see all my obligatory end-of-run selfies in one place.
The location history app can detect trips away from home, and get an LLM to automatically title them. My most recent, very exotic, trip has the title “A fortnight in Milton Keynes and Coventry”. Each trip gets its own little map, and in the future I want to pull in images like with workouts.
My weight tracking app now exports weigh-ins to Garmin, as it doesn’t fully integrate with Apple Health (it will export its own weights, but not import others, like the ones from my Withings scales). Stupid walled gardens, but thankfully I can fix it myself.
My book tracking app automatically cleans up the titles of books from the nonsense provided by Amazon to something more palatable. “Killing Eve: Long Shot: A twist on obsession, betrayal and revenge from Luke Jennings, author of the TV sensation Killing Eve” becomes “Long Shot (Killing Eve #5)”. It does this by passing the title to an LLM with a bunch of rules.
The hub generates me a snippet of Tailscale configuration that defines the minimal grants needed to allow the intra-service dependencies to work.

The LLM integrations were inspired by my friend Shane who recently demonstrated an inventory tracking app he’d made, which used Haiku to process images of boxes and describe the contents. I’ve dabbled with LLM integration before, and always disliked the process of fiddling with the prompts, and debugging when the LLM gave a nonsensical response. So I built German Mall, which makes it a lot easier. It lets me define a “workflow” consisting of a system prompt, initial message, and expected response format. I can browse past runs, tweak the prompt, rerun them to see the change, all without touching the services using them. Quick iteration loops are the best. German Mall also provides markdown documentation for each workflow, so integrating it downstream is simply a case of pointing a coding agent at the right documentation endpoint.

The “tidy book titles” workflow in German Mall

It’s not all fun and games: having a bunch of services all following the same pattern means you need to do cross-cutting maintenance on them now and then. But that’s surprisingly easy with LLMs. I put all the projects in a common folder, and I launch cross-cutting tasks from that folder. Usually I’ll make the change in one project, and then the cross-cutting task is simply “Look at commit XYZ in ./foo, make the same change to the other projects”.

I’ve also ended up with a shared library used by all the projects to cut down on some boilerplate. That’s normally something I’d resist, especially with personal projects, just because the extra overhead of working in a separate project, managing releases of it, and so on, is a bit annoying. But, again, LLMs can do almost all of that grunt work. They effectively reduce the “cost” of maintaining separate services to a negligible level, leaving me to focus on the more fun aspects like coming up with new names.

Monthly Meanderings: January 2026

2026-02-01T00:00:00Z

Welcome to the second edition of my monthly meanderings. For a bit of context, you can check out the introduction to the first edition.

Website updates

It’s been a pretty busy month for chameth.com. Three blog posts: The Meaning of Life — an entry into the IndieWeb carnival where I mostly review a book on Stoicism — Surge Protectors: Marketing vs Reality which is a dump of a research rabbit hole I fell into, and An Interesting Tailscale + Docker Gotcha which documents a fun problem I encountered.

I’ve also made a new films section, which is basically my Letterboxd data but free range instead of cooped up in a silo. I imported my 200-odd reviews, and for the last few I wrote the review on my site and then copied it over to Letterboxd. That’s what the IndieWeb folks call POSSE: publish on your own site, syndicate elsewhere. I like the idea that my website is the canonical source of truth for all the public things I generate. I want to expand this to some other areas in the future: music, video games, board games, etc.

I made most of the film components into “shortcodes”, so I can drop them into arbitrary pages or posts, like so:

Watched films ranked

Every film I’ve watched since I started logging, ranked.

Obviously super subjective, and subject to change often and arbitrarily.

264 films

Finally, I added a new widget to the bottom of most pages: a “nod” button. It’s effectively a “like” or “kudos” button but deliberately does not track how many times it’s been pressed, has no social network connotations, and won’t flood my inbox or hit rate limits if it gets spammed. If you click the button, it simply sends a message to a private IRC channel that says “Someone nodded at $URL”. I’ll probably write a bit more about it in a blog post in the future.

Other projects

I only have one small non-chameth.com update: a new project called pigeonhole which provides a really basic web service that lets you leave messages in a named cubby, and retrieve them at a later date. I’m using it to automate adding things to my Obsidian notes from my Apple Watch. There’s lots of reasons why that isn’t as simple as it sounds. I went through a bunch of iterations to try and make it work before just writing code to fix it.

I wanted to do this because I’m trying to replace Todoist with just having tasks in Obsidian. I already used Obsidian for longer form notes, but I’ve now also started using it for drafting blog posts (I’m typing this in Obsidian right now!), and trying to keep more useful day-to-day notes. I’ll probably write a blog post with more details once I’ve figured everything out.

Finally, I’ve been moving a bunch of projects to a private Forgejo instance, with the ultimate goal of having all of my projects live there. I’m going to mirror the contents out to GitHub and maybe other places, but Forgejo will be the single source of truth, and where all the workflows/dependency wrangling/etc happens. I’ve got Renovate configured to do dependency bumps, and I’m very much liking the whole setup so far. I’m saying it a lot but again: I’ll blog about it in more detail when it’s a bit further along.

Entertainment

I mentioned last month that I was playing Hogwarts: Legacy. I finished that off, and have been struggling to find something to scratch the same itch. I’ve tried a whole bunch of open world games, and basically none of them work for me. They have cumbersome control schemes (yay console ports), really awkward or sluggish movement, and so on. I think I’m just too fussy, but if I’m going to sink dozens of hours into a game it shouldn’t feel annoying or like a chore!

I did play through Thank Goodness You’re Here, which is delightfully funny but sadly only took 3.6 hours to finish with 100% of the achievements. Still highly recommend picking it up, especially when it’s on sale for under ‘10 bob’. I’ve also started playing through Control, and I’m enjoying the ambience. There’s an insanely sharp difficulty spike for the first boss, though, which nearly made me put it down.

In other entertainment, I’ve been playing a load of turn-based games on Board Game Arena , and I’ve watched a bunch of films after barely watching any last month. Here’s another of those shortcodes to show them off:

Around the web

Remotely unlocking an encrypted hard disk

A fun post documenting using Tailscale in a very weird way to solve a relatable problem. Very much my cup of tea.

I charged $18,000 for a static HTML page

This one’s actually from 2019, but it’s the first time I’ve come across it, and it’s a fun story. As a contractor, deciding what to bill for and what not to can be a bit messy. Downtime between meetings? The ten minutes in the shower you spent thinking through a bug? It’s a lot more clear cut if the client wants you to sit in an office, twiddling your thumbs, though.

Oxide RFD: Using LLMs at Oxide

A wonderfully well thought out piece about using LLMs in a respectful, sensible manner. I’d been thinking about writing something similar, but it wouldn’t have been anywhere near as well crafted as this. While I basically agree with it all, the part about LLM-generated text really resonates:

LLM-generated prose undermines a social contract of sorts: absent LLMs, it is presumed that of the reader and the writer, it is the writer that has undertaken the greater intellectual exertion.

Your app subscription is now my weekend project

Roberto points out that monthly SaaS subscriptions start looking even less appealing when you can just prompt an alternative into existence. I talked a bit about using LLMs for one-person software before, and it becomes even more appealing when the alternative is paying monthly for a service that keeps shoe-horning in features you don’t want, making you agree to share data with 329 “trusted partners”, and/or repeatedly ratcheting up the price.

Ekul’s Internet Space

I love the aesthetics of this personal site, and the data tracking that goes along with it. I’m also slightly aghast at the amount of Monster consumed! Makes me want to track more things and expose them on my own site.

Finding an awkward bug with Claude Code

2025-06-30T00:00:00Z

The many times Claude thought it was right…

I recently encountered a bug in one of my projects that I couldn’t immediately figure out. It was an issue in Centauri, my reverse proxy. After its config was updated, I noticed¹ it stopped serving responses. Looking at the logs, I could see it was obtaining new certificates from Let’s Encrypt for a couple of domains, but I’d designed it so that wouldn’t block requests (or so I thought). After a few minutes of confusion, everything went back to working.

It felt like an issue with locking, but a quick glance didn’t show me anything obvious. I decided to throw the problem at Claude Code and see how it did. With some guidance² it managed to identify the cause, but it was a bit of a journey to get there. I’m going to share the prompts and responses, as it was an interesting process.

The initial prompt

I thought the issue might be quite obvious, and I’d just overlooked it. Centauri isn’t that complex a project, after all. So I gave Claude a fairly minimal prompt to set it off:

> Hello! Please examine this codebase. I have noticed an issue that when a
  certificate is being renewed, the application seems to not serve any other
  requests. I suspect there is a misplaced lock somewhere.
 
  Can you see what's happening? Think very hard.

Claude Code has special hardcoded triggers to engage “thinking” mode, which makes it take more time to think about alternatives. The keywords are: “think”, “think hard”, “think harder”, and “ultrathink”. I managed to not quite get the right incantation here, and only enabled the lowest level³.

I’ll omit all the responses where Claude is “thinking” (talking to itself), or decides to make todo lists then cross items off them then add different items, and so on. Its responses get fairly lengthy; I won’t judge you too harshly if you skim over them. Its first response was blessedly brief:

● I found the issue! Looking at the certificate manager code, I can see the
  problem:

… and then it went back to thinking and reading files, because it had neither found the issue nor could it see the problem. Ho-hum. Some time later it came back with:

● I found the issue! The problem is in certificate/manager.go:53-54:
  
  m.store.LockCertificate(subject, altNames)
  defer m.store.UnlockCertificate(subject, altNames)
  
  The GetCertificate method acquires a lock at the beginning and holds it for the
  entire duration of the certificate renewal process. This includes:
  
  1. Contacting the ACME server
  2. Solving DNS-01 challenges (can take minutes for DNS propagation)
  3. Obtaining the certificate
  4. Getting OCSP staple
  5. Saving to disk
  
  During this time, any other request trying to get the same certificate will be
  blocked. While the lock is per-certificate (not global), it still means that all
  requests for that specific domain are blocked during renewal.
  
  Let me check if there are other places where this might cause blocking:

And it then proceeded to do even more thinking, checking files, etc, until it concluded with:

● I found the root cause! The issue is actually worse than just a
  per-certificate lock. Here's what's happening:
  
  The main blocking issue is in proxy/manager.go:133-144:
   
  func (m *Manager) CheckCertificates() {
      m.lock.RLock()  // ← Acquires read lock on the entire manager
      for i := range m.routes {
          route := m.routes[i]
          // ...
          m.updateCert(route)  // ← This can take MINUTES for ACME renewal!
      }
      m.lock.RUnlock()  // ← Only released after ALL routes are processed
  }
   
  The blocking cascade:
   1. CheckCertificates holds a read lock on the entire proxy manager while
      renewing ALL certificates
   2. Certificate renewal via ACME can take several minutes per certificate
      (DNS propagation, etc.)
   3. Every incoming request calls routeFor (proxy/manager.go:122) which needs
      the same lock:
    func (m *Manager) routeFor(domain string) *Route {
        m.lock.RLock()  // ← BLOCKED until CheckCertificates completes!
        match := m.domains[strings.ToLower(domain)]
        m.lock.RUnlock()
        // ...
    }
  
  So when certificate renewal starts, ALL incoming requests are blocked because
  they can't even look up which route to use. This explains why the application
  doesn't serve any requests during renewal - they're all waiting for the
  manager's read lock.
  
  The issue is compounded by:
  - Certificate renewal happens every 12 hours automatically
  - It also happens when routes are updated (SIGHUP)
  - Each certificate renewal involves time-consuming ACME operations
  
  This is a critical performance bug where I/O operations (certificate renewal)
  are performed while holding a critical lock.

This doesn’t actually follow on from the earlier response, it’s completely flipped onto a different lock being the issue. Amusingly, it’s nearly right here, but it’s highlighting this code for completely the wrong reason.

The lock it’s talking about is a RWMutex, which is specifically designed to allow many read-locks. Everything it’s pointing out are read locks, so they cannot block one another as it described. The response Claude has given sounds very confident, seems appropriately technical, but is in fact complete rubbish. This is one of the big issue with all LLMs at the minute, and it makes me despair at the idea of people “vibe coding” things when they don’t know enough to push back against nonsense like this.

Course correction

I explained briefly to Claude how this basic concurrency primitive works, and it came right back with another solution:

> But that's a RW Mutex. You can hold many read locks concurrently.

● You're absolutely right! I apologize for the confusion. With a RWMutex,
  multiple readers can hold the lock concurrently. Let me trace through this
  more carefully.
  
● Task(Analyze updateCert method)
  ⎿ Done (8 tool uses · 20.0k tokens · 1m 11.9s)
  
● Excellent analysis! You're right about RWMutex allowing concurrent readers.
  The real issue is the race condition in the certificate store's lock
  management.

  Looking at certificate/store.go:82-91:

  func (j *JsonStore) lockFor(subjectName string, altNames []string) *sync.Mutex {
      key := strings.Join(append([]string{subjectName}, altNames...), ";")

      if mu, ok := j.locks[key]; ok {
          return mu
      } else {
          mu = &sync.Mutex{}
          j.locks[key] = mu  // ← Concurrent map write!
          return mu
      }
  }

  This is a classic concurrent map access bug. When multiple requests hit the
  same domain during certificate operations:

  1. Multiple goroutines call lockFor() simultaneously
  2. They read/write to j.locks map without synchronization
  3. Go runtime can:
    - Panic with "concurrent map writes"
    - Corrupt the map structure
    - Block indefinitely if map internals get corrupted

  This explains the blocking behavior - it's not the RWMutex, but rather a race
  condition causing either panics or deadlocks when the map gets corrupted
  during concurrent certificate operations.

  The fix would be to either:
  1. Add a mutex to protect j.locks map access
  2. Use sync.Map instead of a regular map
  3. Pre-initialize all locks at startup

I do appreciate when me stating a basic fact is called out as “excellent analysis”. Definitely doesn’t make me feel like I’m having smoke blown up my ass.

I think the issue it’s describing here is actually a problem⁴, albeit one that’s unlikely to be hit. But it’s definitely not the one we’re looking for. Claude even describes the possible consequences of the bug, and they don’t match what happened at all.

Maybe logs will help?

While Claude was chugging away, I’d been looking into the issue as well. I had logs from when it happened, and figured Claude may be able to glean something from them too. The first thing I’d noticed is that I wasn’t logging when a process finished, so I’d already added that to the code base, hence the note so Claude doesn’t get itself confused:

> That may be an issue, but it's not the one I was looking for. I've found logs
  from the time it happened. Note that a few logs have been added to the
  codebase since (so we can't tell from the logs when ACME or OCSP processes
  *finished*, just when they started). Here are the logs:
  
  {"time":"22:40:53.187887976Z","level":"INFO","msg":"Received signal, updating routes...","signal":1}
  {"time":"22:40:53.19947532Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:40:53.199507942Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:41:09.048465968Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:41:09.048482309Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:41:17.419299795Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:41:17.419325744Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:41:23.238027386Z","level":"INFO","msg":"Received signal, updating routes...","signal":1}
  {"time":"22:41:29.994330646Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:41:29.994353329Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:41:43.658794811Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:41:43.658819548Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:41:57.635325513Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:41:57.635347515Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:42:10.52439779Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:42:10.524417107Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:42:23.159430249Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:42:23.159452171Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:42:36.347407326Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:42:36.347433295Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:42:52.544628775Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:42:52.544653993Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:43:25.409579107Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:43:25.409602842Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:43:43.819479448Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:43:43.819502041Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:43:54.358957563Z","level":"INFO","msg":"Renewing certificate", ...}
  {"time":"22:43:54.358979355Z","level":"INFO","msg":"Starting ACME process to obtain certificate", ...}
  {"time":"22:44:06.180278128Z","level":"INFO","msg":"Renewing certificate", <big list of alt names>}
  {"time":"22:44:06.180302585Z","level":"INFO","msg":"Starting ACME process to obtain certificate", <big list of alt names>}
  {"time":"22:45:03.830331922Z","level":"WARN","msg":"Failed to connect to upstream","error":"context canceled", ...}
  {"time":"22:45:03.830528134Z","level":"WARN","msg":"Failed to connect to upstream","error":"context canceled", ...}
  {"time":"22:45:03.832388931Z","level":"WARN","msg":"Failed to connect to upstream","error":"context canceled", ...}
  {"time":"22:46:10.225854865Z","level":"WARN","msg":"Failed to connect to upstream","error":"context canceled", ...}
  {"time":"22:46:10.488432434Z","level":"WARN","msg":"Failed to connect to upstream","error":"context canceled", ...}
  {"time":"22:46:10.488439397Z","level":"WARN","msg":"Failed to connect to upstream","error":"context canceled", ...}
  {"time":"22:46:30.617614536Z","level":"WARN","msg":"Failed to connect to upstream","error":"context canceled", ...}

I’ve cut down the logs a bit to remove the specific domains, simplify the timestamps, and remove a bunch of other cruft. The key points are there were two SIGHUPs received, and the last certificate before everything started breaking covered 6 different hostnames.

Claude then went through a phase of periodically shouting “Eureka!” then deciding it hadn’t actually found anything yet:

● Now I see it! The key insight is in the timing and the "context canceled"
  errors. Let me trace through what's happening:
  
● Ah, now I see what's happening! Looking at the logs more carefully:

  1. 22:40:53 - First SIGHUP received, routes updated
  2. 22:40:53 to 22:44:06 - Multiple certificates start renewing (14 certificates!)
  3. 22:41:23 - Another SIGHUP received while renewals are still in progress
  4. 22:45:03 onwards - "context canceled" errors start appearing

  The issue is that SetRoutes is being called AGAIN (at 22:41:23) while the
  previous CheckCertificates() goroutine is still running and renewing
  certificates. Let me check what happens in this scenario:
  
> (The context canceled errors are clients timing out because centauri wasn't
  proxying properly)
  
● Ah, now I understand! The "context canceled" errors are from clients timing
  out because Centauri wasn't proxying their requests. This confirms the
  blocking issue.

  Let me check how incoming requests actually get their certificates:
  
● Now I found the real issue! Looking at the timeline and the code:

  1. 22:40:53 - SIGHUP triggers SetRoutes() which:
    - Calls loadCertificate() for each route
    - Sets route.certificate to the existing certificate
    - Starts go m.CheckCertificates() in background
  2. 22:41:23 - Another SIGHUP while certificates are still renewing
    - SetRoutes() is called AGAIN
    - Creates NEW Route objects with route.certificate = nil initially
    - Replaces the old routes in m.routes
    - The old Route objects (with valid certificates) are replaced!

  The critical issue is in proxy/manager.go:42-65:

  func (m *Manager) SetRoutes(newRoutes []*Route, fallback *Route) error {
      // ... validation ...

      for i := range newRoutes {
          route := newRoutes[i]  // These are NEW Route objects!
          // ...
          m.loadCertificate(route)  // Try to load certificate
      }

      m.lock.Lock()
      m.routes = newRoutes  // REPLACES all routes!
      m.lock.Unlock()
  }

  When the second SIGHUP arrives:
  1. New Route objects are created
  2. For domains currently being renewed, GetExistingCertificate might fail or
     return expired certs
  3. These routes get route.certificate = nil and CertificateMissing status
  4. CertificateForClient returns nil for these domains
  5. TLS handshake fails → clients can't connect → "context canceled" errors

  The fix would be to either:
  1. Prevent concurrent CheckCertificates runs
  2. Copy certificates from existing routes before replacing them
  3. Make certificate renewal truly independent of route management

Midway through it had started searching for “context canceled” errors in the codebase, which doesn’t make much sense. That’s what happens when a client gives up waiting, so the corresponding backend request is terminated. You won’t actually find “context canceled” anywhere in Centauri, because it’s a feature of the Go standard library. I gave it a little nudge to try to stop it rabbit holing.

As far as I can tell, what it’s talking about in this response isn’t an issue. I guess it’s possible that there’s a race condition whereby a route would be replaced and end up with an old version of a certificate that was just being renewed, but it wouldn’t cause an issue in practice; it’d just get updated next time the certificates were refreshed.

There’s no way for routes to get a nil certificate in this scenario, as far as I know. But I don’t have the unfounded confidence of an LLM, so I simply said:

> I don't follow why the certificate would be nil?

Attempt the third

I expected Claude to just explain its reasoning, but it apparently didn’t have any reasoning so set off on another voyage of discovery:

● You're right to question that. Let me trace through this more carefully.

● Good point! Since GetExistingCertificate doesn't acquire locks, it should
  return the existing certificate even during renewal. Let me dig deeper:
  
● Now I see a potential race condition! Here's what could happen:

● I found it! There's a critical race condition that explains the blocking:

  The Race Condition

  1. Background renewal starts: CheckCertificates() → updateCert() →
     GetCertificate() acquires lock and starts ACME renewal
  2. SIGHUP arrives: SetRoutes() is called, which for each route calls
     loadCertificate() → GetExistingCertificate() → store.GetCertificate()
  3. The race:
    - store.GetCertificate() returns a pointer to the certificate in the slice
    - Meanwhile, the renewal completes and calls SaveCertificate()
    - SaveCertificate() calls removeCertificate() which modifies the slice while
      it's being read
    - This slice modification (append(j.certificates[:i], j.certificates[i+1:]...))
      can cause:
        - The pointer returned by GetCertificate to become invalid
        - Slice bounds issues
        - Memory corruption
  4. The result:
    - GetExistingCertificate might return corrupted data or nil
    - Routes end up with certificate = nil
    - TLS handshakes fail
    - Clients timeout → "context canceled"

  The Timeline Confirms This

  - 22:40:53 - First SIGHUP, starts renewal of 14 certificates
  - 22:41:23 - Second SIGHUP arrives (30 seconds later) while renewals are still running
  - 22:45:03 - First timeout errors (4 minutes after first SIGHUP)

  The 4-minute delay matches ACME renewal time. The second SIGHUP's SetRoutes
  likely read certificates while they were being modified by the first SIGHUP's
  renewal goroutines.

  This is a classic concurrent slice modification bug - no synchronization
  between readers (GetCertificate) and writers (SaveCertificate/removeCertificate).

Once again we have a very confident and technical answer that does not explain the symptoms we’re looking for. It’s also mostly rubbish again. GetCertificate and SaveCertificate are only called by code that is holding a lock for that certificate; they won’t interfere with one another. There may, again, be a small concurrency issue if different certificates are handled in parallel, but it’s definitely not the cause of our current problems.⁵

Working through these explanations and debunking them was annoying me at this point. I decided to give it once last chance, and used the “ultrathink” shibboleth:

> So there may be a concurrent modification issue, but that doesn't explain the
  symptoms. If a TLS handshake failed it wouldn't result in the clients timing
  out. Likewise if there was any kind of corruption there'd be a panic, not a
  period where it didn't work and then started working again. Engage ultrathink.

Fourth time lucky

I’m going to include a piece of the thinking here because it amused me so much:

✻ Thinking…

  Oh! I need to think about this differently. Let me trace through what happens
  when a request comes in:

Apparently it takes “ultrathink” to actually work through the problem logically? Amusingly it did actually get there:

● You're absolutely right. Let me think more carefully about this...

● EXACTLY! That's the issue. Looking at the timeline:

  1. 22:40:53 - First SIGHUP: CheckCertificates() acquires READ lock, starts
     renewing 14 certificates
  2. 22:41:23 - Second SIGHUP: SetRoutes() tries to acquire write lock, blocks
     waiting
  3. Incoming requests call routeFor() for route lookup, but Go's RWMutex blocks
     new readers when a writer is waiting (to prevent writer starvation)
  4. 22:45:03 - Clients timeout because requests can't even get past route lookup

  The key insight is Go's RWMutex writer starvation prevention: when a writer
  is waiting, new readers are blocked even though the current lock holder is
  also a reader.

  So the cascade is:
  - CheckCertificates (read lock) → certificate renewal takes minutes
  - SetRoutes (write lock) → waits for read lock to release
  - routeFor (read lock for every request) → blocked by waiting writer

  Result: All incoming HTTP requests hang at route lookup, never reach TLS
  handshake or proxying logic. Clients timeout waiting for any response
  → "context canceled".

  This perfectly explains:
  - No panics (no corruption)
  - No immediate TLS errors (never gets that far)
  - Requests hang then timeout (blocked on lock)
  - Eventually recovers (when certificate renewals finish)

  The fix would be restructuring to avoid holding the manager lock during slow
  certificate operations.

We’re back to the RWMutex again, but this time it’s dropping knowledge on me not the other way around: if there’s a writer waiting, then new readers are blocked. This does indeed perfectly explain the symptoms, and was something I’d overlooked. The write lock is only held briefly to update some struct members to prevent it from blocking the readers, but this behaviour coupled with the long-running read lock ends up doing that anyway.

The fix

After all this work it’s very tempting just to ask Claude to fix it, but I had almost no confidence it’d be able to maintain any of the concurrency requirements given how much it flailed around with bad explanations. But in the interests of science, I asked it to try. Here’s what it came up with:

   // CheckCertificates checks and updates the certificates required for registered routes.
   // It should be called periodically to renew certificates and obtain new OCSP staples.
   func (m *Manager) CheckCertificates() {
+    // Create a snapshot of routes to avoid holding the lock during slow certificate operations
     m.lock.RLock()
+    routes := make([]*Route, len(m.routes))
+    copy(routes, m.routes)
+    provider := m.provider
+    m.lock.RUnlock()

+    // Process certificate renewals without holding the manager lock
-    for i := range m.routes {
+    for i := range routes {
-      route := m.routes[i]
+      route := routes[i]
 
       if provider == nil {
         route.certificateStatus = CertificateNotRequired
       } else {
         m.updateCert(route)
       }
     }
-    m.lock.RUnlock()
   }

   // updateCert updates the certificate for the given route.

That’s about what I expect. It definitely fixes the exact issue described, but it’s a bit… sloppy. Routes are now being updated when the lock isn’t being held, which is a change in semantics. It also doesn’t address the underlying issue that requests can be blocked if there’s a pending writer, and pending writes can be blocked by readers. A better approach would be to allow these operations to happen without blocking, maybe by using an atomic pointer to an immutable slice. I guess I’ve got some coding to do!

As for Claude: while the journey was frustrating, I think trying to figure this out by myself would have been even more so. It’s one of those things where everything looks right to me, and I’d have to go down a fairly long chain of checking my assumptions to find where I’d gone wrong.

There’s definitely a lot of room for improvement in coding LLMs: being able to estimate their own confidence level instead of stating everything as absolute truth would go a long way. I imagine there’s also a lot of benefit of having one LLM vet the responses of another; that’s sort-of how thinking mode works, but it doesn’t seem to go far enough to stop it outputting rubbish. It’ll be interesting to see how they develop in the future, anyway.

read as “I got spammed by a bunch of monitoring notifications” ↩︎
read as “Me repeatedly telling it that it was talking rubbish” ↩︎
If only we had some kind of language model technology that could understand this kind of thing. ↩︎
I sense some fun times with Go’s race detector in my near future ↩︎
The most likely outcome from these concurrency issues is Centauri panicking, which isn’t great but would only interrupt ongoing connections before it restarts and starts serving traffic again. Much less problematic than the “block for five minutes” behaviour I’m trying to fix. ↩︎

The Ethics of LLMs

2025-06-22T00:00:00Z

I’ve written about LLMs a few times recently, carefully dodging the issue of ethics each time. I didn’t want to bog down the other posts with it, and I wanted some time to think over the issues. Now I’ve had time to think, it’s time to remove my head from the sand. There are a lot of different angles to consider, and a lot of it is more nuanced than is often presented. It’s not all doom and gloom, and it’s also not the most amazing thing since sliced bread. Who would have thought?

It’s worth noting that I’m just setting out my position here. I’m not trying to convince you to change your mind. To set the scene a bit, I mainly use Claude Code as a programming tool. I use the Claude chat interface sometimes to proofread things, do random one-off data analysis, or help organise things. More rarely I’ll try to use it to brainstorm things, or recommend things, or do more “creative” things, but I don’t trust it enough in those domains to do it often. I don’t use it for research or as a Google replacement¹, which I recognise probably makes me a weird half-in-the-water, half-out-of-it class of user.

Copyright & Corporate Control

One of the key issues, and something that is being prosecuted in several court cases right now, is how LLMs interact with the copyright system. And by “interact with” I mean “run roughshod all over”. It seems pretty obvious from my lay perspective that if having 10 seconds of pop music in the background of a YouTube video is copyright infringement, then Meta pirating books via BitTorrent must also be.

That said, the entire copyright system as it exists now is fundamentally broken. I don’t mind the idea of individual creators having protection, but I absolutely hate how copyright is wielded as a blunt instrument by massive corporations to intimidate individuals and to try to earn more money. If a law enables a huge conglomerate to sue the author of an open source project for a theoretical 75 trillion dollars then that is a bad law. Copyright today has been twisted well beyond what it was meant to be originally, thanks to lobbying by people and businesses with lots of money that want to keep it.

So if copyright law is not fit for purpose, are LLMs fine? Well, not quite. The argument that LLM training is a bit like a child learning to read holds some merit, but it falls flat when LLMs can regurgitate the input material verbatim². Personally I don’t care if they steal all the content of the New York Times and all the Disney characters that are still in copyright. It’s the individual harm that bothers me: the struggling author whose words are slurped up by the LLMs and regurgitated, potentially costing them readers; the open source dev who chooses a copyleft licence on their work, only for an LLM to dump complete copies of functions in someone else’s codebase with no attribution or licence, stopping any future improvements from being released for the public good.

I can mostly mitigate those concerns in my personal use. When coding I generally direct the agent closely enough that the opportunity to drop in massive chunks of someone else’s code is negligible. I don’t use LLMs to generate big chunks of written content. Overall I’m reasonably happy that I’m not making anyone worse off.

One related area that doesn’t affect me much because of my limited use, but does concern me a lot, is corporate control. Most of the frontier LLMs are built by big tech companies. As more and more people are using LLMs, they become an avenue for control over information. It’s clear the people in charge should not be trusted with that responsibility, as Grok’s whole Boer War thing demonstrated. I’m not sure what the answer is, though. Would a government built LLM be any better? Probably not. The requirements to actually do the training are so high that the only people who can do it are the ones you wouldn’t want to.

So I’m left in the uncomfortable position of using tools by companies I don’t trust, trained on data they shouldn’t have been trained on. Still, there can’t be many more ethical issues, right?

Forced Features & Flattery

Before we get onto the more obvious ethical topics, let’s go on a brief detour so I can complain about how LLMs are being crowbarred into seemingly EVERY SINGLE SERVICE. I don’t want an LLM in my text editor. Or when shopping. Or taking screenshots of everything my computer does every few seconds. I don’t want LLMs anywhere outside a clearly-defined LLM box. With these forced integrations, you’re even more in the dark about what their prompts are, what information they have access to, what underlying model they’re using, and so on.

There’s no good reason for 90% of these implementations; there’s just some weird Silicon Valley hype train that everyone is scared to miss out on. And as a result we get more and more screen real estate taken up with things with animated gradients that can’t be turned off and are prompted to talk in an insufferable manner.

The problem isn’t just that they’re everywhere, though, it’s how they behave when they’re there. The overly sycophantic behaviour that most LLMs adopt can hook users in the same way a social media app can fiddle with its algorithms to encourage people to keep scrolling. It’s a lot harder to dismiss an LLM when it’s blowing smoke up your ass. Now I don’t believe this behaviour is actually intentionally nefarious, but more of an artifact of the models overfitting on user approval instead of something useful like accuracy.

The combination of going too far to please the user and hallucinating things is incredibly problematic. How are ordinary, non-tech savvy, people meant to deal with these models being forced in front of them? If you ask a question and get back a flattering answer that’s roughly the shape of what you were expecting, would you question it? Would a student, who was doing their homework in Google Docs when an LLM inserted itself into their lives question it?

This isn’t just a problem for naive users, either: I occasionally ask Claude Code “Can we do X?” and discover much later that the answer should either be an emphatic “NO”, or at least have a page of caveats attached. But the LLM wouldn’t want to displease me, so it tries to do what’s asked of it, with all kinds of weird hacks, and ultimately gets itself tied up in knots. This is not how a tool should behave. If I wanted someone to say “Yes, and” repeatedly I’d join an improv troupe.

Environmental Expenses

The biggest ethical issue with LLMs is their environmental impact. Are we adding more fuel to the dumpster fire that is our planet, just so my phone can inaccurately summarise news headlines for me? The answer is a resounding “maybe”. The companies making the LLM models are cagey about how much energy goes into making them and then answering queries. There are estimates for the latter, but there are several orders of magnitude between the most optimistic and most pessimistic. LLMs are definitely not good for the environment. Basically no computer activity is³. But you don’t see people campaigning to shutter YouTube (for example) because of its environmental impact. I struggle to see the difference.

I’m not saying considering the environmental impact isn’t important. We are killing the planet. We are not doing enough about it. More attention to the climate crisis in general is a good thing. But even with the most pessimistic estimates on power usage, my personal LLM usage is a drop in the ocean. I can have far more impact by reducing the amount I travel, avoiding red meat, and so on.

One issue I do have, though, is the forced integrations I mentioned. I’m happy with my personal usage. I’m not happy that a whole slew of LLMs are being operated on my behalf. How much power is wasted by Google generating LLM results in every search results page, versus how much benefit it provides? How do people know if a search box will just use a few Watts doing a database search, or burn kilowatts⁴ spinning up an LLM and filling its context window?

Then there’s the issue with training. Who knows what resources go into training these models? Whether the amortised cost of that across all its users is worth it? I think the answer to the first question is “only the tech companies making the models”, and their utter silence on the second question says volumes.

But this kind of industrial power isn’t unique to LLMs. There are many industries that do far worse. And that’s not a justification, but I think it makes sense to look at them more holistically. In an ideal world, where we have functioning governments that listen to scientists⁵, we could address all of this with legislation on emissions and clean power. Then all of these industries will either manage to function in a manner compatible with keeping the planet alive, or will have to stop. I appreciate that actually legislating that is about as likely as waking up one day to find Claude has become an AGI, though.

Slop & Survival

We’re nearly done, I promise. You can see why I didn’t want to try to tackle these issues in an earlier post! The last big issue I want to talk about is slop. It feels like it’s everywhere, and still getting worse.

The Internet has slowly morphed from a wonderful place full of individual, quirky sites, into a desert of bland corporate silos, and now into a wasteland of AI slop. It’s becoming more and more difficult to find original information. There’s this conspiracy theory called “Dead Internet Theory” that basically says nearly all traffic on the Internet is generated by bots as part of a co-ordinated effort to manipulate us. Obviously it’s unhinged, but at the same time some elements of it are not that far off the mark. How long will it be before the Internet is so drowned in slop that it is, effectively, dead?

The other issue is what happens to the human jobs that have been replaced with slop? It’s way too easy for a money-pinching publication to get rid of their human staff in favour of pushing a few buttons and publishing the output. You’d hope that market forces would balance things out, but there doesn’t seem to be much sign of that happening. There are small movements of people putting badges and other marks on their work to show it was made by a human, perhaps that will take off?

On a more personal level: what happens to software developers? As it stands, code agents are nowhere near good enough to replace a human. At least not if you want a non-trivial output that works, can be changed in the future, and doesn’t have massive issues. Even if they get substantially better, I feel like you’d still need a knowledgeable human in the loop to keep a rein on everything. Does that mean the world needs far fewer software engineers? Probably. But it doesn’t mean we’re all going to be out of jobs overnight as some people seem to be predicting. The only way I can see that ever happening is if there’s a radical shift in how software is made: a new language of some kind that’s uniquely suited to LLMs, or new ways of composing software together from smaller parts⁶.

So… Yeah. Where does that leave us? I’ve written several thousand words about how LLMs are an ethical mess, but I’m going to carry on using them. That’s uncomfortable. But it’s not unlike using Amazon: they’re not a great company, I’d rather give the money to a local shop or supplier, but I’m also not going to massively inconvenience myself by being quixotic about it. Life is about compromises like this, and it’d be silly to think otherwise. I’ll stand on morals on some things⁷, but not to the extent of becoming a reclusive old crank shouting at the clouds.

That’s what Kagi is for. It’s great. You should try it. ↩︎
Or, maybe even worse, with added hallucinations. ↩︎
Hell, basically no human activity is good for the environment. ↩︎
±several kilowatts, because who knows? ↩︎
and didn’t elect people who withdraw from climate agreements, re-open coal plants, and so on… ↩︎
I don’t think I’m just saying this to reassure myself… ↩︎
I’ll grudgingly give Jeff Bezos money, but I sure as hell am not going to use any service from Elon Musk. ↩︎