<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet href="/feeds.xsl" type="text/xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:base="https://chameth.com/">
    <title>Chameth.com - posts not debugging-beyond-the-debugger, modern-css-is-fun, monthly-meanderings-2025-12, why-you-should-be-using-https</title>
    <subtitle>Personal homepage of Chris Smith</subtitle>
    <link href="https://chameth.com/feeds/posts/unlike/debugging-beyond-the-debugger,modern-css-is-fun,monthly-meanderings-2025-12,why-you-should-be-using-https/" rel="self"/>
    <link href="https://chameth.com/"/>
    <icon>https://chameth.com/favicon.png</icon>
    <updated>2026-07-23T00:00:00Z</updated>
    <id>https://chameth.com/</id>
    <author>
        <name>Chris Smith</name>
    </author>
    <entry>
        <title>Tracking PBs without the dark patterns</title>
        <link href="https://chameth.com/tracking-pbs-without-the-dark-patterns/"/>
        <updated>2026-07-23T00:00:00Z</updated>
        <id>https://chameth.com/tracking-pbs-without-the-dark-patterns/</id>
        <content xml:lang="en" type="html">&lt;p&gt;I’ve recently been bitten by some kind of strange exercise bug. I’ve gone from being mostly sedentary to struggling to not go out for a cheeky bike ride on a day that’s meant to be a rest day. Maybe this is what a midlife crisis feels like?&lt;/p&gt;
&lt;p&gt;Anyway, one of the things I’ve enjoyed about this development is seeing my personal best efforts gradually go up. There’s something incredibly rewarding about doing a run, and then seeing a nice little gold medal because as part of that you ran your fastest 1km ever. Unfortunately the only software I’ve found that does that is Strava, and boy do I not get on with it.&lt;/p&gt;
&lt;h3 id=&#34;the-many-shortcomings-of-strava&#34;&gt;The many shortcomings of Strava&lt;/h3&gt;
&lt;p&gt;Strava is basically &lt;em&gt;the&lt;/em&gt; app for running and cycling. Everyone talks about it. It sounds like they like it. But I really, really don’t. It’s like someone went through a catalogue of user-hostile behaviours and said “we’ll take them all!”.&lt;/p&gt;
&lt;p&gt;The problems started in the setup wizard: it touts a load of privacy features, but doesn’t offer any way to enable them. All your activities are public by default, and you have to spelunk around in the settings to turn that off. I wasn’t actually sure if I wanted my activities to be public or not, and ended up flip-flopping around a bit. There &lt;em&gt;is&lt;/em&gt; a clunky workflow to bulk change past activities, but it’s arbitrarily time-gated so you can’t use it too often…&lt;/p&gt;
&lt;p&gt;The bulk operations are extremely limited, too: you can set the activity visibility, and your heart rate visibility, and that’s it. Want to bulk delete things? You’re out of luck. Bulk rename? No chance. You either do them one-by-one or use some janky third-party script. I think I pressed around 300 little ‘delete’ links in the course of trying to get my activity history imported in a way I liked.&lt;/p&gt;
&lt;p&gt;Those problems give a bit of a signal about Strava’s priorities, but I could live with them. Once I’ve stopped faffing with the initial import I’m not really going to want to change the privacy settings or bulk edit things. Alas, the problems didn’t end there.&lt;/p&gt;
&lt;figure class=&#34;image right&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/strava.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/strava.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/strava.png&#34; alt=&#34;Screenshot of Strava notifications that say basically nothing but demand engagement&#34; loading=&#34;lazy&#34; width=&#34;979&#34; height=&#34;500&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;Look at me! Look at me! GIVE ME ENGAGEMENT!&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;The most obvious one is that everything is an upsell. Vast swathes of the app are premium-only, and they don’t ever get out of your way: they stick around acting as adverts. We’re all used to freemium apps that try to hawk upgrades, but the sheer amount of screen real estate taken up by these is egregious. I started a premium trial, expecting things to get a lot better, and… they didn’t really. All the premium features are still branded as such, so now instead of “look at what you could have if you paid us” it’s “look at what you’re getting for paying us (don’t even think about cancelling)”. Subscribing also makes your avatar get a premium outline, and there’s no way to disable it. I really hate that. Now &lt;em&gt;I’m&lt;/em&gt; an advert myself. I don’t want to be part of their weird dark patterns, thank you very much.&lt;/p&gt;
&lt;p&gt;And while we’re talking about dark patterns: there’s so much engagement bait that I physically cringe at it. You get a notification for logging an activity with a vague call to action to check it out. You get a separate notification when you break or come close to one of your PBs, but it doesn’t tell you which; there’s plenty of space in the notification for it, but they want you to hit the app instead. It all just feels so slimy to me. I don’t know whether it doesn’t bother other people as much, or if they’re desensitised to it, or just don’t realise how pleasant things could actually be.&lt;/p&gt;
&lt;h3 id=&#34;making-a-replacement&#34;&gt;Making a replacement&lt;/h3&gt;
&lt;p&gt;I realised pretty early on that if I only want automatic PB tracking, I could just write something myself to do it. I already use &lt;a href=&#34;https://www.healthyapps.dev/&#34;&gt;Health Auto Export&lt;/a&gt; to exfiltrate data from Apple Health, including workouts. I can just make a service that receives that data, crunches some numbers, and activates the dopamine receivers in my brain when I set a new record. But I didn’t really want to bother doing that if I could avoid it. I tried a whole slew of different apps and none of them did the thing, at least not without a subscription. And Strava had soured me on fitness subscriptions. I finally resigned myself to making something.&lt;/p&gt;
&lt;p&gt;And by “myself” I mean “an LLM”. This is one of the first projects where I’ve not written a single line of code, and I’ve barely even reviewed anything produced by the LLM. I gave it an overall architecture and some sample data, and had it write a plan. I critiqued the plan, had it broken down into a few different phases, and then had it work through them. I occasionally stopped it when I saw it doing something stupid, and gave it a couple of technical steers, but my feedback has mostly been functional. I’m probably not going to be releasing it publicly any time soon for that reason — I don’t want to publish it under my name when I’ve not even looked at the code. I also wouldn’t be comfortable making software this way if other people were going to rely on it, but for non-critical, personal-use-only software I’m happy with the approach.&lt;/p&gt;
&lt;p&gt;Anyway, I called it “Pompei Band”. Project names are hard. One of my common techniques is to put a term into an anagram finder; in this case “PB dopamine” anagrams to “Pompei Band”, and it even abbreviates to PB. Weird, but I like it. This is how it shows recent activities on the homepage:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/pb1.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/pb1.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/pb1.png&#34; alt=&#34;Screenshot of Pompei Band showing a running activity. Below the activity name, distance and date is a table of best efforts, showing values for 400m, 800m, 1km, 1mi, 2km, and 3km. Each effort shows the time taken, pace and grade-adjusted pace. Some have a golden laurel next to them with the next &amp;#39;New PB&amp;#39;, one has a silver second place medal with &amp;#39;2nd fastest&amp;#39;.&#34; loading=&#34;lazy&#34; width=&#34;975&#34; height=&#34;402&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;Dashboard entry for a recent run in PB&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;You can click through to see the standard activity details most tracking apps show: a map, heart rate, pace, gradient, and heart rate zones. You can also click on each effort to see a history of them, which gives a nice overview of progress:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/pb2.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/pb2.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/pb2.png&#34; alt=&#34;Screenshot of Pompei Band showing the PB history for a 1 mile run. Shows six activities, with a pace increasing from 8:55/km to 8:18/km. Each activity shows the name, date, time, pace, grade-adjusted pace, speed, and difference to the previous record. At the bottom is a graph showing all efforts as dots, with a line showing the PB improving over time.&#34; loading=&#34;lazy&#34; width=&#34;1487&#34; height=&#34;992&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;1 mile run PB history&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;It pretty much does exactly what I want, and has none of the annoying behaviours that Strava leans into. When I first drafted this post I wrote “there’s a few more bits I’d like to add at some point, but I can happily prompt them into existence when I want them”, but in the time I spent procrastinating before taking screenshots I ended up implementing them all. My favourite is this little history widget I added to the homepage, shown here next to Strava’s:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/history.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/history.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/tracking-pbs-without-the-dark-patterns/history.png&#34; alt=&#34;Side-by-side screenshots of Pompei Band and Strava showing recent workout history. PB&amp;#39;s is a grid of squares, a la a GitHub activity timeline, with each square coloured by activity or activities, with the brightness corresponding to how long the activity was; there&amp;#39;s a scale at the bottom showing four reference colours for running, walking, and cycling. Strava&amp;#39;s is a mini calendar of circles, with some of them being slightly bigger than others; on the right are weekly line charts showing time done on each type of activity.&#34; loading=&#34;lazy&#34; width=&#34;956&#34; height=&#34;202&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;Recent activity calendar in Pompei Band and in Strava&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;The Strava one just confuses me. I’m not sure what the scale of the circles means. I’m guessing it’s time, which in effect means it just highlights my bike rides which are longer than runs. The colours for the line charts don’t work for me, and I’m not sure what three zig-zag lines are meant to represent. Pompei Band’s version uses distinct, obvious colours for different activities, and scales the brightness independently for each type of activity based on length (so long cycle rides don’t affect the short runs). I also flipped the axes so it looks more like a GitHub activity timeline, because that’s how my brain works. Nothing groundbreaking, but also none of the paper-cuts the Strava version inflicted on me.&lt;/p&gt;
&lt;h3 id=&#34;software-doesnt-have-to-be-awful&#34;&gt;Software doesn’t have to be awful&lt;/h3&gt;
&lt;p&gt;This blog post has been rattling around in my head for a few days. I didn’t want it to just be a litany about Strava being bad, and I didn’t want to sound like an LLM hype piece, which made it tricky to figure out what I was actually trying to say. I think the core of it is: software doesn’t have to be awful.&lt;/p&gt;
&lt;p&gt;Everyone’s heard of &lt;a href=&#34;https://en.wikipedia.org/wiki/Enshittification&#34;&gt;enshittification&lt;/a&gt;, and it’s so prevalent that it’s easy to accept it as a fact of life. That’s just how late-stage capitalism &lt;em&gt;works&lt;/em&gt;, right? But we can just make better software. And as LLMs slowly become more and more capable, that “we” grows to a broader and broader group. It’s easy to be negative about that: we’re going to drown in vibe-coded products, much like we’re already drowning in written LLM slop. On the other hand, a future where you can just say “make me single-player Strava without all the dark patterns” and get something that works just how you want it is pretty exciting.&lt;/p&gt;
&lt;p&gt;The one thing I can’t decide: if everyone can whip up their own individual software, will the existing platforms actually improve? If Strava didn’t have all the dark patterns and engagement bait I would have just have paid for it. Will they realise that? Or will they double down even harder, try to lock you into their platform, and build moats around user data? Now I’ve said it, it seems pretty obvious they’ll try to do that. But I hope they won’t.&lt;/p&gt;
</content>
    </entry>
    <entry>
        <title>The Case of the Unchanging Config</title>
        <link href="https://chameth.com/the-case-of-the-unchanging-config/"/>
        <updated>2026-04-09T00:00:00Z</updated>
        <id>https://chameth.com/the-case-of-the-unchanging-config/</id>
        <content xml:lang="en" type="html">&lt;p&gt;Last week I was attempting to make it so I could share pictures on IRC directly from my client. This sounds simple, but it involves a bouncer&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:1&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt; that proxies the request to a standalone image hosting service that I had to modify to be compatible. At one point my testing loop was:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Make a change to the hosting service&lt;/li&gt;
&lt;li&gt;Commit it&lt;/li&gt;
&lt;li&gt;Tag a new release&lt;/li&gt;
&lt;li&gt;Wait for it to build&lt;/li&gt;
&lt;li&gt;Update the version I’m running on my server&lt;/li&gt;
&lt;li&gt;Reconfigure the bouncer to pass the new parameter or change the URL or whatever&lt;/li&gt;
&lt;li&gt;Try uploading a photo from my phone&lt;/li&gt;
&lt;li&gt;Realise I’ve overlooked something and go back to step one&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;I &lt;em&gt;could&lt;/em&gt; have set up a local copy of everything and tested it sensibly. I &lt;em&gt;should&lt;/em&gt; have set up a local copy of everything and tested it sensibly. But it seemed like such a trivial change, and setting up the whole environment seemed like such a pain. After the third or so iteration of failure I was pretty annoyed with myself, computers, and basically everything.&lt;/p&gt;
&lt;p&gt;My biggest annoyance was that my bouncer would not pick up the new URL from the config when I changed it. It’s meant to reload the config when it receives a &lt;code&gt;SIGHUP&lt;/code&gt;, and it claimed to in the logs, but I could clearly see it was still hitting the old URL. Restarting the bouncer to update the config is a pain, as it disconnects me from all the IRC networks, and has to reconnect to them all, reauthenticate, etc. It also mildly spams everyone who shares a channel with me&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:2&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;When I finally got everything working I had a look at the bouncer source, and thought I’d spotted the issue. I raised a bug report, ending in this remark:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;It looks like the config is reloaded properly, but the handler for uploads is created once at startup and has its own copy of the uploader, so effectively snapshots the config to whatever it is at startup:&lt;/p&gt;
&lt;/blockquote&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-nx&#34;&gt;fileUploadHandler&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-o&#34;&gt;:=&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;http&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nf&#34;&gt;HandlerFunc&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;chroma-kd&#34;&gt;func&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;http&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;ResponseWriter&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;r&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-o&#34;&gt;*&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;http&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;Request&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;cfg&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-o&#34;&gt;:=&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;srv&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nf&#34;&gt;Config&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;()&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;h&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-o&#34;&gt;:=&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;fileupload&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;Handler&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;Uploader&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;cfg&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;FileUploader&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;DB&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;          &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;db&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;Auth&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;cfg&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;Auth&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;        &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;HTTPOrigins&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;cfg&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;HTTPOrigins&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-w&#34;&gt;    &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;h&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;.&lt;/span&gt;&lt;span class=&#34;chroma-nf&#34;&gt;ServeHTTP&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;(&lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;w&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-nx&#34;&gt;r&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;)&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;})&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;I’m not quite sure what I thought I saw there, but that’s not right. I want to call it a hallucination, but that feels like a very overloaded word these days. I guess I was just seeing what I wanted to see, and that was a good excuse to stop investigating. Still, even if I misidentified the cause, the bug was still present, right? … Right?&lt;/p&gt;
&lt;h3 id=&#34;the-science-of-deduction&#34;&gt;The Science of Deduction&lt;/h3&gt;
&lt;p&gt;When the developer got back and said he couldn’t reproduce my issue, I went back to my install and immediately reproduced it. With a calmer head, I figured it was probably something with my particular set up. First thing to check: am I actually running the version I think I am?&lt;/p&gt;
&lt;p&gt;Unfortunately the bouncer doesn’t actually seem to expose the version anywhere that I can see. It’s not in the logs, it doesn’t have a &lt;code&gt;-version&lt;/code&gt; flag, and none of the IRC-based status commands seem to include it. But I know it’s a Go app, and I know Go embeds the version information. &lt;code&gt;go version -m &amp;lt;binary&amp;gt;&lt;/code&gt; will dump it all out, but the binary is inside a Docker image, and the Docker image is one of my &lt;a href=&#34;https://chameth.com/artisanal-docker-images/&#34;&gt;nice, minimal, artisanal ones&lt;/a&gt; so doesn’t ship a &lt;code&gt;go&lt;/code&gt; binary. No problem, &lt;code&gt;docker compose cp bouncer:/bnc ./bnc&lt;/code&gt; yoinks the binary out of the container, and then dumping the version shows that, yes, I am running the version I thought I was. Hmm.&lt;/p&gt;
&lt;p&gt;The next tool I reached for in &lt;a href=&#34;https://chameth.com/debugging-beyond-the-debugger/&#34;&gt;my toolbox&lt;/a&gt; was &lt;code&gt;strace&lt;/code&gt;. Maybe it’s not actually reading the file for some reason? I immediately executed &lt;code&gt;strace -p &amp;lt;pid&amp;gt; -e trace=openat,open,read,pread64 -f&lt;/code&gt; without having to look any part of that up. Yep. Definitely. Then I edited the config, &lt;code&gt;HUP&lt;/code&gt;’d the bouncer, and saw that it was… reading the config file. As it’s meant to. By default &lt;code&gt;strace&lt;/code&gt; truncates strings to 32 bytes, so I couldn’t actually see the line I’d changed. Some more definitely-not-RTFMing later, and rerunning it with an extra &lt;code&gt;-s 65536&lt;/code&gt; let me see the full config. Surprise! The config hadn’t changed!&lt;/p&gt;
&lt;p&gt;To confirm my findings, I used &lt;code&gt;docker compose cp&lt;/code&gt; again, this time yoinking the config file from inside the container. The inside config file was definitely different to the outside config file. What? My hypothesis at this point was “something something Docker nonsense”. I mount the config as read-only, and was wondering if that meant that Docker was doing something &lt;em&gt;weird&lt;/em&gt; instead of just bind mounting it. A quick trip to &lt;code&gt;/proc/&amp;lt;pid&amp;gt;/mounts&lt;/code&gt; showed that it was, in fact, not doing anything weird, and was just bind mounting it.&lt;/p&gt;
&lt;p&gt;If the file is bind mounted, then surely it’s the same file? I ran &lt;code&gt;stat&lt;/code&gt; on the file on the host, noted the inode number, then pondered how to actually run &lt;code&gt;stat&lt;/code&gt; on the file inside the container, given the aforementioned awkwardly minimal image. The solution was easy: access it via &lt;code&gt;/proc/&amp;lt;pid&amp;gt;/root/&lt;/code&gt;. I could’ve saved myself a bunch of &lt;code&gt;docker compose cp&lt;/code&gt; if I’d thought about that earlier. Oh well. The inode of that file was different. What?&lt;/p&gt;
&lt;p&gt;The answer was DNS. It’s always DNS. Oh, sorry, force of habit. I meant the answer was &lt;em&gt;vim&lt;/em&gt;. I was editing the config in vim, and when it saves files, by default, it writes the new content to a temporary file and does an atomic rename. That’s normally a good thing: it prevents corruption if the write fails midway through&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:3&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt;. In this case, though, that means the new file has a new inode. Bind mounting a file binds to the inode, so the container just keeps clutching onto the original config from when it was started, blissfully unaware that the party has relocated down the street.&lt;/p&gt;
&lt;p&gt;This problem is likely to happen whenever you bind mount a file into a container. When I mentioned this to a friend, he immediately responded “oh yeah, never do that”, and went on to describe the horrible hacks he’s had to add to Ansible to sidestep the issue. The nicer solution is to just bind mount an entire directory if you can, as then it doesn’t matter what happens to the files within it. I really like having the config files sat alongside the Docker compose files, though; having to create a directory just to work around some bind mount weirdness upsets me.&lt;/p&gt;
&lt;p&gt;Now I knew what the problem was, I found there was an issue &lt;a href=&#34;https://github.com/moby/moby/issues/6011&#34;&gt;raised against Docker&lt;/a&gt; twelve years ago&lt;sup id=&#34;fnref:4&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:4&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt;. The first response was “that’s expected”, and I fully agree with the author: “respectfully, that might be expected by you, but it was not expected by me”!&lt;sup id=&#34;fnref:5&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:5&#34; role=&#34;doc-noteref&#34;&gt;5&lt;/a&gt;&lt;/sup&gt; As a result of the issue they &lt;a href=&#34;https://github.com/moby/moby/pull/6854/changes&#34;&gt;added a nice note to the docs&lt;/a&gt;:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;
Many tools used to edit files including &lt;code&gt;vi&lt;/code&gt; and &lt;code&gt;sed --in-place&lt;/code&gt; may result
in an inode change. Since Docker v1.1.0, this will produce an error such as
“&lt;em&gt;sed: cannot rename ./sedKdJ9Dy: Device or resource busy&lt;/em&gt;”. In the case where
you want to edit the mounted file, it is often easiest to instead mount the
parent directory.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;But in the intervening twelve years, both the note and the functionality described have gone missing. Ho-hum.&lt;/p&gt;
&lt;p&gt;I found it interesting how I’ve only just hit this problem, given how long I’ve used Docker. But I realised that almost everything I run I’m happy to just restart. Cattle, not pets, and so on. My IRC bouncer is one of the few exceptions to that. The only other thing I regularly hot reloaded was &lt;a href=&#34;https://github.com/csmith/centauri&#34;&gt;Centauri&lt;/a&gt;, my reverse proxy, but that had a whole config directory mounted because it was shared between containers, so nicely sidestepped the foot-gun.&lt;/p&gt;
&lt;p&gt;So lesson learnt: check for weird bind mount issues before raising issues about config hot reloading. It’ll join the esteemed company of “maybe the drive is full and causing completely unrelated problems?”, “perhaps everything is dog slow because the kernel ran out of entropy?”, and “did systemd sneakily take over that functionality while you weren’t looking?” in the troubleshooting checklist.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr/&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;An IRC bouncer is basically an always-on proxy. It connects to the IRC networks for you, then your clients connect to your bouncer. The bouncer can then send incoming messages to all your different clients, cache them when you’re offline and replay them later, and lots of other nice things people take for granted in their chat apps these days. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:1&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;
&lt;p&gt;Join/part/quit spam is part of IRC, and clients have ways of handling it, but I still &lt;em&gt;feel&lt;/em&gt; bad about doing it excessively. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:2&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34;&gt;
&lt;p&gt;You can disable this in vim by setting &lt;code&gt;backupcopy=yes&lt;/code&gt;. Good luck redoing all the debugging if you ever accidentally remove that from your &lt;code&gt;vimrc&lt;/code&gt;, though! &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:3&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34;&gt;
&lt;p&gt;Which is weird, because Docker can’t possibly be that old. That would make me much older than I’m prepared to accept. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:4&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:5&#34;&gt;
&lt;p&gt;If you read the thread they didn’t actually mean to sound so dismissive, but it’s still pretty funny. “Yes, it’s expected that the foot-gun causes your foot to hurt. Duh. What did you expect?” &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:5&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</content>
    </entry>
    <entry>
        <title>The longest way to represent a date</title>
        <link href="https://chameth.com/the-longest-way-to-represent-a-date/"/>
        <updated>2026-03-14T00:00:00Z</updated>
        <id>https://chameth.com/the-longest-way-to-represent-a-date/</id>
        <content xml:lang="en" type="html">&lt;p&gt;The other day, someone on IRC posed this question: “What is the longest way to represent a date using any means possible that isn’t just repeated filler?”&lt;/p&gt;
&lt;p&gt;Some people jumped for writing the date out in languages that had longer translations. My immediate reaction was instead to suggest an obnoxious string-based representation of a unix timestamp: “one second after one second after one second after … midnight on January 1st 1970”. It’s &lt;em&gt;very&lt;/em&gt; repetitive, but it’s not filler: taking out any of the repetitions would change the value. Effectively it’s a &lt;a href=&#34;https://en.wikipedia.org/wiki/Successor_function&#34;&gt;successor function&lt;/a&gt; for unix timestamps, writ long. Some quick napkin maths suggests the full string representation would be around 30GiB&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:1&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;An obvious approach to make it bigger is to make it more precise. If we do the same sort of successor function for &lt;em&gt;milliseconds&lt;/em&gt; it’d be a bit over 1000x longer&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:2&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;. It actually ends up being just over 35TiB&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:3&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;There’s no reason to stop there, though. You can step down to microseconds, nanoseconds, and so on… or… we can just skip right to the end. How about: “one oscillation of the caesium-133 hyperfine transition frequency after one oscillation of the caesium-133 hyperfine transition frequency after … midnight on January 1st 1970”? There are over 9 billion of those every second, and the phrasing is &lt;em&gt;wordy&lt;/em&gt;. That comes out to almost 1ZiB&lt;sup id=&#34;fnref:4&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:4&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt;. If we pooled all the data storage on the planet together we could save a couple hundred of these timestamps.&lt;/p&gt;
&lt;p&gt;One final step, then: why use the unix epoch when we could use the universe’s own&lt;sup id=&#34;fnref:5&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:5&#34; role=&#34;doc-noteref&#34;&gt;5&lt;/a&gt;&lt;/sup&gt;? We’re something around 4×10^17 seconds past the big bang, so… hold on, I need to look up some units… that’d be over 200,000 yobibytes&lt;sup id=&#34;fnref:6&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:6&#34; role=&#34;doc-noteref&#34;&gt;6&lt;/a&gt;&lt;/sup&gt;. There’s not actually a standard prefix for that order of magnitude, according to Wikipedia. I think it should be 200 robibytes. Does anyone have a contact at the IEC?&lt;/p&gt;
&lt;p&gt;That is a big number. Is it impossibly big? If we wanted to track our current time, we’d need to write around 600GiB/s&lt;sup id=&#34;fnref:7&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:7&#34; role=&#34;doc-noteref&#34;&gt;7&lt;/a&gt;&lt;/sup&gt; to disk. That’s a bit more than we can manage at the minute, but isn’t completely outrageous. What about storage? Apparently hard drives require around a million atoms to store a bit of data. There’s some research showing it’s possible to use as few as 12, but I don’t think it’s anything that can be used at scale anytime soon&lt;sup id=&#34;fnref:8&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:8&#34; role=&#34;doc-noteref&#34;&gt;8&lt;/a&gt;&lt;/sup&gt;. So just for the direct data storage, not counting all the other infrastructure you need for a disk drive to &lt;em&gt;work&lt;/em&gt; we’d need around 10^36 atoms. If we sourced our atoms for our local &lt;a href=&#34;https://en.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizer&#34;&gt;paperclip maximiser&lt;/a&gt; we’d need around 200 trillion paperclips. For just one timestamp.&lt;/p&gt;
&lt;p&gt;On the plus side, this timestamp format would compress &lt;em&gt;wonderfully&lt;/em&gt;.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr/&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;17 bytes of repetition * 1773528583 seconds ~= 3×10^10. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:1&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;
&lt;p&gt;There’d be a thousand times more repetitions, and each one would have the extra five bytes from the “milli” prefix. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:2&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34;&gt;
&lt;p&gt;22 bytes of repetition * 1773528583000 milliseconds ~= 3×10^13. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:3&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34;&gt;
&lt;p&gt;72 bytes of repetition * 1773528583 seconds * 9192631770 oscillations per second ~= 1×10^21. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:4&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:5&#34;&gt;
&lt;p&gt;Other than it being impractical, not known to a decent accuracy, and the myriad other problems, of course. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:5&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:6&#34;&gt;
&lt;p&gt;72 bytes of repetition * 4×10^17 seconds * 9192631770 oscillations per second ~= 3×10^29. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:6&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:7&#34;&gt;
&lt;p&gt;72 bytes of repetition * 9192631770 oscillations per second ~= 6×10^11. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:7&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:8&#34;&gt;
&lt;p&gt;Not that we could scale conventional drives up this much, either. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:8&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</content>
    </entry>
    <entry>
        <title>An interesting Tailscale + Docker gotcha</title>
        <link href="https://chameth.com/tailscale-docker-gotcha/"/>
        <updated>2026-01-29T00:00:00Z</updated>
        <id>https://chameth.com/tailscale-docker-gotcha/</id>
        <content xml:lang="en" type="html">&lt;p&gt;As I’ve &lt;a href=&#34;https://chameth.com/how-i-use-tailscale/&#34;&gt;written&lt;/a&gt; &lt;a href=&#34;https://chameth.com/avoiding-the-consequences-of-dumb-laws-with-tailscale/&#34;&gt;about&lt;/a&gt; &lt;a href=&#34;https://chameth.com/exposing-game-servers-over-tailscale/&#34;&gt;before&lt;/a&gt;, I use &lt;a href=&#34;https://tailscale.com/&#34;&gt;Tailscale&lt;/a&gt; for a lot of things. I thought I had it set up in a reasonably secure manner, but I recently noticed a problem.&lt;/p&gt;
&lt;p&gt;I use Tailscale’s ACLs to limit what each node can access, based on the tags I apply to it. So an &lt;code&gt;app&lt;/code&gt; node can’t access anything via Tailscale, while an &lt;code&gt;integration&lt;/code&gt; or &lt;code&gt;server&lt;/code&gt; node can access things tagged with either &lt;code&gt;app&lt;/code&gt; or &lt;code&gt;integration&lt;/code&gt;. This is expressed pretty simply in the Tailscale ACL JSON:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tagOwners&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-c1&#34;&gt;// Servers that can be SSH&amp;#39;d into
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tag:server&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-c1&#34;&gt;// Applications that are exposed on tailscale but never connect out
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tag:app&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-c1&#34;&gt;// Things which talk to other services over tailscale (connecting to apps etc)
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;grants&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-c1&#34;&gt;// Users can access everything
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:member&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ip&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-c1&#34;&gt;// Servers and integrations can access integrations and apps
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:server&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:app&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ip&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Over the past week I’ve been setting up a private &lt;a href=&#34;https://forgejo.org/&#34;&gt;Forgejo&lt;/a&gt; instance behind Tailscale, complete with an actions runner that runs things using a docker-in-docker container. I didn’t want the runner knowing anything about Tailscale, so I had it configured to speak to Forgejo direct over HTTP (&lt;code&gt;forgejo:3000&lt;/code&gt;) instead of using full Tailscale HTTPS URL that I use when accessing it (&lt;code&gt;http://git.example-net.ts.net/&lt;/code&gt;)&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:1&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;Everything was going fine, until I forgot to do that translation… and it worked. My Forgejo action runners could access anything on my tailnet. I run several things on Tailscale that just have authentication turned off, on the basis that only authorised devices can access them. Things like the admin interface for this website. I definitely didn’t intend for any workflow I run on my git server to have access to edit my website!&lt;/p&gt;
&lt;p&gt;This wasn’t just limited to Forgejo, either. Any docker container I was running could access the tailnet. In hindsight it’s fairly obvious why: the host is running Tailscale, connected as a node tagged with my &lt;code&gt;server&lt;/code&gt; tag. That creates a &lt;code&gt;tailscale0&lt;/code&gt; interface, and automagically sets up iptables rules to route Tailscale traffic over the interface. Docker also automagically sets up iptables rules to bridge traffic, and apparently these two sets of rules interact in such a way that traffic from Docker containers is allowed to route via the &lt;code&gt;tailscale0&lt;/code&gt; interface.&lt;/p&gt;
&lt;p&gt;I say it’s fairly obvious in hindsight — there’s no reason why Docker would special case any particular host interface after all — but it still feels pretty surprising. Because both bits of software inject their own iptables rules, I never really had a good mental model for how they interact. The host Tailscale node was a completely separate building block to Docker. It would be a pain to use either of them if they didn’t do these rules, but it’s also one of the reasons I don’t really like “magical” things&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:2&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;There are lots of ways to fix this, but none of them feel particularly great. You can configure both Tailscale and Docker to not automatically fiddle with iptables and handle the rules yourself, but I really hate dealing with iptables&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:3&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt;. As a stopgap I did hold my nose and add some iptables rules to drop traffic to the &lt;code&gt;tailscale0&lt;/code&gt; interface if it originated from the IP ranges that Docker was configured to use:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 192.168.0.0/16 -o tailscale0 -j DROP
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 172.17.0.0/12 -o tailscale0 -j DROP
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The &lt;code&gt;DOCKER-USER&lt;/code&gt; chain is a nice little escape hatch; it comes before the main auto-generated &lt;code&gt;DOCKER&lt;/code&gt; chain, and Docker leaves the rules in it alone.&lt;/p&gt;
&lt;p&gt;So I added these rules, and felt pretty good about myself, and then… everything started breaking in weird ways. After some debugging I realised the problem was DNS&lt;sup id=&#34;fnref:4&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:4&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt;. The server’s DNS resolver is &lt;code&gt;100.100.100.100&lt;/code&gt;, a special Tailscale address. This is so that it can resolve tailnet hostnames via &lt;a href=&#34;https://tailscale.com/kb/1081/magicdns&#34;&gt;MagicDNS&lt;/a&gt; (and so I don’t have to configure my custom DNS servers manually on each device; Tailscale does it for me). My new iptables rules inadvertently dropped all the DNS packets coming from docker containers. D’oh.&lt;/p&gt;
&lt;p&gt;Obviously the solution here is to double down and add MORE iptables rules:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 192.168.0.0/16 -d 100.100.100.100 -o tailscale0 -j ACCEPT
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 172.17.0.0/12 -d 100.100.100.100 -o tailscale0 -j ACCEPT
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;This got DNS working again, but it all feels a bit gross. Not least because to actually make these persist you need to use &lt;code&gt;iptables-save&lt;/code&gt;, and then you also get all the rules that both Docker and Tailscale have inserted. I came across &lt;a href=&#34;https://github.com/giangi/iptables-docker-filter&#34;&gt;a script&lt;/a&gt; to filter out the Docker ones, but… yuck&lt;sup id=&#34;fnref:5&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:5&#34; role=&#34;doc-noteref&#34;&gt;5&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;I realised a cleaner idea might just be to lock down what &lt;code&gt;server&lt;/code&gt; tagged nodes could access within Tailscale itself. That way I can avoid touching iptables at all. It’s handy to be able to &lt;code&gt;curl&lt;/code&gt; services from the host when debugging, but it’s not really necessary. So I removed that access, and… stuff broke again. I’m using my Forgejo instance as a registry for some of the docker images I run, so the docker daemon needs to be able to reach it. I ended up making a new tag for &lt;code&gt;infrastructure&lt;/code&gt;, which can be accessed from &lt;code&gt;server&lt;/code&gt; devices. This does still allow all the docker containers to reach Forgejo, but I already have it set up with appropriate access controls and public/private repository splits. Forgejo is a service designed to run publicly, so this seems a reasonable trade-off for convenience. I used the &lt;code&gt;tests&lt;/code&gt; feature of Tailscale’s ACL config to make sure I’d got the rules right:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tests&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-c1&#34;&gt;// Servers can only access infrastructure
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;   &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:server&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;proto&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tcp&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;allow&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:infrastructure:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;deny&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;100.84.16.43:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;me@example.com:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:server:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:app:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The whole situation still feels a bit messy. If I ever get around to switching to nftables I might loop back and manually craft some rules for routing traffic, instead of leaving Tailscale and Docker to do their own thing.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr/&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;I discovered later on that logging into the container registry at &lt;code&gt;forgejo:3000&lt;/code&gt; actually issued a redirect to &lt;code&gt;https://git.example-net.ts.net/&lt;/code&gt; so this was all basically for naught… &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:1&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;
&lt;p&gt;I didn’t use Tailscale’s MagicDNS for a long time just because the word “magic” put me off. Only when I eventually got around to learning how it worked, and seeing that it wasn’t really that magical under the hood, did I change my mind. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:2&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34;&gt;
&lt;p&gt;nftables seems far better in lots of ways, but I can’t really be bothered migrating. Maybe next time I reimage the server for whatever reason… &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:3&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34;&gt;
&lt;p&gt;It’s always DNS… &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:4&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:5&#34;&gt;
&lt;p&gt;Again, nftables would almost certainly help here. It actually has (gasp) configuration files. But again, I really didn’t want to spend the time migrating. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:5&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</content>
    </entry>
    <entry>
        <title>Surge protectors: marketing vs reality</title>
        <link href="https://chameth.com/surge-protectors-marketing-vs-reality/"/>
        <updated>2026-01-24T00:00:00Z</updated>
        <id>https://chameth.com/surge-protectors-marketing-vs-reality/</id>
        <content xml:lang="en" type="html">&lt;p&gt;A while back I went down a deep rabbit hole looking into surge protectors, and what all the different numbers mean, and how that affects things in case of a voltage spike. Then I didn’t really do anything with the information, other than bore a few friends, and look around in despair at all the shockingly bad products out there. Time to fix that!&lt;/p&gt;
&lt;p&gt;I’m coming at this from the angle of a computer user in a country with very good electrical regulations. If you’re protecting something else, or live somewhere that doesn’t believe in grounding things, your mileage may vary.&lt;/p&gt;
&lt;h3 id=&#34;building-a-better-mental-model&#34;&gt;Building a better mental model&lt;/h3&gt;
&lt;p&gt;I think when most of us think of surge protectors, we think of an extension lead with some magical property that stops surges and protects everything plugged into them. It’s a bit like the shield on the USS Enterprise. If we put the shields up in time, they’ll stop everything thrown at them, until at some point they’re overloaded and stop working. Only then will we have problems. There’s even a little LED that goes out when she cannae take it any more, cap’n.&lt;/p&gt;
&lt;p&gt;Of course, if that was actually the case, I wouldn’t be writing a blog post. Surge protectors are more like the crumple zone on a car. If you hit something, the crumple zone will absorb some of the impact, but you can quite easily still get injured. If the impact is big enough then the crumple zone will bleed some energy, but you’re still going to have a very bad time. My point here is that it’s not a perfect shield, can be overcome with a single excessive impact, and doesn’t magically recharge back to full strength.&lt;/p&gt;
&lt;h3 id=&#34;the-numbers-mason-what-do-they-mean&#34;&gt;The numbers, Mason! What do they mean?&lt;/h3&gt;
&lt;p&gt;To understand what protection these things really offer, we need to look at a couple of numbers. Unfortunately, they’re not the numbers that are displayed in the marketing. Sometimes they’re not even on the spec sheet. Most of the time they’re on the actual device, and if they’re not then it’s safe to just assume things are bad.&lt;/p&gt;
&lt;p&gt;The most important number is the let-through voltage, U&lt;sub&gt;p&lt;/sub&gt;. It may also be called the clamping voltage, the voltage protection rating, or the VPR. This is the voltage that will be let through, before the surge protector does &lt;em&gt;anything&lt;/em&gt;&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:1&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;. We use 230V mains, with a +10%/-6% tolerance. So we shouldn’t be seeing anything above 253V. If you were designing a surge protector, you’d want it to engage a little above this, right? Maybe call it 300V so it doesn’t cut in prematurely? If you have a surge protector nearby, I invite you to try and find its U&lt;sub&gt;p&lt;/sub&gt; value. If you don’t have one, you can follow along with mine:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/surge-protectors-marketing-vs-reality/surge.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/surge-protectors-marketing-vs-reality/surge.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/surge-protectors-marketing-vs-reality/surge.jpg&#34; alt=&#34;The back of a surge protector, showing some specs. Uoc = 4kV, Uc = 250V, Up = 1.5kV&#34; loading=&#34;lazy&#34; width=&#34;1000&#34; height=&#34;750&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;A not-very-good surge protector. Please take a moment to consider how difficult it was to make this text readable.&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;Yes, that’s 1,500 Volts. Up until that point the surge protector does nothing. Your computer’s power supply just has to deal with it. That’s not even the highest I’ve seen, either. It’s just the closest I had to hand.&lt;/p&gt;
&lt;p&gt;So what are the other numbers? U&lt;sub&gt;c&lt;/sub&gt; is the maximum continuous operating voltage. That’s probably &lt;em&gt;fine&lt;/em&gt; — in the worst case it’ll slowly degrade over time if the mains rides the 253V edge — but at the same time, would you not spec it for 275V or more given that? For the most part, we don’t really care about this, though. If the surge protector has the right kind of plug on it, then it’s probably got a U&lt;sub&gt;c&lt;/sub&gt; in the right ballpark.&lt;/p&gt;
&lt;p&gt;Then we have U&lt;sub&gt;oc&lt;/sub&gt;, which is the open circuit voltage. This is one of the numbers that might end up on the marketing, because it can be big! This is the surge voltage that the device can sustain without failing itself. So for this surge protector, it won’t do anything for surges up to 1.5kV, between 1.5kV and 4kV it will clamp the voltage, and above 4kV it might fail in some manner. That failure could be failing open and leaving your computer to deal with the rest of the surge (the little LED would go out, though!).&lt;/p&gt;
&lt;p&gt;The number not on the device that’s on all the marketing materials is the “Joule rating”. That’s how much energy the thing can absorb before it fails. That can be gradually drained by small surges over time, or by a big one. Something in the realm of 1kJ seems to be a “good” value, but what does it actually mean? Say we had a surge of 1.5kV, our 1kJ of protection would cover 0.66 Amp seconds. Surges are typically very short; let’s say one lasts 2µs. That energy budget would allow for 333kA of current to be handled! That’s an order of magnitude more than a lightning strike! Amazing! Except… There’s also a maximum surge current rating, and I guarantee it’s less than that. The actual number on the Joule rating is basically useless given all the other constraints, but the bigger the number the more hardy the protector will be, in general.&lt;/p&gt;
&lt;p&gt;Oh, one more thing on that Joule rating. Sometimes surge protectors will have multiple different protection devices inside, especially when they protect other connectors like coax or telephone cables. Sometimes the Joule rating will just be the sum of all the individual protectors, so is even more useless. Yay marketing.&lt;/p&gt;
&lt;h3 id=&#34;how-much-abuse-can-a-psu-take-anyway&#34;&gt;How much abuse can a PSU take, anyway?&lt;/h3&gt;
&lt;p&gt;OK, so it turns out surge protectors are… underwhelming, shall we say? If there’s a surge, your computer is going to be &lt;em&gt;involved&lt;/em&gt;. So what can PSUs actually deal with?&lt;/p&gt;
&lt;p&gt;Turns out modern PSUs have surge protection built-in, along with all sorts of other “why is the electricity not electricitying right?” safeguards. I can’t find a single manufacturer that actually puts any numbers to that, though.&lt;/p&gt;
&lt;p&gt;If you were to open one up and look inside, you’d see something like this:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/surge-protectors-marketing-vs-reality/mov.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/surge-protectors-marketing-vs-reality/mov.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/surge-protectors-marketing-vs-reality/mov.jpg&#34; alt=&#34;Inside of a computer PSU: a printed circuit board with various components on it. Highlighted is a component wrapped in heatshrink, with the label ZNR1 next to it.&#34; loading=&#34;lazy&#34; width=&#34;600&#34; height=&#34;421&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;I spy with my little eye… something beginning with MOV!&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;That little thing hidden in heatshrink is a MOV, or Metal Oxide Varistor. Also known as a Zinc-Oxide Non-liner Resistor, or ZNR, as it’s labelled in the picture. It’s basically a resistor that varies its resistance based on the voltage. So it can have a high resistance at low voltages, and then after, say, 300V it could start dropping off. Can you guess what component is inside basically all consumer surge protectors? Yeah, these things! So, as mentioned, I don’t have numbers to back this up but I’m going to go out on a limb and say that the MOV in a £130 PSU can probably handle &lt;em&gt;at least&lt;/em&gt; the same as the MOVs in a £7 surge protector.&lt;/p&gt;
&lt;p&gt;It’s hard to imagine a situation where there’s a surge that would have destroyed the PSU that would be mitigated by an external surge protector. It’s probably either going to take both of them out, or they’ll both survive. No Enterprise shields here, I’m afraid.&lt;/p&gt;
&lt;p&gt;So is it not worth having a surge protector at all? Not quite. MOVs degrade with use, so if a surge protector handles some smaller surges, or takes bites out of bigger ones, it might &lt;em&gt;prolong&lt;/em&gt; the life of the PSU. Maybe that’s worth it, especially if you find one of the (increasingly rare) ones with a decently low clamping voltage.&lt;/p&gt;
&lt;h3 id=&#34;addendum-covered-equipment-warranties-and-magic-smoke&#34;&gt;Addendum: covered equipment warranties and magic smoke&lt;/h3&gt;
&lt;p&gt;A bunch of surge protectors come with a warranty for downstream equipment. That sounds like a great deal, right? Even if there’s a huge surge that the protector can’t handle, at least you can replace the equipment? Alas, no. These warranties only cover if the surge protector doesn’t operate &lt;em&gt;within its specifications&lt;/em&gt;. If you go over the max voltage, or the max current, or the max energy and all your equipment blows up, then the surge protector is merely working as designed. It’s &lt;em&gt;meant&lt;/em&gt; to fail in those circumstances, and at that point all bets are off. No warranty money for you.&lt;/p&gt;
&lt;p&gt;The other thing to bear in mind is that — in the UK at least — significant power surges aren’t common at all. If you travel with your computer then you’re more likely to come across dodgy electrics that can fry your computer than you are to hit a power surge. I’ve been to a lot of LAN events and have never heard of a surge protector popping and saving a computer; on the other hand I &lt;em&gt;have&lt;/em&gt; seen an entire row of computers release their magic smoke because the electrician hadn’t connected the three-phase supply properly. There’s absolutely no protection to be had from that!&lt;/p&gt;
&lt;p&gt;Photo credit: thanks to &lt;a href=&#34;https://greboid.com/&#34;&gt;Greg&lt;/a&gt; for supplying the picture of the PSU so I didn’t have to take my computer apart.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr/&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;Well, actually, it does a tiny bit before U&lt;sub&gt;p&lt;/sub&gt; because it’s not a binary switch, and life is messy. It won’t do &lt;em&gt;much&lt;/em&gt; of anything before U&lt;sub&gt;p&lt;/sub&gt;. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:1&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</content>
    </entry>
</feed>
