<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet href="/feeds.xsl" type="text/xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:base="https://chameth.com/">
    <title>Chameth.com - posts like tailscale-docker-gotcha, top-sites-dns-providers but not why-you-should-be-using-https</title>
    <subtitle>Personal homepage of Chris Smith</subtitle>
    <link href="https://chameth.com/feeds/posts/like/tailscale-docker-gotcha,top-sites-dns-providers/unlike/why-you-should-be-using-https/" rel="self"/>
    <link href="https://chameth.com/"/>
    <icon>https://chameth.com/favicon.png</icon>
    <updated>2026-01-29T00:00:00Z</updated>
    <id>https://chameth.com/</id>
    <author>
        <name>Chris Smith</name>
    </author>
    <entry>
        <title>An interesting Tailscale + Docker gotcha</title>
        <link href="https://chameth.com/tailscale-docker-gotcha/"/>
        <updated>2026-01-29T00:00:00Z</updated>
        <id>https://chameth.com/tailscale-docker-gotcha/</id>
        <content xml:lang="en" type="html">&lt;p&gt;As I’ve &lt;a href=&#34;https://chameth.com/how-i-use-tailscale/&#34;&gt;written&lt;/a&gt; &lt;a href=&#34;https://chameth.com/avoiding-the-consequences-of-dumb-laws-with-tailscale/&#34;&gt;about&lt;/a&gt; &lt;a href=&#34;https://chameth.com/exposing-game-servers-over-tailscale/&#34;&gt;before&lt;/a&gt;, I use &lt;a href=&#34;https://tailscale.com/&#34;&gt;Tailscale&lt;/a&gt; for a lot of things. I thought I had it set up in a reasonably secure manner, but I recently noticed a problem.&lt;/p&gt;
&lt;p&gt;I use Tailscale’s ACLs to limit what each node can access, based on the tags I apply to it. So an &lt;code&gt;app&lt;/code&gt; node can’t access anything via Tailscale, while an &lt;code&gt;integration&lt;/code&gt; or &lt;code&gt;server&lt;/code&gt; node can access things tagged with either &lt;code&gt;app&lt;/code&gt; or &lt;code&gt;integration&lt;/code&gt;. This is expressed pretty simply in the Tailscale ACL JSON:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tagOwners&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-c1&#34;&gt;// Servers that can be SSH&amp;#39;d into
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tag:server&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-c1&#34;&gt;// Applications that are exposed on tailscale but never connect out
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tag:app&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-c1&#34;&gt;// Things which talk to other services over tailscale (connecting to apps etc)
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;grants&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-c1&#34;&gt;// Users can access everything
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:member&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ip&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-c1&#34;&gt;// Servers and integrations can access integrations and apps
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:server&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:app&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ip&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;Over the past week I’ve been setting up a private &lt;a href=&#34;https://forgejo.org/&#34;&gt;Forgejo&lt;/a&gt; instance behind Tailscale, complete with an actions runner that runs things using a docker-in-docker container. I didn’t want the runner knowing anything about Tailscale, so I had it configured to speak to Forgejo direct over HTTP (&lt;code&gt;forgejo:3000&lt;/code&gt;) instead of using full Tailscale HTTPS URL that I use when accessing it (&lt;code&gt;http://git.example-net.ts.net/&lt;/code&gt;)&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:1&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;Everything was going fine, until I forgot to do that translation… and it worked. My Forgejo action runners could access anything on my tailnet. I run several things on Tailscale that just have authentication turned off, on the basis that only authorised devices can access them. Things like the admin interface for this website. I definitely didn’t intend for any workflow I run on my git server to have access to edit my website!&lt;/p&gt;
&lt;p&gt;This wasn’t just limited to Forgejo, either. Any docker container I was running could access the tailnet. In hindsight it’s fairly obvious why: the host is running Tailscale, connected as a node tagged with my &lt;code&gt;server&lt;/code&gt; tag. That creates a &lt;code&gt;tailscale0&lt;/code&gt; interface, and automagically sets up iptables rules to route Tailscale traffic over the interface. Docker also automagically sets up iptables rules to bridge traffic, and apparently these two sets of rules interact in such a way that traffic from Docker containers is allowed to route via the &lt;code&gt;tailscale0&lt;/code&gt; interface.&lt;/p&gt;
&lt;p&gt;I say it’s fairly obvious in hindsight — there’s no reason why Docker would special case any particular host interface after all — but it still feels pretty surprising. Because both bits of software inject their own iptables rules, I never really had a good mental model for how they interact. The host Tailscale node was a completely separate building block to Docker. It would be a pain to use either of them if they didn’t do these rules, but it’s also one of the reasons I don’t really like “magical” things&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:2&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;There are lots of ways to fix this, but none of them feel particularly great. You can configure both Tailscale and Docker to not automatically fiddle with iptables and handle the rules yourself, but I really hate dealing with iptables&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:3&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt;. As a stopgap I did hold my nose and add some iptables rules to drop traffic to the &lt;code&gt;tailscale0&lt;/code&gt; interface if it originated from the IP ranges that Docker was configured to use:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 192.168.0.0/16 -o tailscale0 -j DROP
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 172.17.0.0/12 -o tailscale0 -j DROP
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The &lt;code&gt;DOCKER-USER&lt;/code&gt; chain is a nice little escape hatch; it comes before the main auto-generated &lt;code&gt;DOCKER&lt;/code&gt; chain, and Docker leaves the rules in it alone.&lt;/p&gt;
&lt;p&gt;So I added these rules, and felt pretty good about myself, and then… everything started breaking in weird ways. After some debugging I realised the problem was DNS&lt;sup id=&#34;fnref:4&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:4&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt;. The server’s DNS resolver is &lt;code&gt;100.100.100.100&lt;/code&gt;, a special Tailscale address. This is so that it can resolve tailnet hostnames via &lt;a href=&#34;https://tailscale.com/kb/1081/magicdns&#34;&gt;MagicDNS&lt;/a&gt; (and so I don’t have to configure my custom DNS servers manually on each device; Tailscale does it for me). My new iptables rules inadvertently dropped all the DNS packets coming from docker containers. D’oh.&lt;/p&gt;
&lt;p&gt;Obviously the solution here is to double down and add MORE iptables rules:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 192.168.0.0/16 -d 100.100.100.100 -o tailscale0 -j ACCEPT
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;iptables -I DOCKER-USER -s 172.17.0.0/12 -d 100.100.100.100 -o tailscale0 -j ACCEPT
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;This got DNS working again, but it all feels a bit gross. Not least because to actually make these persist you need to use &lt;code&gt;iptables-save&lt;/code&gt;, and then you also get all the rules that both Docker and Tailscale have inserted. I came across &lt;a href=&#34;https://github.com/giangi/iptables-docker-filter&#34;&gt;a script&lt;/a&gt; to filter out the Docker ones, but… yuck&lt;sup id=&#34;fnref:5&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:5&#34; role=&#34;doc-noteref&#34;&gt;5&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;I realised a cleaner idea might just be to lock down what &lt;code&gt;server&lt;/code&gt; tagged nodes could access within Tailscale itself. That way I can avoid touching iptables at all. It’s handy to be able to &lt;code&gt;curl&lt;/code&gt; services from the host when debugging, but it’s not really necessary. So I removed that access, and… stuff broke again. I’m using my Forgejo instance as a registry for some of the docker images I run, so the docker daemon needs to be able to reach it. I ended up making a new tag for &lt;code&gt;infrastructure&lt;/code&gt;, which can be accessed from &lt;code&gt;server&lt;/code&gt; devices. This does still allow all the docker containers to reach Forgejo, but I already have it set up with appropriate access controls and public/private repository splits. Forgejo is a service designed to run publicly, so this seems a reasonable trade-off for convenience. I used the &lt;code&gt;tests&lt;/code&gt; feature of Tailscale’s ACL config to make sure I’d got the rules right:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;tests&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-c1&#34;&gt;// Servers can only access infrastructure
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;   &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:server&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;proto&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tcp&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;allow&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:infrastructure:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;deny&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;100.84.16.43:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;me@example.com:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:server:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:app:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;				&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration:8080&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;	&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;The whole situation still feels a bit messy. If I ever get around to switching to nftables I might loop back and manually craft some rules for routing traffic, instead of leaving Tailscale and Docker to do their own thing.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr/&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;I discovered later on that logging into the container registry at &lt;code&gt;forgejo:3000&lt;/code&gt; actually issued a redirect to &lt;code&gt;https://git.example-net.ts.net/&lt;/code&gt; so this was all basically for naught… &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:1&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;
&lt;p&gt;I didn’t use Tailscale’s MagicDNS for a long time just because the word “magic” put me off. Only when I eventually got around to learning how it worked, and seeing that it wasn’t really that magical under the hood, did I change my mind. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:2&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34;&gt;
&lt;p&gt;nftables seems far better in lots of ways, but I can’t really be bothered migrating. Maybe next time I reimage the server for whatever reason… &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:3&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34;&gt;
&lt;p&gt;It’s always DNS… &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:4&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:5&#34;&gt;
&lt;p&gt;Again, nftables would almost certainly help here. It actually has (gasp) configuration files. But again, I really didn’t want to spend the time migrating. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:5&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</content>
    </entry>
    <entry>
        <title>Exposing game servers over Tailscale</title>
        <link href="https://chameth.com/exposing-game-servers-over-tailscale/"/>
        <updated>2025-12-07T00:00:00Z</updated>
        <id>https://chameth.com/exposing-game-servers-over-tailscale/</id>
        <content xml:lang="en" type="html">&lt;p&gt;I’ve recently been playing a lot of &lt;a href=&#34;https://factorio.com/&#34;&gt;Factorio&lt;/a&gt; with a friend. I’ve been
hosting, but my desktop computer is behind far too many layers of NAT, and I can’t be bothered
dealing with setting up port forwards. Up until today we made do with Steam’s networking support,
which in our case ended up relaying the connection via one of their servers. This is amazing as
a free, no-hassle service, but the performance was so-so. We’d get random lag spikes, or the
initial map download would crawl along at about 1/100th of the speed we should be able to get.
We’re both fans of &lt;a href=&#34;https://tailscale.com/&#34;&gt;Tailscale&lt;/a&gt; though, so maybe there’s an easy solution
there?&lt;/p&gt;
&lt;p&gt;Tailscale even have an article on &lt;a href=&#34;https://tailscale.com/blog/factorio-multiplayer-video&#34;&gt;sharing a Factorio server&lt;/a&gt;
but it’s a ten minute long video&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:1&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;, and focused on setting up an actual dedicated server in Docker
rather than just clicking the convenient “Host” button in the game client. What we did instead was:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;I shared my desktop’s Tailscale node with my friend, by creating a sharing link in the &lt;a href=&#34;https://login.tailscale.com/admin/machines&#34;&gt;Admin console&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;My friend accepted the link, and could then see my machine in their admin panel&lt;/li&gt;
&lt;li&gt;I used &lt;code&gt;tailscale lock&lt;/code&gt; to sign my friend’s desktop’s node key, as I have &lt;a href=&#34;https://tailscale.com/kb/1226/tailnet-lock&#34;&gt;tailnet lock&lt;/a&gt; enabled&lt;/li&gt;
&lt;li&gt;I added a grant to my tailnet’s ACL to permit access from my friend to the specific IP/port on my tailnet:&lt;/li&gt;
&lt;/ol&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;friend@example.com&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;100.0.0.42&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;fd7a::42&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;			&lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ip&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;udp:34197&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;		&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;After this, my friend could connect to Factorio by entering the Tailscale IP address, or the Tailscale hostname (as they use &lt;a href=&#34;https://tailscale.com/kb/1081/magicdns&#34;&gt;MagicDNS&lt;/a&gt;).
Running &lt;code&gt;tailscale status&lt;/code&gt; shows that Tailscale managed to establish a direct connection despite the many layers of NAT involved,
and the performance improvements were significant and immediate.&lt;/p&gt;
&lt;p&gt;We did a bit of testing, and confirmed that no other services on my machine are accessible. And sharing a machine by default
only allows it to accept incoming connections, so I can’t get unwanted access to anything on my friend’s Tailnet, either.&lt;/p&gt;
&lt;p&gt;I’ve &lt;a href=&#34;https://chameth.com/how-i-use-tailscale/&#34;&gt;written before&lt;/a&gt; about all the different ways I use Tailscale; this is yet
another new way it’s solved a problem for me.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr/&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;I’m not a fan of video tutorials at the best of times, but especially not when I want to do something as quickly as possible so I can get back to &lt;del&gt;my addiction&lt;/del&gt; growing the factory. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:1&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</content>
    </entry>
    <entry>
        <title>Avoiding the Consequences of Dumb Laws with Tailscale</title>
        <link href="https://chameth.com/avoiding-the-consequences-of-dumb-laws-with-tailscale/"/>
        <updated>2025-09-30T00:00:00Z</updated>
        <id>https://chameth.com/avoiding-the-consequences-of-dumb-laws-with-tailscale/</id>
        <content xml:lang="en" type="html">&lt;p&gt;More and more sites are implementing privacy-invading age checks or just
completely blocking the UK thanks to the &lt;a href=&#34;https://www.legislation.gov.uk/ukpga/2023/50/contents&#34;&gt;Online Safety Act&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Protecting kids from &lt;em&gt;some&lt;/em&gt; content online is certainly a noble goal, but
the asinine guidance from Ofcom, threats of absolutely disproportionate fines,
and the stupidly broad categories of content have resulted in companies just
giving up or going through a tick-box exercise that offers very little
protection but lots of inconvenience and a complete invasion of privacy.&lt;/p&gt;
&lt;p&gt;Instead of uploading my ID to some third party company, I’ve taken to proxying
my traffic through to a country that doesn’t have such stupid laws. Thankfully,
Tailscale makes this really easy. I’ve discussed &lt;a href=&#34;https://chameth.com/how-i-use-tailscale/&#34;&gt;how I use Tailscale&lt;/a&gt;
before, but not really covered &lt;em&gt;app connectors&lt;/em&gt;. I find Tailscale’s description
of these pretty confusing, but they basically amount to automatic, DNS-based
subnet routing configurations (or, to put it another way, a per-website exit
node). You can safely ignore all references to ‘SaaS apps’ in their docs.&lt;/p&gt;
&lt;p&gt;I create a custom app connector, and give it the domains to be included:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/avoiding-the-consequences-of-dumb-laws-with-tailscale/apps.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/avoiding-the-consequences-of-dumb-laws-with-tailscale/apps.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/avoiding-the-consequences-of-dumb-laws-with-tailscale/apps.png&#34; alt=&#34;Screenshot of the app section in the Tailscale admin console. It shows a table with two entries: &amp;#39;reddit&amp;#39; and &amp;#39;bluesky&amp;#39;. Each entry has a list of domain names like &amp;#39;*.reddit.com, *.reddit.it&amp;#39;.&#34; loading=&#34;lazy&#34; width=&#34;1151&#34; height=&#34;397&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;App configuration in the Tailscale admin console&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;Tailscale then magically resolves those domains, and has the ‘connector’
advertise routes for them. Any client that accepts routes will start sending
requests to the connector, which passes them onto the Internet at large. Any
other traffic is left alone, unlike when you use an exit node.&lt;/p&gt;
&lt;p&gt;The special bit here is how you can specify wildcard domains. Tailscale proxies
the DNS requests from clients (so it can inject responses for nodes on your
tailnet), which means it can dynamically update the routes as you resolve new
domains. I tried to set this up more manually, and quickly came unstuck: despite
using the same DNS servers, my server and my desktop would get different responses
for the same query as it varied by geography. Trying to get the full set of
IPs (and keeping them updated) would have been a nightmare. Tailscale expanding
the wildcards nicely sidesteps all of that.&lt;/p&gt;
&lt;p&gt;At first I was just proxying the traffic to one of my servers, but just today
I added a new connector for Imgur and found I was still blocked, just for
different reasons. They not only block my entire country but also a load
of known datacenter IP ranges. Hmph. I fixed this by hacking up a new side
project: &lt;a href=&#34;https://github.com/csmith/tsv&#34;&gt;tsv&lt;/a&gt;. It’s a simple Go app that accepts
traffic from the tailnet (advertising itself as both an app connector and an
exit node), and passes it on to another VPN.&lt;/p&gt;
&lt;p&gt;There are lots of other ways you could accomplish this, but this makes it so
all my devices can still access services without any additional configuration.
As long as Tailscale is installed, the Internet will still work as it’s meant
to, without all the nonsense. If I come across a site that doesn’t work, adding
it is trivial: I just make a new app connector in Tailscale.&lt;/p&gt;
&lt;p&gt;Obvious disclaimer: the laws in the UK are binding on the service providers,
not the end user. Doing this sort of thing in other countries might be illegal.
I don’t know; do your own research! Also all of this is a workaround
for something that should be fixed at a legislative level, but I’m not holding
my breath.&lt;/p&gt;
</content>
    </entry>
    <entry>
        <title>How I use Tailscale</title>
        <link href="https://chameth.com/how-i-use-tailscale/"/>
        <updated>2025-06-25T00:00:00Z</updated>
        <id>https://chameth.com/how-i-use-tailscale/</id>
        <content xml:lang="en" type="html">&lt;figure class=&#34;image right&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/how-i-use-tailscale/logo.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/how-i-use-tailscale/logo.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/how-i-use-tailscale/logo.png&#34; alt=&#34;The Tailscale logo: a 3x3 grid of dots, with the middle row and bottom middle dot in white, forming a T shape&#34; loading=&#34;lazy&#34; width=&#34;500&#34; height=&#34;494&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;The Tailscale logo&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;I’ve been using &lt;a href=&#34;https://tailscale.com/&#34;&gt;Tailscale&lt;/a&gt; for around four years to connect my disparate devices,
servers and apps together. I wanted to talk a bit about how I use it, some cool
features you might not know about, and some stumbling blocks I encountered.&lt;/p&gt;
&lt;p&gt;I’m not sure Tailscale needs an introduction for the likely audience of this
blog, but I’ll give one anyway. Tailscale is basically a WireGuard&lt;sup id=&#34;fnref:1&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:1&#34; role=&#34;doc-noteref&#34;&gt;1&lt;/a&gt;&lt;/sup&gt;
orchestration service, with lots of nice features sprinkled on top. It’s a
subscription product, but it has an insanely generous free tier that covers
basically anything you’d ever want to do as an individual. They also open source
all their client software, and there’s a third party control server
implementation called &lt;a href=&#34;https://github.com/juanfont/headscale&#34;&gt;Headscale&lt;/a&gt; if you want to avoid the hosted system entirely.&lt;/p&gt;
&lt;h3 id=&#34;basic-connectivity&#34;&gt;Basic connectivity&lt;/h3&gt;
&lt;p&gt;At its core, Tailscale lets you easily connect from one device to another,
even if they’re not directly exposed to the Internet. You install the Tailscale
client wherever you like (on your phone, computer, servers, Raspberry Pi, etc),
authenticate the machine with the control server, and it can then talk to all
the other machines on the tailnet using their private Tailscale IP addresses.&lt;/p&gt;
&lt;p&gt;That isn’t anything revolutionary: it’s the basic premise behind VPNs. But
Tailscale makes it so easy. You don’t have to bother with any networking
configuration. You don’t have to distribute keys. You just install the
client&lt;sup id=&#34;fnref:2&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:2&#34; role=&#34;doc-noteref&#34;&gt;2&lt;/a&gt;&lt;/sup&gt;, and login.&lt;/p&gt;
&lt;!--more--&gt;
&lt;p&gt;For example, my &lt;a href=&#34;https://chameth.com/home-automation-without-megacorps/&#34;&gt;home automation service&lt;/a&gt;
runs on a Raspberry Pi that sits behind two different routers. I installed
Tailscale on it&lt;sup id=&#34;fnref:3&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:3&#34; role=&#34;doc-noteref&#34;&gt;3&lt;/a&gt;&lt;/sup&gt;, logged in, and immediately I could SSH into it from
my computer or my phone even when they’re on different networks.&lt;/p&gt;
&lt;p&gt;Speaking of SSH, Tailscale has special support for it whereby it handles any
incoming connection to port 22 from the Tailscale network, and deals with
authentication itself. No public keys or passwords: if you’re logged into
Tailscale you can be logged into the machine. This is particularly handy when
you SSH from a phone, as proper credential management is a bit of a nightmare
there.&lt;/p&gt;
&lt;p&gt;Exposing entire machines is just the start, though. If you run multiple services
on the machine you need to remember which port they’re all on. Yuck. Instead,
you can expose individual services as their own node on a tailnet. There
are lots of options for doing so: there’s &lt;a href=&#34;https://tailscale.com/kb/1282/docker&#34;&gt;an official Docker image&lt;/a&gt;,
a &lt;a href=&#34;https://pkg.go.dev/tailscale.com/tsnet&#34;&gt;Go library&lt;/a&gt;, and many third-party
tools (such as my own &lt;a href=&#34;https://github.com/csmith/centauri&#34;&gt;Centauri&lt;/a&gt; and &lt;a href=&#34;https://github.com/csmith/tsp&#34;&gt;tsp&lt;/a&gt;).&lt;/p&gt;
&lt;h3 id=&#34;not-just-a-vpn&#34;&gt;Not just a VPN&lt;/h3&gt;
&lt;p&gt;Exposing individual services wouldn’t be much use if you still had to remember
their IP addresses. For a long time I manually added DNS entries for Tailscale
nodes, allowing me to connect to &lt;code&gt;http://grafana/&lt;/code&gt; instead of
&lt;code&gt;http://100.73.190.100&lt;/code&gt;. Recently I switched to Tailscale’s &lt;a href=&#34;https://tailscale.com/kb/1081/magicdns&#34;&gt;MagicDNS&lt;/a&gt;,
though, which takes care of that for you. Every machine on the tailnet gets
an automatic DNS entry based on its name.&lt;/p&gt;
&lt;p&gt;I initially shied off of this because it changes the DNS resolver on every
machine it’s installed on, and it all seemed a bit too “magic” for me&lt;sup id=&#34;fnref:4&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:4&#34; role=&#34;doc-noteref&#34;&gt;4&lt;/a&gt;&lt;/sup&gt;. I got
fed up with manually adding DNS entries though, so I spent a bit of time
learning about it and got over my knee-jerk “what on earth is going on?!”
reaction. The DNS controls also let you force a specific upstream DNS server.
I use &lt;a href=&#34;https://nextdns.io&#34;&gt;NextDNS&lt;/a&gt; everywhere, so having Tailscale automatically
configure this for me on all my devices was a bonus&lt;sup id=&#34;fnref:5&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:5&#34; role=&#34;doc-noteref&#34;&gt;5&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;
&lt;p&gt;As well as the short names, you can resolve nodes on the tailnet using a
slightly longer form: &lt;code&gt;machine.your-tailnet.ts.net&lt;/code&gt;. The “your-tailnet” bit
is randomly generated, and you can reroll it in the control panel to find a fun
one. But why waste time say lot word, when few word do trick? Because Tailscale
can make these addresses globally routable, and issue TLS certs for them too.&lt;/p&gt;
&lt;p&gt;Say you’re developing something on your local machine, and want to get feedback
from someone. You can use Tailscale’s “funnel” feature to expose it publicly:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;tailscale funnel 127.0.0.1:8080
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;With no other options, this will expose the service over HTTPS on port 443.
You can then invite people to visit &lt;code&gt;https://machine.your-tailnet.ts.net&lt;/code&gt; and
they’ll connect through to port 8080 on your machine. They don’t even need
Tailscale installed. I don’t use this terribly often, but it’s a great
quality-of-life improvement on the occasions I do want to share something.&lt;/p&gt;
&lt;p&gt;Tailscale also has a “serve” command, which does much the same but only
exposes it over your tailnet. This is how you’d use the Tailscale Docker image
to expose a normal service onto your tailnet, but it’s also useful for similar
development scenarios. Occasionally I’ll want to test something on my phone,
and rather than make sure it’s connected to the right Wi-Fi network, recompiling
the app to not bind to localhost, figuring out my machine’s IP, and so on, and
so forth, I just run &lt;code&gt;tailscale serve&lt;/code&gt; and access it over Tailscale.&lt;/p&gt;
&lt;h3 id=&#34;authentication-two-ways&#34;&gt;Authentication, two ways&lt;/h3&gt;
&lt;p&gt;One of my minor frustrations with Tailscale was how cumbersome the authentication
process was. I logged in using GitHub, and every time I tried to login to
Tailscale, I had to bounce through the GitHub OAuth flow and then select which
organisation I wanted to use. Even if I was logged in to GitHub, it still shows
the permission page every time. It’s not a dealbreaker, but it’s definitely
not smooth. This is what it looks like:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/how-i-use-tailscale/auth.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/how-i-use-tailscale/auth.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/how-i-use-tailscale/auth.png&#34; alt=&#34;Screenshots of the Tailscale/GitHub login flow: (1) The Tailscale login form (2) The GitHub login form (3) GitHub permissions screen (4) Tailscale tailnet selection screen&#34; loading=&#34;lazy&#34; width=&#34;2000&#34; height=&#34;647&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;Logging in to Tailscale with a GitHub account&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;Tailscale have since added the ability to sign in with a custom OIDC provider,
so I set up a &lt;a href=&#34;https://www.keycloak.org/&#34;&gt;KeyCloak&lt;/a&gt; instance, and my login flow
now looks like this:&lt;/p&gt;
&lt;figure class=&#34;image full&#34;&gt;
  &lt;picture&gt;
      &lt;source srcset=&#34;https://chameth.com/how-i-use-tailscale/keycloak.avif&#34; type=&#34;image/avif&#34;/&gt;
      &lt;source srcset=&#34;https://chameth.com/how-i-use-tailscale/keycloak.webp&#34; type=&#34;image/webp&#34;/&gt;
      &lt;img src=&#34;https://chameth.com/how-i-use-tailscale/keycloak.png&#34; alt=&#34;Screenshots of the Tailscale/KeyCloak login flow: (1) The Tailscale login form (2) The KeyCloak login form&#34; loading=&#34;lazy&#34; width=&#34;1145&#34; height=&#34;647&#34;/&gt;
  &lt;/picture&gt;
  &lt;figcaption&gt;&lt;p&gt;Logging in to Tailscale with KeyCloak as an OIDC provider&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;p&gt;It’s a significantly shorter flow, and because KeyCloak is entirely under my
control I can configure how long I stay logged in for. It also reduces my
reliance on third parties: if GitHub goes down, or decides to ban my account&lt;sup id=&#34;fnref:6&#34;&gt;&lt;a class=&#34;footnote-ref&#34; href=&#34;#fn:6&#34; role=&#34;doc-noteref&#34;&gt;6&lt;/a&gt;&lt;/sup&gt;
or something, I can still access Tailscale.&lt;/p&gt;
&lt;p&gt;That’s how you log in to Tailscale, but Tailscale can also assist with logging
in to other services. If you connect to a HTTP service over Tailscale, it will
add in the following headers:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-err&#34;&gt;Tailscale-User-Login: alice@example.com
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-err&#34;&gt;Tailscale-User-Name: Alice Smith
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-err&#34;&gt;Tailscale-User-Profile-Pic: https://example.com/photo.jpg
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;A lot of software has support for accepting login credentials provided by a
reverse proxy, which is basically what’s happening here. For example, I run
a Grafana instance that’s only exposed over Tailscale. Its config (set
in environment variables) is:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-nt&#34;&gt;GF_USERS_AUTO_ASSIGN_ORG&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;true&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-nt&#34;&gt;GF_USERS_AUTO_ASSIGN_ORG_ROLE&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;Admin&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-nt&#34;&gt;GF_AUTH_PROXY_ENABLED&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;true&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-nt&#34;&gt;GF_AUTH_PROXY_HEADER_NAME&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;Tailscale-User-Login&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-nt&#34;&gt;GF_AUTH_PROXY_HEADER_PROPERTY&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;email&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-nt&#34;&gt;GF_AUTH_PROXY_AUTO_SIGN_UP&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt; &lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;true&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-w&#34;&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;So anyone who presents a &lt;code&gt;Tailscale-User-Login&lt;/code&gt; header gets an admin account
created with that as their e-mail address. It’s super convenient, but obviously
make sure it’s &lt;em&gt;only&lt;/em&gt; exposed via Tailscale if you do that!&lt;/p&gt;
&lt;p&gt;Other services I run that support this include &lt;a href=&#34;https://miniflux.app/docs/configuration.html#auth-proxy-header&#34;&gt;Miniflux&lt;/a&gt;
(an RSS reader), &lt;a href=&#34;https://bayang.github.io/jelu-web/configuration/#configuring-proxy-authentication&#34;&gt;Jelu&lt;/a&gt; (a book tracker),
&lt;a href=&#34;https://linkding.link/options/#ld_enable_auth_proxy&#34;&gt;Linkding&lt;/a&gt; (a bookmark manager),
and &lt;a href=&#34;https://manual.seafile.com/latest/config/remote_user/&#34;&gt;Seafile&lt;/a&gt; (a file storage system).
If you’re hosting something yourself, look out for “proxy auth” or “remote users”
in the documentation or configuration. It’s amazingly convenient to be able to access things
without entering credentials.&lt;/p&gt;
&lt;h3 id=&#34;acls-tags-and-mistakes&#34;&gt;ACLs, tags and mistakes&lt;/h3&gt;
&lt;p&gt;For a long time I didn’t bother with any kind of ACLs within my Tailscale
network. Every machine could access every other machine. Then one day a thought
hit me. The default Tailscale SSH config re-prompts you for user authentication,
but I’d disabled that by tweaking the config… That means anyone with access to
any of my machines (or who managed to get Tailscale credentials out of one of
my apps) would be able to SSH into anywhere else on my network. Even as root!
Compromising a small bit of my network would immediately lead to a complete
take-over. What’s the point in defence-in-depth if you roll out a red carpet
over the drawbridge for any attackers?&lt;/p&gt;
&lt;p&gt;The first and most obvious fix for this was to restrict root access over SSH.
I changed my Tailscale ACL to prompt for reauthentication for root access, but
still allow regular SSH unprompted:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;  &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ssh&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;action&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;accept&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;    &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:member&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;    &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:self&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;users&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:nonroot&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;action&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;check&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;    &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:member&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;    &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:self&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;users&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;root&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;  &lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;This still didn’t sit right with me, though. Why should a random webserver be
able to (theoretically) SSH into my desktop computer unchallenged? Tailscale
allows you to do more advanced ACLs by applying tags to devices. I dutifully
created a bunch of tags, and went through and tagged every device: &lt;code&gt;user&lt;/code&gt;, &lt;code&gt;server&lt;/code&gt;,
&lt;code&gt;app&lt;/code&gt;, &lt;code&gt;container&lt;/code&gt;, and so on. This, it turns out, was a mistake. I could no
longer SSH to anywhere. If I’d read
&lt;a href=&#34;https://tailscale.com/kb/1068/tags&#34;&gt;the docs&lt;/a&gt; properly, I might have noticed
how they describe tags:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Tailscale tags are how you authenticate and identify non-user devices, such as
servers and ephemeral nodes. They serve two primary purposes: to provide an
identity to non-user devices and to let you manage access control policies based on purpose.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;By applying a nice &lt;code&gt;user&lt;/code&gt; tag to all of my devices (computer, phone, tablet and
so on), I’d effectively removed my own identity from them. They no longer
counted for the &lt;code&gt;autogroup:member&lt;/code&gt; source in the SSH config. Whoops. And it’s
not easy to fix either:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;You cannot remove all tags from a device. A device with a tag-based identity
must have at least one tag.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;So I’d have to remove all the affected devices, then re-add them. At the time
I wanted to roll back &lt;em&gt;all&lt;/em&gt; the tags, which would mean reconnecting dozens of
nodes to the Tailnet. I did end up doing that, but I also took it as an
opportunity to switch to KeyCloak as I mentioned earlier. Setting everything
up again was a bit less painful when it wasn’t just correcting a dumb mistake.&lt;/p&gt;
&lt;p&gt;I now have a much more sensible tag setup: servers are tagged &lt;code&gt;server&lt;/code&gt;,
apps are tagged &lt;code&gt;app&lt;/code&gt;, and a few special apps are tagged &lt;code&gt;integration&lt;/code&gt;. I then
use Tailscale’s grants to limit access within the tailnet:&lt;/p&gt;
&lt;pre class=&#34;chroma-chroma&#34;&gt;&lt;code&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;  &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;grants&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-c1&#34;&gt;// Users can access everything
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;autogroup:member&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ip&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;{&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-c1&#34;&gt;// Servers and integrations can access integrations and apps
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;src&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:server&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;dst&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt; &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:integration&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;,&lt;/span&gt; &lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;tag:app&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;      &lt;span class=&#34;chroma-nt&#34;&gt;&amp;#34;ip&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;:&lt;/span&gt;  &lt;span class=&#34;chroma-p&#34;&gt;[&lt;/span&gt;&lt;span class=&#34;chroma-s2&#34;&gt;&amp;#34;*&amp;#34;&lt;/span&gt;&lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;    &lt;span class=&#34;chroma-p&#34;&gt;},&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;  &lt;span class=&#34;chroma-p&#34;&gt;],&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class=&#34;chroma-line&#34;&gt;&lt;span class=&#34;chroma-cl&#34;&gt;&lt;span class=&#34;chroma-p&#34;&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;p&gt;This stops anyone but a user device from accessing servers, and stops plain
apps from accessing anything at all. So for example my Grafana instance is
tagged as an &lt;code&gt;app&lt;/code&gt;; it doesn’t connect out over Tailscale for anything, it only
exposes its interface over Tailscale. My home automation software is tagged as
an &lt;code&gt;integration&lt;/code&gt;, as it reaches out to other services over Tailscale (e.g., to
push metrics it receives from devices).&lt;/p&gt;
&lt;p&gt;You could go even further and limit which specific machines talk to which
others, but I think this tier system sets reasonable boundaries for my use
without getting too much in the way.&lt;/p&gt;
&lt;h3 id=&#34;even-more&#34;&gt;Even more&lt;/h3&gt;
&lt;p&gt;There are lots of other cool features offered by Tailscale that I’ve not
covered: exit nodes (allowing you to route normal Internet traffic over
Tailscale; very handy when you’re behind a router you don’t trust, or if
you want to appear as being in another country for totally legitimate reasons);
Tailscale lock (a system where existing nodes have to verify any new ones,
guarding against both unauthorised access and Tailscale themselves inserting
nodes into your network); file sharing, and more I’ve probably forgotten.&lt;/p&gt;
&lt;p&gt;As I mentioned before, an awful lot is available on the free personal plan.
It’s definitely worth checking out if you manage servers, or self-host anything.
I can’t imagine going back to managing servers and private apps without
Tailscale, and I’m excited to see what else they do going forward.&lt;/p&gt;
&lt;div class=&#34;footnotes&#34; role=&#34;doc-endnotes&#34;&gt;
&lt;hr/&gt;
&lt;ol&gt;
&lt;li id=&#34;fn:1&#34;&gt;
&lt;p&gt;If you’re not familiar with it, WireGuard is a modern VPN included in
the Linux kernel. It’s basically the only VPN that isn’t terrible. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:1&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:2&#34;&gt;
&lt;p&gt;They have installation instructions and packaged versions of the client
for just about every device you can imagine: Windows, Mac, every major Linux
distribution, iOS, Android, etc, etc &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:2&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:3&#34;&gt;
&lt;p&gt;They even have an apt repository specifically for raspbian builds, so it’s
super easy. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:3&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:4&#34;&gt;
&lt;p&gt;Lots of people seem to treat “magic” as a good thing in software. I think
it’s the absolutely opposite. I want to know how something works, not for it to
be inscrutable! &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:4&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:5&#34;&gt;
&lt;p&gt;This is entirely optional, of course, you can just let each device do
their own thing as well. &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:5&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li id=&#34;fn:6&#34;&gt;
&lt;p&gt;I don’t think this is likely, but it’s one of my biggest concerns about
using “social” logins. More-or-less all of these services can arbitrarily ban
you with no recourse; why would you trust them as keyholders to other things? &lt;a class=&#34;footnote-backref&#34; href=&#34;#fnref:6&#34; role=&#34;doc-backlink&#34;&gt;↩︎&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;/div&gt;
</content>
    </entry>
    <entry>
        <title>Docker reverse proxying, redux</title>
        <link href="https://chameth.com/docker-proxying-redux/"/>
        <updated>2022-12-06T00:00:00Z</updated>
        <id>https://chameth.com/docker-proxying-redux/</id>
        <content xml:lang="en" type="html">&lt;p&gt;Six years ago, &lt;a href=&#34;https://chameth.com/docker-automatic-nginx-proxy/&#34;&gt;I described&lt;/a&gt;
my system for configuring a reverse proxy for docker containers.
It involved six containers including a key-value store and a webserver.
Nothing in that system has persisted to this day. Don’t get me wrong – it
worked – but there were a lot of rough edges and areas for improvement.&lt;/p&gt;
&lt;h3 id=&#34;microservices-and-their-limitations&#34;&gt;Microservices and their limitations&lt;/h3&gt;
&lt;p&gt;My goal was to follow the UNIX philosophy of “do one thing and do it well”.
Unfortunately, that doesn’t really work when applied to network services that
have to interact with one and other. UNIX tools are built upon a common file
system and simple data passed over STDIN. Microservices don’t have that
shared foundation. You could make one:
companies that use microservices in anger often have a team that deals with
the “developer experience” of creating and using
microservices. But as a solo developer that’s not something I wanted to
spend my time doing.&lt;/p&gt;
&lt;!--more--&gt;
&lt;p&gt;This became very apparent when trying to debug issues. In the UNIX world,
if your series of commands piped together does something strange you can simply
echo the data at various stages. Not so much when that data is flying around
on a network, going into and out of things like etcd. Trying to figure out why
a certificate hadn’t been acquired was a case of searching through logs from
four containers, none of which had particularly good logging.
There are many ways to get insight into what’s
happening with microservices but, again, that’s not something I wanted to do myself.&lt;/p&gt;
&lt;p&gt;Over time, and with experience in other projects, I came to realise that
microservices only really make sense if you’re unable to deploy a monolith.
For tech companies this naturally happens when different teams
contribute to the same service: splitting it out into smaller services that
are wholly owned by individual teams makes sense. For solo developers,
that never happens. You can still gain the other benefits
of microservices – such as code separation and having clearly defined APIs –
by sticking to certain coding standards.&lt;/p&gt;
&lt;h3 id=&#34;proxy-inconveniences&#34;&gt;Proxy inconveniences&lt;/h3&gt;
&lt;p&gt;As well as being unhappy with the microservice nature of the solution,
I wasn’t pleased with nginx. If you requested an unknown domain, nginx
would use the first server block in its config to serve a response, instead
of sending an “unrecognised name” alert as I wanted. It was a minor issue, but
it irked me.&lt;/p&gt;
&lt;p&gt;So from nginx I switched to haproxy. It has a &lt;code&gt;strict-sni&lt;/code&gt; option when configuring
TLS connections which makes it behave properly. It also performs a lot better for
this type of workload than nginx. All was well for a while, but then I started getting alerts
that requests were occasionally failing. I couldn’t reproduce the issue, but
my nightly jobs to build and push containers managed to hit it nearly every
night, causing them to fail.&lt;/p&gt;
&lt;p&gt;After some investigation, I found that the haproxy developers had refactored
the header parsing code, and
neglected to properly reset flags when multiple requests were sent over the same
connection. There was a patch, but it wasn’t released. No problem, I thought,
I’ll just cherry-pick it onto the last release… Except that haproxy use
Git in the most convoluted manner I’ve ever seen – they have one
repository per release. This makes it harder to patch, but it also made me question
whether I trusted them to ship stable software: there were no tests for
the header parsing code (which is both fundamental and finicky,
the perfect target for tests), the source code management was weird, and they didn’t
seem in any rush to patch this bug.&lt;/p&gt;
&lt;p&gt;Not long after that issue, &lt;a href=&#34;https://greg.holmes.name/&#34;&gt;Greg&lt;/a&gt; managed to
encounter another bug where haproxy returned a 500 error whenever the
upstream server replied with a particular, perfectly valid, header.
The die was cast – it was time to move to something else.&lt;/p&gt;
&lt;h3 id=&#34;not-invented-here-syndrome&#34;&gt;Not Invented Here syndrome&lt;/h3&gt;
&lt;p&gt;Looking for a new solution, there were many more options than
back in 2016. I’m still convinced, however, that anything
exposed to the Internet should not have access to run docker containers.
It’s the modern equivalent of running a CGI script as root. That
single requirement eliminates most off-the-shelf solutions.
What do you do when nothing quite meets
your specific requirements? You make something yourself! My new solution has two
components: &lt;a href=&#34;https://github.com/csmith/dotege&#34;&gt;Dotege&lt;/a&gt; and
&lt;a href=&#34;https://github.com/csmith/centauri&#34;&gt;Centauri&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Dotege is a replacement for the
microservices that monitored containers and obtained certificates. It’s fundamentally
a templating engine - whenever the containers change, it evaluates a
template and saves the result to disk. The template has access to details about
the containers, their labels, ports, and so on. Dotege can also obtain certificates
from Let’s Encrypt, and raise a signal against another
process whenever the template or certificates change. I used this to generate
the configuration and certificates used by haproxy for a while, and more recently
changed the template so that it works for Centauri.&lt;/p&gt;
&lt;p&gt;Centauri is my own reverse proxy. It’s configured using a simple text file
and can also obtain certificates from an ACME provider. It doesn’t serve static
content, has no knowledge about docker, and avoids the other bells and
whistles that adorn most reverse proxies. It also has good test
coverage to ensure that I don’t, say, accidentally break header parsing.&lt;/p&gt;
&lt;p&gt;As a software engineer I enjoy writing software, but I also enjoy running
simple, easy to understand software. That’s what I’ve achieved here: it’s
very easy to identify where the problem is if anything goes wrong, both are small
Go programs rather than vast sprawling C
monstrosities, and their interaction is primarily through a file written to disk
that can be inspected or edited as needed.&lt;/p&gt;
</content>
    </entry>
</feed>
