Most programming – and sysadmin – problems can be debugged in a fairly straight forward manner using logs,
print statements, educated guesses, or an actual debugger. Sometimes, though, the problem is more elusive.
There’s a wider box of tricks that can be employed in these cases but I’ve not managed to find a nice overview of
them, so here’s mine. I’m mainly focusing on Linux and similar systems, but there tend to be alternatives
available for other Operating Systems or VMs if you seek them out.
Networking
tcpdump
tcpdump prints out descriptions of packets on a network interface. You can apply filters to limit
which packets are displayed, chose to dump the entire content of the packet, and so forth.
One thing that always confuses me with Docker is how exactly mounting volumes behaves. At a basic level it’s
fairly straight forward: you declare a volume in a Dockerfile, and then either explicitly mount something there
or docker automatically creates an anonymous volume for you. Done. But it turns out there’s quite a few edge
cases…
Changing ownership of the folder
Perhaps the most common operation done on a Docker volume other than simply mounting it is trying to change
the ownership of the directory. If your Docker process runs as a certain user you probably want the directory to
be writable by that user.
For the past few years I’ve been taking part in Eric Wastl’sAdvent of Code, a coding challenge that provides a 2-part problem each
day from the 1st of December through to Christmas Day. The puzzles are always interesting — especially as they
get progressively harder — and there’s an awesome community of folks that share their solutions in a huge variety
of languages.
To up the ante somewhat, Shane and I usually have a little informal
competition to see who can write the most performant code. This year, though, Shane went massively overboard and
wrote an entire benchmarking suite
and webapp to measure our performance, which I took as an invitation and personal challenge to try to beat
him every single day.
For the past three years I’d used Python exclusively, as its vast standard library and awesome syntax lead to
quick and elegant solutions. Unfortunately it stands no chance, at least on the earlier puzzles, of beating the
speed of Shane’s preferred language of PHP. For a while I consoled myself with the notion that once the
challenges get more complicated I’d be in with a shot, but after the third or fourth time that Shane’s solution
finished before the Python interpreter even started I decided I’d have to jump ship. I started using Nim.
DNS-over-TLS is a fairly recent specificiation described in RFC7858, which enables DNS clients to communicate with servers over a
TLS (encrypted) connection instead of requests and responses being sent in plain text. I won’t ramble on about
why it’s a good thing that your ISP, government, or neighbour can’t see your DNS requests…
I use an EdgeRouter Lite from Ubiquiti Networks at
home, and recently configured it to use DNS-over-TLS for all DNS queries. Here’s how I did it.
Installing unbound
Out of the box, the ERL uses dnsmasq to service DNS requests from local clients. To get
DNS-over-TLS support I switched to using Unbound, an open source DNS resolver
with support for many modern features such as DNSSEC and DNS-over-TLS.
I was thinking about switching DNS providers recently, and found myself whoising random domains
and looking at their nameservers. One thing lead to another and I ended up doing a survey of the nameservers of
the top 100,000 sites according to Alexa.
Most popular providers
The top providers by a large margin were, unsurprisingly, Cloudflare and AWS Route 53. Between them they
accounted for around 30% of the top 100k sites.
