The HTC Dream, the first phone released running Android.
For the past decade I’ve exclusively used Android phones. I got the HTC Dream (aka the T-Mobile G1) shortly
after it came out, and dutifully upgraded every 1-2 years. In that timespan I used Android as the basis for my
Master’s Thesis, took a job on the Android team at Google, and eventually became a contractor specialising in
Android app development. So when I switched to using an iPhone earlier this year a few people were surprised.
The good old days
When Android was announced in 2007 – alongside the formation of the Open Handset Alliance – it was positioned
as a bastion of openness: it would be built on open standards and the operating system would be open source. At
the time iPhones were strongly coupled to iTunes and Apple was exercising strict control over what app developers
could do.
Fuzz testing is a bit like the infinite monkey theorem, but instead of Shakespeare you get crashes.
Fuzz testing, if you’re not aware, is a form of testing that uses procedurally generated random inputs to see
how a program behaves. For instance, if you were fuzz testing a web page renderer you might generate a bunch of
HTML - some valid, and some not - and make sure the rendering process didn’t unexpectedly crash.
Fuzz testing doesn’t readily lend itself to all types of software, but it particularly shines in cases where
some kind of complex user input is accepted and processed in some way - like the aforementioned web page
renderer. I was recently adding a library to parse EXIF data to images to an Internet-facing service and realised
it was a perfect opportunity to do some fuzz testing. Even if I didn’t find any issues, I’d improve my confidence
that the library was safe enough to expose to the Internet.
Breaking my EXIF library
I wrote a quick harness to run go-fuzz on the library, and
gave it some pre-existing demo files as sample input. The way go-fuzz works is that it instruments your code and
then mutates the inputs to try to improve the coverage. For example, if I had some sample data that had an EXIF
tag with a value of 1 then go-fuzz might change it to a 2 and see if the code follows a different path. In most
cases it won’t but when it does, they tend to be very interesting cases.
I’m a huge fan of (software) containers. Most people I know fall in to one of two camps: either they also use,
and are fans of, containers, or they haven’t yet really figured them out and view them as some kind of voodoo
that they don’t really want or need.
I’m writing this short guide to explain a little how containers work - and how running something in a
container isn’t really that much different to running it normally - to hopefully enable more people in that
second group to give them a try. It’s aimed at people who have a fairly good grasp of how Linux works.
Containers are often mentioned in the same breath as VMs, which is not a helpful comparison or analogy. Think
of containers as standard units of software, much like how Intermodal containers are standard units of freight
transport across the world. When a company internationally ships goods in volume there isn’t a question about how
they’re packaged - they go in an intermodal container. The same container can be deployed on a freight train, a
lorry, or a ship. The haulage company doesn’t need to care what’s in the container because they’re completely
standardised. Likewise, with software containers you don’t really need to care about what’s inside: the software
you’re deploying could be written in Go, Python2, Python3, Bash, PHP, LOLCODE, or anything.
Obfuscating code is the process of modifying source code or build output in order to make it harder for humans
to understand. It’s often employed as a tactic to deter reverse engineering of commercial applications or
libraries when you have no choice but to ship binaries or byte code. For Android apps, ProGuard is part of the default toolchain and obfuscation
is usually only a config switch away.
I was recently working on an Android library written in Kotlin that my client wanted obfuscated to try and
protect some of their trade secrets that were included. Not a problem, I thought: it’s just a few lines of
ProGuard config and we’re away. Four hours and lots of hair pulling later I finally got it working…
Most programming – and sysadmin – problems can be debugged in a fairly straight forward manner using logs,
print statements, educated guesses, or an actual debugger. Sometimes, though, the problem is more elusive.
There’s a wider box of tricks that can be employed in these cases but I’ve not managed to find a nice overview of
them, so here’s mine. I’m mainly focusing on Linux and similar systems, but there tend to be alternatives
available for other Operating Systems or VMs if you seek them out.
Networking
tcpdump
tcpdump prints out descriptions of packets on a network interface. You can apply filters to limit
which packets are displayed, chose to dump the entire content of the packet, and so forth.
One thing that always confuses me with Docker is how exactly mounting volumes behaves. At a basic level it’s
fairly straight forward: you declare a volume in a Dockerfile, and then either explicitly mount something there
or docker automatically creates an anonymous volume for you. Done. But it turns out there’s quite a few edge
cases…
Changing ownership of the folder
Perhaps the most common operation done on a Docker volume other than simply mounting it is trying to change
the ownership of the directory. If your Docker process runs as a certain user you probably want the directory to
be writable by that user.
For the past few years I’ve been taking part in Eric Wastl’sAdvent of Code, a coding challenge that provides a 2-part problem each
day from the 1st of December through to Christmas Day. The puzzles are always interesting — especially as they
get progressively harder — and there’s an awesome community of folks that share their solutions in a huge variety
of languages.
To up the ante somewhat, Shane and I usually have a little informal
competition to see who can write the most performant code. This year, though, Shane went massively overboard and
wrote an entire benchmarking suite
and webapp to measure our performance, which I took as an invitation and personal challenge to try to beat
him every single day.
For the past three years I’d used Python exclusively, as its vast standard library and awesome syntax lead to
quick and elegant solutions. Unfortunately it stands no chance, at least on the earlier puzzles, of beating the
speed of Shane’s preferred language of PHP. For a while I consoled myself with the notion that once the
challenges get more complicated I’d be in with a shot, but after the third or fourth time that Shane’s solution
finished before the Python interpreter even started I decided I’d have to jump ship. I started using Nim.
DNS-over-TLS is a fairly recent specificiation described in RFC7858, which enables DNS clients to communicate with servers over a
TLS (encrypted) connection instead of requests and responses being sent in plain text. I won’t ramble on about
why it’s a good thing that your ISP, government, or neighbour can’t see your DNS requests…
I use an EdgeRouter Lite from Ubiquiti Networks at
home, and recently configured it to use DNS-over-TLS for all DNS queries. Here’s how I did it.
Installing unbound
Out of the box, the ERL uses dnsmasq to service DNS requests from local clients. To get
DNS-over-TLS support I switched to using Unbound, an open source DNS resolver
with support for many modern features such as DNSSEC and DNS-over-TLS.
I was thinking about switching DNS providers recently, and found myself whoising random domains
and looking at their nameservers. One thing lead to another and I ended up doing a survey of the nameservers of
the top 100,000 sites according to Alexa.
Most popular providers
The top providers by a large margin were, unsurprisingly, Cloudflare and AWS Route 53. Between them they
accounted for around 30% of the top 100k sites.
I’ve been spending some time recently setting up automated testing for our collection of Android apps and
libraries at work. We have a mixture of unit tests, integration tests, and UI tests for most projects, and
getting them all to run reliably and automatically has posed some interesting challenges.
Running tests on multiple devices using Spoon
Spoon is a tool developed by Square that handles distributing
instrumentation tests to multiple connected devices, aggregating the results, and making reports.
As part of our continuous integration we build both application and test APKs, and these are pushed to the
build server as build artefacts. A separate build job then pulls these artefacts down to a Mac Mini we have in
the office, and executes Spoon with a few arguments:
Spoon finds all devices, deploys both APKs on them, and then begins the instrumentation tests. We use two
physical devices and an emulator to cover the form factors and API versions that are important to us; if any test
fails on any of those devices, Spoon will return an error code and the build will fail.