Chameth.com - posts like apple-google-aligned-incentives, monitoring-power-with-wemo, thinking-more-about-backups, why-you-should-be-using-https

Thinking more about backups

2025-11-01T00:00:00Z

The Backblaze logo

Almost a year ago I wrote about how I do backups with Restic and Hetzner. That system has been ticking along well ever since, but recently I had some… thoughts. These backups are all well and good if I accidentally delete a file, or a database gets corrupted, or something, but there are two glaring issues:

Firstly, I’m backing up my Hetzner server to Hetzner cloud storage. If something happens to Hetzner — or my Hetzner account — then all my eggs go down with that basket. Obviously Hetzner are a big organisation and aren’t likely to just vanish overnight, but I’m less confident about my account. Could a false abuse report get it suspended? What if the UK passes even more dumb laws and Hetzner decide it’s easier just to not do business with people here? This is the same sort of concern I have about Google accounts: if you have half of your life in Google Drive and Google Mail, what happens if you comment on a YouTube video, get flagged by an AI moderation process, and your account gets suspended? It’s probably not very likely, but these are things my brain likes to dwell on.

Secondly, the credentials to access the backups sit on each machine that is backed up. If someone malicious gained access to the machine, they’d also have access to delete or tamper with all the backups. It feels a little silly that the same attack could take down both the originals and the backups. There’s no way to avoid that with Hetzner’s S3 implementation, as far as I can tell.

Exploring options

I toyed with the idea of making local copies of the backups, but the only way to avoid the same problems would be to keep them offline and do a manual copy every now and then. I didn’t really want to do that, and was concerned that if I did a monthly offline backup then I stood to lose up to a month of data in the worst case.

I then looked around at other S3 providers. Amazon’s glacier offering is tempting due to its very low storage costs, but you pay for that if you ever want to restore anything. There are also lots of weird pricing edge cases around moving data between storage classes, minimum file sizes, and so on. A much better option is Backblaze’s B2 product. Their pricing is much more straight-forward, and they have an interesting feature that’s particularly useful in this case: lifecycle rules. Coupled with the ability to create API keys that don’t have access to delete files (just “hide” them), this allows for what’s effectively an append-only store.

This works more-or-less out of the box with Restic. Joseph Price has a guide that goes into the setup in a bit more depth. Basically, whenever Restic would delete a file (e.g. during a “forget” or “prune” operation), it instead gets hidden and is only deleted when the B2 lifecycle rules decide it should be. I’ve kept the existing Hetzner S3 backups for now, and just added an extra step to the end of my script: a simple restic copy and a restic forget. B2 actually works out cheaper than the Hetzner storage, as they don’t bill you for a minimum of 1TB storage; my current usage is around $3/month. Not a bad price for some extra peace of mind!

How tech companies failed to build the Star Trek computer

2025-07-16T00:00:00Z

A computer core room on the Enterprise-D

In most Star Trek series, the ship or station computer is ever-present in the background, waiting to be called on by the main characters¹. It nearly always does exactly the right thing, and there’s little limit to the functions it can perform. Take this mundane example from DS9:

KIRA: Computer, establish link with the Bajoran Medical Index for the Northwestern District.
COMPUTER: Link established.
KIRA: Access all information on Doctor Surmak Ren.
COMPUTER: There are no records matching that name.
KIRA: Try the Northeastern District, same search.
COMPUTER: Doctor Surmak Ren, currently serving as Chief Administrator of the Ilvian Medical Complex.
KIRA: Computer, open a channel to the Ilvian Medical Complex. Administrator’s office.

The computer is doing some kind of networking to a database only identified by name. It does a search and summarises the lack of results. It then repeats the process with another database, and succinctly announces the results. Finally, it opens a communication channel to a specific room in a facility, based only on its name.

This whole interaction is remarkably boring². Kira doesn’t have to know any URLs or API endpoints, or what protocol she wants to use. She doesn’t have to open a specific app and then login and then try the query again. She just says what she wants and the computer does it.

It seems like this should be one of the most easily obtainable bits of sci-fi wizardry with our current technology. We have multiple massive companies throwing lots of money at digital assistants, LLMs that are improving at an insane rate, but we’re somehow not even close to the usability or usefulness of the Trek computers. What gives?

Boring is, well, boring.

Larry Page once said something that might help explain it:

The Star Trek computer doesn’t seem that interesting. They ask it random questions, it thinks for a while. I think we can do better than that.

This is the same Larry Page that founded Google, whose mission statement is “to organize the world’s information and make it universally accessible and useful”. Of all people, surely he should find an omnipresent computer that can answer ‘random questions’ interesting?! It seems like it should be the epitome of Google’s mission!

Google’s “better than that” seems to have been to stuff LLMs into every product they can, even when you don’t want them there. Even when they’re worse than the normal content they displace. These things look exciting when they’re part of a scripted demo at Google I/O, but they fall flat and just get in the way when they’re exposed to the reality of day-to-day use.

The Star Trek computer is the opposite: it isn’t snazzy, but it is genuinely useful. That means it’s not an attractive target for the company execs who want marketing opportunities, and it’s not appealing for engineers who need to demonstrate “impact”. But even if Google did try to make the Trek computer, there are other problems…

Assistants need to be free

A significant amount of tech companies’ business models currently revolves around trapping users in walled gardens. They want you using their ecosystem; that way they get more data from you, and you’re more likely to spend more money on their other offerings that work together. There’s barely any incentive to allow any kind of interoperability with other platforms outside carefully contracted integrations.

I remember trying to help a family member move their photos from iCloud to Google Photos. At one point they turned around and said, exasperated, “why is this so hard? Aren’t they both in the cloud?!”. It’s easy to dismiss that as someone who hasn’t quite grasped the fundamental idea that “the cloud” is just someone else’s computers, but that’s not the whole story. There’s no reason why there shouldn’t be a quick and easy transfer: both services already allow uploading and downloading, there’s just no incentive for the companies involved to make it so³.

These kinds of misaligned incentives and walled garden business models cause even more problems when it comes to digital assistants. Siri is basically never going to be able to interact with, say, your Google Drive; ~~Bard~~ Gemini is never going to be able to send a message via iMessage. Even when there are appropriately blessed interactions, they’re so clunky. Can you imagine Captain Picard saying “Computer, ask the turbolift skill to take me to deck 5”?

Someone else’s computer

Software issues aside, there’s still a key difference between the Star Trek computers and our current batch of digital assistants: where they run. The Trek computers are all housed within the ship or station they serve; they can connect elsewhere to gather information, but they run entirely independently. If they go wrong, a local engineer can go in and fix things. While some of our assistants may have physical hardware in your home, they don’t work without a vast cloud apparatus behind them. If your Internet connection fails, they become paperweights. If the company running them decide to remove some functionality you depend on, you have no recourse.

That kind of helplessness isn’t limited to assistants, either. There’s a rapidly growing trend of being unable to modify or repair hardware you fully own and control. Part of this is just that they’re becoming more complex: it’s a lot harder to replace a microchip than a gear, but companies are also going out of their way to make it more difficult for users through draconian DRM regimes⁴ and aggressive intellectual property enforcement. If the US Navy can’t repair their own equipment because a corporation says so, what hope do consumers have?

We’re approaching a point where you don’t actually own anything. Software is cloud and subscription based, hardware is unrepairable. Even cars can be remotely updated and have features added or removed. The Federation wouldn’t allow a third party control over their ships⁵, so why are we so happy to put up with it in everything we consume?

A small ray of hope?

The most promising way of tackling all of these problems is through legislation. The EU’s Digital Market Act is an attempt to force ‘gatekeepers’ like Google, Apple and Meta, to allow third-party access to their services. It seems like a pretty reasonable approach, but the tech companies are unsurprisingly resisting it. Apple in particular have gone out of their way to refuse to comply, and when forced to do so have limited the functionality to people in Europe. Still, the DMA is a promising start, and if similar legislation is introduced (and robustly enforced) elsewhere it might start forcing companies to behave a bit better.

There are also smaller companies that actually do the right thing. Framework make laptops that are user-serviceable; Fairphone do the same for mobile phones. Smaller software companies provide useful, open APIs. The average person on the street will probably have never heard of these, unfortunately, but they do still exist. Maybe as the bigger tech companies tighten the screws more, people will turn to alternatives like this? Or maybe we’ll just keep accepting that our computers work for everyone but us?

Unless, of course, the computer is playing the role of the episode’s MacGuffin and has contracted space-computer-COVID or something, then it’s a lot less in-the-background. ↩︎
It’s almost like it only exists to move the plot along. ↩︎
You can generally export your data, thanks to a combination of legislation and efforts like Google’s “Data Liberation Front”, but I’ve never seen an export format that could then just be imported into an equivalent commercial product. ↩︎
Oh, you’ve changed the screen on your iPhone? Better hope it can do the secret handshake with the Apple hardware. ↩︎
I think there might actually have been an episode where that did in fact happen. We’ll just ignore that as a plot contrivance. ↩︎

Home Automation Without the Megacorps

2025-05-21T00:00:00Z

I first experimented with home automation in 2016, by picking up a Samsung “SmartThings” hub. It was terrible. The UI to configure things was slow and clunky, firmware updates were applied whether you wanted them or not, and everything stopped working if their cloud services stopped. You were also locked into whatever integrations they deigned to support, of course. After that broke for the umpteenth time I scaled back and for years the closest I got to home automation was a couple of Hue bulbs.

Recently I’ve been building it out again, though. This time using off-the-shelf components that interop using Zigbee, open-source software, and some code I wrote myself. It’s great; it runs entirely locally and has had basically zero downtime. The Zigbee ecosystem lets me integrate all sorts of things without having to spend lots of money on “smart” alternatives. I think I’ve spent less on this incarnation than I did on the original SmartThings hub all those years ago (even without adjusting for inflation!).

My current setup

I run everything on a Raspberry Pi 4, with a Sonoff USB Zigbee adapter based on the CC2652P chipset¹. Interfacing with the Zigbee stack is handled by zigbee2mqtt (z2m for short), an open-source project that basically bridges your devices to an MQTT broker². When a device reports some data, it will send a new message over MQTT; when you want to make a device do something you just post a message back. It’s incredibly lightweight, but supports a huge array of devices out of the box. And as it’s just using MQTT, it’s trivial to integrate with other software or build on top of.

I know a lot of people building out this kind of home automation use Home Assistant, but I don’t get on with it terribly well. It felt incredibly sluggish, and the entire project is just a bit… much. I want a binary or package or docker image I can just run, not an entire operating system. That’s not how computers are meant to work! In contrast, z2m is simple to set up, super light weight and responsive.

Anyway. z2m exposes Zigbee devices over MQTT, so I wrote some code in Go to connect to the MQTT broker, and listen to the messages. It’s grown a bit beyond this now, and I’m skipping some boring bits like error handling and JSON parsing, but at first I had something like:

for {
    message, topic, err := c.ReadSlices()
    
    if string(topic) == "zigbee2mqtt/desk-button" {
        if strings.Contains(message, "single") {
            c.Publish(nil, []byte("{\"state\": \"TOGGLE\"}"), "zigbee2mqtt/room-lights/set")
        }
    }
}

I think I actually made this harder to read by trying to simplify it here, but hopefully you can follow that it was relatively straight forward to listen for a particular action to happen (in this case me “single”-pressing on the device called “desk-button”) and then make another device do something in response (toggling the state of the “room-lights” device).

So what actually are these devices? At present in z2m I have the following:

Some generic buttons
An air quality sensor (USB powered), and a separate temperature sensor (battery powered)
A light switch
Some 240V relay modules, that I use to turn non-smart devices on and off
Some USB relay modules, for the same purpose
Some “smart” plugs that I mostly use to monitor power usage
A motorised blind roller

I think all of these are either Tuya or Moes branded, and were all sourced from AliExpress. You can get the same things from Amazon or elsewhere, but they tend to be 3-4x more expensive for the same product.

One of the cool things about Zigbee devices is that the powered ones work together to create a mesh network, so you don’t have to worry about network repeaters or signal strength like you do with Wi-Fi networks. z2m even makes a map showing the connections:

The network map produced by z2m. You can see how all the powered devices form a mesh that the lower power ones can connect to.

Some interesting automations

A Zigbee relay spliced into the power cable for the fan

So what do I actually do with all of this? Some of it is just what I’ll call “laziness automation”: I want to be able to turn on the lights without getting up and walking to the light switch. So there’s a Zigbee button on my desk that does it. Then there are some less manual automations: my blinds are automatically closed at sunset³, for example. It feels a bit trivial, but it’s surprisingly nice just not to have to think about that at all.

Another nice quality of life automation is the light I have above my monitor. It’s USB powered, and I now have it going through a Zigbee USB relay that lets it be toggled on and off. Then I have a small agent running on my desktop that turns the light on when the computer is unlocked, and off when it locks or shuts down. I keep meaning to make a “film mode” that detects when I’m watching a film, and turning all the lights and blinds down, but I haven’t got around to it yet.

The most complex automation is probably for a window fan. It’s not smart in any way, so I cut the power cable and inserted a Zigbee relay. The whole thing is housed in a little project box to keep it secure. The relay basically acts as a switch: the live wire leading to the fan runs to the “normally open”⁴ contact, while the upstream power goes to “common”. When the relay is told to turn the fan on, the two get connected and the fan gets power. Oh, it’s also not actually a window fan, but I designed and 3D printed some adapters so that it sits right in the window.

So now we have a window fan that can be turned off and on automatically. But I don’t want to have to press a button to do that. I’m lazy, remember? Instead, I made it so that the Go code constantly monitors the temperature reported by the air quality sensor, and queries the free OpenWeatherMap API to get the rough temperature outside. It can then turn the fan on if the room is too hot, and outside is cool enough to make a difference (there’s no point in blowing hotter air in!).

A graph of room temperature, showing the effect of the fan being turned on and off

I don’t want it to do that all the time though. The room doesn’t need to be cooled if I’m not around. There are various Zigbee presence sensors you can get, but the cheaper ones seem to be of dubious quality. Instead, I wrote some code to guess whether I’m present. It uses the state of the monitor light as a proxy for “is the computer in use”, and then makes some guesses based on the last time the computer was used and the current time (if I turn the computer off at midnight it probably means I’m going to sleep; if I turn it off at 2pm it probably means I’m going out somewhere).

You can see in the graph that the fan has a decent impact on temperature. I coded it to cool the room to 19 degrees, but then not turn on again until it was above 20. This prevents it flip-flopping on and off constantly. You can clearly see the pattern in the graph, as it actively cools and then slowly the room warms back up. This arrangement is much better than being woken up at 5am because it’s painfully cold. Trust me.

Bonuses: metrics and 3D printer

One of the things my custom Go code does is collate all the various stats reported by the Zigbee devices, and inserts them into a VictoriaMetrics database. I originally hosted this on the Pi itself, and it performed fine, but I’ve since moved it onto a server so that I can use it for some other things as well.

I set up Grafana to point to VM, and can create dashboards showing power usage, what devices are turned on when, and a bunch of environmental conditions. This also makes it easy to spot how good the data coming from the devices are. For example, the air quality sensor reports a figure for the amount of Carbon Dioxide in the air, as well as the amount of Volatile Organic Compounds (VOCs). The graphs are basically identical, but on a different scale. It turns out this particular device has no actual way of detecting CO₂, so it just “calculates” it from the VOCs figure. Useful to know if you want to actually use that data.

Recently I came across a go library for interacting with Bambu Lab printers, so I’ve also hooked that into my automation. It exports metrics about the printer, so I now have a way of seeing what’s going on when I’m not physically present⁵. I even added a HTTP endpoint (exposed over Tailscale) that shows pictures from the built-in camera.

On the automation front, I’ve made it automatically turn off the printer’s light when it finishes a print, and also added push notifications via PushOver whenever the state changes. No longer will I be sat in another room blissfully unaware it ran out of filament seconds after I walked out the door!

Was it worth it?

These days you could probably just buy some kind of ‘smart home’ hub that works well enough, and do most of what I’ve done with a lot less effort and no coding. I’m still of the opinion that for something so essentially local, it should itself be managed entirely locally. I don’t trust companies like Google or Amazon not to kill their products, or change or remove an API I rely on.

Obviously the “Not Invented Here” approach of coding everything yourself doesn’t suit everyone, but as someone who enjoys coding and enjoys having things work just so it works very well for me.

It turns out the chipset is important. I initially used a similar Sonoff dongle that used a CC2531 chipset, and its performance wasn’t great. It often failed to pair with new devices, and dropped links to existing ones. The zigbee2mqtt docs do explicitly advise against the CC2531 chips for that reason. At the time they recommended CC2652Ps, so that’s what I went with. If you’re starting new I’d go with whatever their latest recommendation was. ↩︎
MQTT is basically a network protocol for publishing and subscribing to arbitrary messages. A broker is the thing that sits in the middle and routes the messages. I use Mosquitto but any will do. ↩︎
They originally also re-opened at sunrise but that was a terrible mistake. Who knew the sun rose so early?! ↩︎
“open” meaning “there’s a gap so it doesn’t work” not “open for business”. This confusion in terminology also extends to drawbridges. ↩︎
I use the printer in LAN mode, which means there’s no way to monitor it from a phone, even if you’re connected to the same network still. ↩︎

Simple backups with Restic and Hetzner Cloud

2024-12-06T00:00:00Z

Restic’s mascot, who’s dual-wielding umbrellas to save you from a rainy day.

I have a confession: for the past few years I’ve not been backing up any of my computers. Everyone knows that you should do backups, but actually getting around to doing it is another story.

Don’t get me wrong: most of my important things are “backed up” by virtue of being committed to remote git repositories, or attached to e-mails, or re-obtainable from the original source, and so on. I don’t think any machine failing completely would be a disaster for me, but it would certainly be a pain.

This week I finally got around to doing something, and it ended up being a lot more straight forward than my previous forays into backup-land.

Restic

After soliciting a few opinions, the choice of backup software came down to either Borg or Restic. I’m pretty sure either would have done what I want, but I leaned towards Restic for a few reasons: it has a more informative website, it’s written in Go rather than Python¹, and Borg seems to be transitioning between major releases at the moment².

The way Restic works is pretty simple: you initialise a ‘repository’, and can then call restic backup /some/path and it’ll get backed up to the repository. Restic handles keeping different backups separate, and only sending data that’s changed, and deduplicating, and so on. You basically point it at a thing you don’t want to lose, and it sorts it out for you. Perfect.

There are equally straight-forward commands for removing old snapshots (restic forget) and verifying backups (restic check). One of the nice things about modern backup solutions is they support a whole range of backends. I was originally going to spin up a small VPS to host my backups, but noticed that Restic supported S3-compatible stores…

Hetzner Cloud Object Storage

I host my servers with Hetzner, and was going to use them to spin up a VPS as well. Despite the “Cloud” branding on a bunch of products, they offer reasonable prices and good service. A couple of months ago, they started offering S3-compatible object storage.

The pricing isn’t totally straight forward, but for continuous use the “free quota” amounts to 1TB of storage and 1TB of egress a month. Ingress is free, as is traffic within the eu-central region (where all my servers are). That quota is only awarded when you pay the “base price”, though, which is €4.99 a month. So basically it’s €5 a month for 1TB of storage and enough egress to fully restore every single byte. That’s better value than any VPS I can find, much cheaper than Amazon S3, and about the same as Backblaze B2.

Getting them to work together

Now you’d think making the tool that supports S3-compatible object storage work with your S3-compatible object storage would be easy, right? Well not quite. The library Restic uses to deal with S3 backends has some strange logic for figuring out the bucket name given a URL. It doesn’t quite seem to work right, though…

Hetzner buckets have URLs like s3://bucketname.hel1.your-objectstorage.com³, but just passing that to Restic gives an error that the bucket is not specified. The docs mention there’s an advanced option to make it use the virtual host for the bucket name: -o s3.bucket-lookup=dns. But that… also doesn’t work. What I ended up doing was specifying the URL as s3://hel1.your-objectstorage.com/bucketname, and also passing in the dns option. The library then seems to muddle its way back to a real, working URL. I’m not sure why it works like this: maybe it’s just a weird aspect of S3 that I’m oblivious to?

The next fun part is that you can configure Restic entirely by using environment variables, except for that -o s3.bucket-lookup=dns argument. That has to go on the command line. I ended up making a little wrapper script to invoke Restic correctly:

#!/bin/sh

# The password restic will use to encrypt your data. You should generate
# something nice and secure.
export RESTIC_PASSWORD=repo-password

# The path to the repository where restic will save the backup. In our case
# this takes the form `s3:<endpoint>/<bucket>`. Your endpoint might be different
# to mine depending on the region your bucket is in.
export RESTIC_REPOSITORY=s3:hel1.your-objectstorage.com/bucket-name

# The access and secret key generated in the 'S3 credentials' section of the
# Hetzner Cloud Console
export AWS_ACCESS_KEY_ID=hetzner-access-key-id
export AWS_SECRET_ACCESS_KEY=hetzner-access-key

# Pass any arguments on to the restic command, along with the magic
# `s3.bucket-lookup` option we need to resolve the S3 URL properly.
exec restic -o s3.bucket-lookup=dns "$@"

Then in the backup script I just alias restic to use the script:

#!/bin/bash

set -eu

# This means any time we use `restic` below, we'll actually execute our special
# script which supplies all the env vars and arguments needed to find the
# repository. Make sure the path matches where you saved the script!
alias restic='~/.bin/restic'

# Actually do the backup. Each directory in the list below will be backed up
# separately.
dirs=(
	"/some/path/to/backup/"
	"/some/other/path/"
)
for i in "${dirs[@]}" 
do
	echo $i
	(cd $i && restic --verbose backup .)
done

# Prune our snapshots. You can tweak the numbers here. Run with `--dry-run` to
# see what effect any changes would have before actually committing to them. 
restic forget --keep-daily 7 --keep-weekly 10 --keep-monthly 24 --keep-yearly 10

The only other notable thing here is that the script changes into the directory to be backed up. If you give Restic an absolute path, it will create a new snapshot if the metadata of any folder in the path changes, which is not what I want.

If it wasn’t for the S3 URL issues, the whole thing would’ve probably taken me about half an hour. That’s including setting up the object storage, installing Restic, and so on. It’s painfully easy. Why didn’t I do this three years ago?!

Addendum: a step-by-step guide

If you want to do this yourself, here’s a quick step-by-step guide:

Log in to the Hetzner Cloud Console, and create a project.
On the “Object Storage” tab, create a new bucket. Note the name and the endpoint.
On the “Security” tab, go to “S3 Credentials” and generate new credentials. Note down the access key and the secret key.
Copy the first shell script above, fill in the password (you can pick!), endpoint, bucket name, access key and secret key.
Run the script with the init argument (e.g. ~/.bin/restic init). This will create a new repository, and only needs to be done once even if you backup multiple machines.
Copy the second shell script above, making sure the restic alias points at the script you saved in step 4. Change the list of directories to whatever you want to backup.
Schedule the script to be run automatically, using crontab or systemd timers or however you prefer.

To check everything is working, you can use the snapshots subcommand to see a list of saved snapshots. You might also want to try to restore a snapshot using the restore subcommand.

I’m not trying to be a language snob, but given the choice between two otherwise equal projects one in Go and one in Python, I’ll take the Go one. I know I’m not going to have weird library issues down the line, and I’m much more comfortable rummaging around the source. ↩︎
With a “Don’t use this in production!” notice on the shiny new version. ↩︎
Aside: I really hate URLs like that. What is with the trend for completely generic domains divorced from the service they’re a part of? ↩︎

Apple, Google and aligned incentives

2020-10-17T00:00:00Z

The HTC Dream, the first phone released running Android.

For the past decade I’ve exclusively used Android phones. I got the HTC Dream (aka the T-Mobile G1) shortly after it came out, and dutifully upgraded every 1-2 years. In that timespan I used Android as the basis for my Master’s Thesis, took a job on the Android team at Google, and eventually became a contractor specialising in Android app development. So when I switched to using an iPhone earlier this year a few people were surprised¹.

The good old days

When Android was announced in 2007 – alongside the formation of the Open Handset Alliance – it was positioned as a bastion of openness: it would be built on open standards and the operating system would be open source. At the time iPhones were strongly coupled to iTunes and Apple was exercising strict control over what app developers could do.

When the HTC Dream was released it lived up to expectations. You could write apps for it without shelling out for a Mac! You could get root access, and it was running Linux under the hood! A whole ecosystem of custom firmwares and bootloaders started to appear, thanks to the open source nature of the OS. It shipped with some Google apps, but they were just normal apps that served as examples of what could be done.

After the Dream came a line of Nexus devices. These were Android’s flagship devices, designed to show off what a good Android phone should look like. Both the hardware and software releases tended to feature interesting, useful upgrades. Some of the original open source apps were replaced with closed source, Google proprietary ones, but that was OK - the open source versions lived on in the open source project as examples of what you could do. The devices allowed flashing custom firmware, and the OS source was always released… eventually.

The downfall

The Pixel and Pixel XL

Over time, Android has got less and less free. The Nexus line of phones gave way to the Pixel range, which got rid of the clearly demarcated border where Android ended and Google began. The Android 11 highlights list is dotted with sections that say “On Pixel devices…”, and every single one is a software feature that could be implemented on any device, but Google have decided to keep it proprietary instead of releasing it as part of the open source platform.

At the same time more and more functionality has been added to Google Play Services. This was originally a shared location for Google specific services - in 2012 it merely handled some Google+ functionality and dealing with OAuth for Google accounts. These days it contains a huge swathe of Google services, as well as platform functionality such as push notifications, barcode scanning, geolocation, and so on. Google Play Services isn’t part of the Android Open Source Project, and is only available under license from Google. You can’t really have a device without these functions, so as a manufacturer you have the choice between agreeing to whatever terms Google requires² or spending an awful lot of development time creating an alternative.

While these are fairly abstract arguments about what a free platform should look like, during the same period there has been a marked restriction in how you can actually use Android devices – both as an end-user and as a developer. iOS has always had very stark restrictions on what apps can do in the background; Android started its life allowing pretty much anything, like any good general purpose computing device. This inevitably lead to lots of apps doing lots of things that ranged from stupid to mildly suboptimal, creating a tragedy of the commons amongst apps. The victim was the phone’s battery life, and Google’s solution was a progressive series of restrictions on what apps can do and when. This includes limits on when push notifications are delivered to a device, how frequently apps can wake up, and so forth. Some of these can be bypassed by the end-user, but not all, and the process is fairly cumbersome.

Aligned incentives

From my point of view, Android and iOS are now pretty much in a similar place. They’re not general purpose computers, but app consoles: much like games consoles they consist of hardware and software that the user lacks control over but accepts in order to access the library of apps/games.

Given there’s no clear winner between them in terms of hardware and software, my decision came down to a more holistic question: how aligned are their incentives to my own? Apple is a product company: they make their money by producing shiny things that people want to purchase; Google is an advertising company: they make their money by using my personal information to show me targeted adverts.

For Google, Android was originally a strategic move to ensure that Apple couldn’t dominate the mobile web, and by extension the revenue from ads. As a company, there’s nothing to really push them forward in any particular direction other than one that facilitates advertising. Of course, Google employs tonnes of good people who want to do good things which counterbalances this, but I’d still prefer to deal with a company that has an intrinsic motivation to do things I want them to do, rather than one forced to by regulation, custom, or the good intent of their employees.

So the decision became obvious: if I have no strong opinions about the software and hardware, Apple is the clear winner because their incentives are a lot better aligned to mine.

Image credits

Photo	Creator	Licence	Source
HTC Dream	Akela NDE	CC BY-SA 3.0	Wikimedia
Pixel and Pixel XL	Maurizio Pesce from Milan, Italia	CC BY 2.0	Wikimedia

Or, at least, politely feigned surprise. ↩︎
And Google’s terms were particularly onerous: as well as requiring Chrome and Google Search to be preinstalled, they prevented manufacturers from selling any devices powered by alternative versions of Android - e.g., Samsung wouldn’t be allowed to sell a refrigerator that ran Amazon’s FireOS. The European Union handed Google a $5,000,000,000 fine for this anti-competitive behaviour, which Google are still in the process of contesting. ↩︎