tommorris.org

Discussing software, the web, politics, sexuality and the unending supply of human stupidity.


apps


Fault tolerance on the Web and the dark side of Postel's Law

I’ve been reading @adactio’s new book. Pretty much all I have read is great, and I highly recommend reading it.

But in the grand spirit of pedantic blogging, I take issue with this:

“Imperative languages tend to be harder to learn than declarative languages”

That tend to is doing a lot of work.

I think Jeremy is conflating declarative with fault tolerant here. HTML and CSS are declarative and fault tolerant. SQL is declarative and decidedly fault intolerant (and quite hard to master to boot). PHP’s type system is a lot more permissive and tolerant (“weak”, one might say) than a language like Python. The former is great for newbies and non-programmers, because adding 1 + "1" (that is, adding a string and an integer together) will give you 2, or at least something that when printed to screen looks vaguely like a two, though under the covers it may be Cthulhu.1 And the behaviour of something like Python is great for jaded old gits like me who don’t want the system to magically convert strings into integers but to blow up as quickly as possible so that it can be fixed before all this nastiness gets stitched together with the rest of the code and causes some real major bugs. The same principle applies on a grander scale with stricter type systems like Java or the broad ML family (including things like Scala, F#, Haskell etc.).

“A misspelt tag in HTML or a missing curly brace in CSS can also cause headaches, but imperative programs must be well‐formed or they won’t run at all.”

Again, depends on the language. PHP is pretty permissive. If you make a very minor mistake, you can often just carry on and keep going. If you are feeling particularly careless about your errors, you prefix your expression with an @ sign and then security people get to laugh at you. I hesitate to say this was “designed” but it was at the very least chosen as a behaviour in PHP.

This may all seem rather pedantic, but it is actually quite important. HTML and CSS are declarative and permissively fault-tolerant. That’s a key strength. In an environment like the web, it creates a certain robustness. If your JavaScript fails to load entirely, you can get some very strange behaviour if, say, a function that is expected to be there isn’t. (I use a site frequently that makes multiple Ajax calls but if one fails, say due to a bad connection, the site is unusable and must be reloaded from scratch. It is also contained in an iOS app, which must be manually restarted.) But if some of your CSS doesn’t load, that’s not the end of the world: you still get something you can read. If some of your HTML has weird new stuff in it, as Jeremy points out elsewhere in the book, that’s still backwards compatible–the browser simply ignores that element and renders its content normally.

This error handling model, this permissiveness of web technologies, isn’t a side-effect of being declarative. It’s actually a property of them being designed for the web, of human choice by the creators. There is a cost to this. It has been incredibly hard for us to secure the web. Permissive error handling can and has enabled a whole class of security vulnerabilities.

If Postel’s Law gives you the ability to use border-radius in your CSS or aside in your HTML and some terrible old version of Internet Explorer happily ignoring it without vomiting XML warnings all across your screen, then Postel’s Law also comes with the cost of having to worry about downgrade attacks. We collectively left SSLv2 running long after it should have been dead and we got DROWN. We did the same with SSLv3 and we got POODLE. These are examples of ungraceful degradation and the sad cost is your server being vulnerable to being pwned.2

With the last few attacks on SSL/TLS, it wasn’t just nasty old versions of Internet Explorer on Windows XP getting shut out of the web, it was non-browser apps that talked to HTTPS-based APIs. The Logjam attack meant that a lot of people upgraded their servers to not serve DH keypairs that are below 1024-bit. For most current day browsers, this was not an issue. Apple, Mozilla, Google, Microsoft and others released patches for their current browsers. Java 6 didn’t get a patch for a very long time. If you had a Java desktop app that consumed an HTTPS-based RESTful API which had patched Logjam, that broke with no graceful degradation, and the general solution was to upgrade to a new version of Java. On OS X, Java used to be distributed by Apple, albeit as something of a reluctant afterthought. Upgrading every desktop Java user on OS X was a bit of a faff. (My particular Java Logjam issue was with… JOSM, the Java OpenStreetMap editor.)

Postel’s Law giveth and it taketh away. One could give this kind of rather pedantic counterexample to Jeremy’s account of Postel’s Law and then conclude hastily “given how badly the web is at solving these security problems, the grass on the native apps side of the fence might just be a little bit greener and yummier”. Oh dear. Just you wait. When every app on a platform is basically reimplementing a site-specific client, you might actually get more fragility. Consider our recent vulnerabilities with SSL/TLS. After something like Logjam, the bat signal went out: fix your servers, fix your clients. On the server side, generally it meant changing a config file for Apache or Nginx in a fairly trivial way and then restarting the web server process. On the client side, it meant downloading a patch for Chrome or Firefox or Safari or whatever. That may have just been rolled into the general system update (Safari) or rolled into the release cycle (Chrome) without the end user even being aware of it. The long tail3 of slightly oddball stuff like Java desktop apps, which tends to affect enterprise users, assorted weirdos like me, and niche use cases, took a bit longer to fix.

If every (client-server) app4 that could be in a browser, in the case of a security fail, fixing all those apps would be as simple as fixing the browser (and the server, but that’s a separate issue). If everything were a native app, you have to hope they are all using the system-level implementations of things like HTTP, TLS and JSON parsing, otherwise you have a hell of a job keeping them secure after vulnerabilities. We already see things going on in native-app-land (Napland?) that would cause a browser to throw a big stonking error: user data being sent in cleartext rather than over TLS being more common than I care to think about. But the native app won’t scream and shout and say “this is a bad, very wrong, no good idea, stop it unless you really really really want to”, because the UI was developed by the person who created the security issue to start with.

The downside to Postel’s Law5 is sometimes the graceful degradation is pretty damn graceless. Sadly, the least graceful degradation is often security-related. The web might still be better at mitigating those than all but the most attentive native app developers, or not. Browser manufacturers may be better at enforcing security policies retroactively than app stores, or they might not. We shall have to wait and see.

The Web is a hot mess.

But we still love it.

  1. Sausage typing: if it looks like a sausage, and tastes like a sausage, try not to think about what it really is.

  2. At risk of giving technical backing to rising reactionary movements, perhaps the modern day variation of Postel’s Law might be: “Be conservative in what you send, and be liberal in what you accept, to the degree that you can avoid catastrophic security failures.”

  3. Wow, it’s been a long time since we talked about them…

  4. Let’s leave aside the semantics of what is or isn’t an app. Incidentally, I was in a pub the other day and saw a few minutes of a football game. There was a big advert during the game from a major UK retail brand that said “DOWNLOAD THE [brand name] APP”. Why? I don’t know. We are back at the ghastly “VISIT OUR WEBSITE, WE WON’T TELL YOU WHY” stage with native mobile apps. I’m waiting for a soft drink brand to exhort me to download their app for no reason on the side of a bottle.

  5. I claim no originality in this observation. See here and here. Interestingly, if you take a look at RFC 760, where Postel’s Law is originally described, it has a rather less soundbitey remark just before it:

    The implementation of a protocol must be robust. Each implementation must expect to interoperate with others created by different individuals. While the goal of this specification is to be explicit about the protocol there is the possibility of differing interpretations. In general, an implementation should be conservative in its sending behavior, and liberal in its receiving behavior. That is, it should be careful to send well-formed datagrams, but should accept any datagram that it can interpret (e.g., not object to technical errors where the meaning is still clear).

    The first two sentences are key…


You (probably) don't need a chatbot

There has been a great hullabaloo in the last few months about the rise of chatbots, and discussions of “conversational UIs” or, even more radically, the concept of “no UI”—the idea that services might not need a UI at all.

This latter concept is quite interesting: I’ve written in the past about one-shot interactions. For these one-shot interactions, UI is clutter. But chatbots aren’t the answer to that problem: because chatbots are UI, just a different sort of UI. Compare…

Scenario 1: App

  1. Alice hears an amazing song playing in a club.
  2. Alice pulls out her iPhone and unlocks it by placing her finger on the TouchID sensor.
  3. Alice searches on the homescreen for the Shazam app.
  4. Alice opens Shazam, then presses the button to start the process of Shazam identifying the song that is currently playing.
  5. Alice waits.
  6. Alice is told what the song is and offered links to stream it or download it from a variety of streaming and download services that vary depending on the day of the week, the cycle of the moon, and how Shazam’s business development team are feeling this week.

Scenario 2: Chat

Someone at Shazam decides that apps are a bit old-fashioned and decides to build a chatbot. They have read an article on Medium.com that tells them that chatbots are better, and decide to build one based solely on this advice rather than any actual empirical evidence.

  1. Alice hears an amazing song playing in a club.
  2. Alice pulls out her iPhone and unlocks it by placing her finger on the TouchID sensor.
  3. Alice searches on the homescreen for the Facebook Messenger app.
  4. Alice opens Facebook Messenger, then locates the existing chat session with the Shazam bot.
  5. Alice scrolls back up the chat to work out what the magic phrase she needs to type in to trigger the chatbot into listening to music.
  6. Alice waits.
  7. Alice is told what the song is and offered whatever extra rich data the chat UI is allowed to show.

As you can see, this is a vast improvement, not because it makes the process less involved or elaborate, but because someone on Medium.com told them that it is new and exciting.

Scenario 3: Idealised One-Shot Interaction

  1. Alice hears an amazing song playing in a club.
  2. Alice taps a button on her smartwatch. Everything else happens in the background. Alice continues partying and enjoying herself rather than being the saddo staring at her phone all night.

For those without a smartwatch, a lockscreen button on the phone could be substituted.

Anyway, this is a slight distraction from the broader point: chatbots are a bit of a silly fashion and a fad and that they seem to be adopted based on fashion rather than based on any actual utility.

But, but, there’s this awesome chatbot I use, and I really like it!

Great. I’m not saying that they have no purpose, but that chatbots are being adopted even though they often are worse at what they do than the alternative. They also come with considerable downsides.

First of all, chatbot UIs are poor at letting a user compare things. When someone browses, say, Amazon or eBay or another e-commerce service, they will often wish to compare products. They’ll open up competing products in different tabs, read reviews, check up on things on third-party sites, ask questions of their friends via messaging apps and social media sites like Facebook. Chatbot UIs remove this complexity and replace it with a linear stream.

Removing complexity sounds good, but when someone is ordering something, researching something or in any way committing to something, navigating through the complexity is a key part of what they are doing.

Imagine this scenario. Apple have 500 different iPhones to choose from. And instead of calling them iPhones, they give them memorable names like UN40FH5303FXZP (Samsung!) or BDP-BX110 (Sony!). Some marketing manager realises the product line is too complex and so suggests that there ought to be a way to help consumers find the product they want. I mean, how is the Average Joe going to know the difference between a BDP-BX110, a BDP-BX210, and a BDP-BX110 Plus Extra? You could build a chatbot. Or, you know, you could reduce the complexity of your product line. The chatbot is just a sticking plaster for a broader business failure (namely, that you have a process whereby you end up creating 17 Blu-Ray players and calling them things like BDP-BX110 rather than calling them something like “iPhone 7” or whatever).

Chatbots aren’t removing complexity as much as recreating it in another form. I called my bank recently because I wanted to enquire about a direct debit that I’d cancelled but that I needed to “uncancel” (rather than setup again). I was presented with an interactive voice response system which asked me to press 1 for payments, 2 for account queries, 3 for something else, and then each of those things had a layer more options underneath them. Of course, I now need to spend five minutes listening to the options waiting for my magic lucky number to come up.

Here’s another problem: the chatbot platforms aren’t necessarily the chat services people use. I’m currently in Brazil, where WhatsApp is everywhere. You see signs at the side of the road for small businesses and they usually have a WhatsApp logo. WhatsApp is the de facto communication system for Brazilians. The pre-pay SIM card I have has unlimited WhatsApp (and Facebook and Twitter) as part of the 9.99 BRL (about USD 3) weekly package. (Net neutrality? Not here.) The country runs on WhatsApp: the courts have blocked WhatsApp three times this year, each time bringing a grinding halt to both business and personal interactions. Hell, during Operação Lava Jato, the ongoing investigations into political corruptions, many of the leaks from judges and politicians have been of WhatsApp messages. Who needs Hillary Clinton’s private email servers when you have WhatsApp?

WhatsApp is not far off being part of the critical national telecoms infrastructure of Brazil at this point. Network effects will continue to place WhatsApp at the top, at least here in Brazil (as well as most of the Spanish-speaking world).

And, yet, WhatsApp does not have a bot platform like Facebook Messenger or Telegram. To get those users to use your chatbot, you need to convince them to set up an account on a chat network that supports your bot. For a lot of users, they’ll be stuck with WhatsApp, the app they use to talk to their friends, and Telegram, the app they use to talk to weird, slightly badly programmed robots. Why bother? Just build a website.

Now, in fairness, WhatsApp are planning to change this situation at some point, but you still have an issue to deal with: what if your users don’t have an account on the messaging service used by the bot platform?

One of the places chatbots are being touted for use is in customer service. “They’ll reduce customer service costs”, say proponents, because instead of customers talking to an expensive human you have to employ (and pay, and give breaks and holidays and parental leave and sick days and all that stuff) to, you just talk to a chatbot which will answer questions.

It won’t though. Voice recognition is still in its infancy, and natural language parsing is still fairly primitive keyword matching. If your query is simple enough that it can be answered by an automated chatbot, it’s simple enough for you to just put the information on your website, which means you can find it with your favourite search engine. If it is more complicated than that, your customer will very quickly get frustrated and need to talk to a human. The chatbot serves only as a sticking plaster for lack of customer service, or business processes that are so complicated that the user needs to talk to customer service rather than simply being able to complete the task themselves.

You know what else will suffer if there were a widespread move to chatbots? Internationalisation. Currently, the process of internationalising and localising an app or website is reasonably understandable. In terms of language, the process isn’t complex: you just replace your strings with calls to gettext or a locale file, and then you have someone translate all the strings. There’s sometimes a bit of back and forth because there’s something that doesn’t really make sense in a language so you have to refactor a bit. There’s a few other fiddly things like address formats (no, I don’t have a fucking ZIP code) and currency, as well as following local laws and social taboos.

In chatbot land, you have the overhead of parsing the natural language that the user presents. It’s hard enough to parse English. Where are the engineering resources (not to mention linguistic expertise) going to come from to make it so that the 390 million Spanish speakers can use your app? Or the Hindi speakers or the Russian speakers. If your chatbot is voice rather than text-triggered, are you going to properly handle the differences between, say, American English and British English? European Portuguese, Brazilian Portuguese and Angolan Portuguese? European Spanish and Latin American Spanish? Français en France versus Québécois? When your chatbot fucks up (and it will), you get to enjoy a social media storm in a language you don’t speak. Have fun with that.

And you can’t use the user’s location to determine how to parse their language. What language should you expect from a Belgian user: French, Dutch or German?

If you tell a user “here’s our website, it’s in English, but we’ve got a rough German translation”, that’s… okay. I use a website that is primarily in German everyday, and the English translation is incomplete. But I can still get the information I need. If, instead, your service promised to understand everything I say, then completely failed to speak my language, that’d be a bit of a fuck you to the user.

In the chatbot future, the engineering resources go into making it work in English, and then we just ignore anyone who speaks anything that isn’t English. World Wide Web? Well, if we’re getting rid of the ‘web’ bit, we may as well get rid of the ‘world’ and ‘wide’ while we’re at it.

Siri and Cortana are still a bit crap at language parsing, even with the Herculean engineering efforts of Apple and Microsoft behind them. An individual developer isn’t going to do much better. Why bother? There’s a web there and it works.

There’s far more to “no UI” or one-shot interactions than chat. But I’m cynical as to whether we’re ever going to reach the point of having “no UI”. We measure our success based on “engagement” (i.e. how much time people spend staring at the stuff we built). But the success criteria for the user isn’t how much time they spend “engaging” with our app, but how much value they get out of it divided by the amount of time they spend doing it. The less time I spend using your goddamn app, the more time I get to spend, oh, I dunno, looking at cat pictures or snuggling with my partner while rewatching Buffy or writing snarky blog posts about chatbots.

But so long as we measure engagement by how many “sticky eyeballs” there are staring at digital stuff, we won’t end up building these light touch “no UIs”, the interaction models of set-it-and-forget-it, “push a button and the device does the rest”. Because a manager won’t be able to stand up and show a PowerPoint of how many of their KPIs they met. Because “not using your app” isn’t a KPI.

Don’t not build a chatbot because of my snarkiness. They may solve a problem that your users have. They probably don’t but they might. But please don’t just build a chatbot because someone on a tech blog or a Medium post told you to. That’s just a damn cargo cult. Build something that delivers value to your users. That may be a chatbot, but most likely, it’s something as simple as making your website/app better.


No, I'm not going to download your bullshit app

How we used to read the news, back in the era of the Web:

  1. Go to newspaper website.
  2. Click on story.
  3. Read.

How we read news in the era of fucking stupid pointless iPhone apps.

  1. Go to website.
  2. Be told you aren’t allowed to read the website.
  3. Be redirected to an App Store.
  4. Download the app. (This may involve typing in a password. Which may involve shuffling over to your password manager app to find your password.)
  5. Wait while a multi-megabyte file downloads over your temperamental, expensive 3G connection.
  6. Open the app up.
  7. Familiarise yourself with an interface that has cryptic, weird gestures that aren’t actually revealed to the user and behave ever so slightly differently from every other similar app.
  8. Struggle as the badly-implemented statefulness gives you a spinning loading wheel (on iOS) or flashing progress bar (on Android) because you had the audacity to use your mobile device on a slow or unreliable connection.
  9. Attempt to find the story you wanted to read using a layout and information architecture that’s completely different from the layout and information architecture of the website that you’ve grown familiar with, because some arsehole decided that the process of reading the electronic equivalent of a newspaper needs to be “disrupted” because he’s been reading far too much Seth Godin or some other bullshit.
  10. Realise that the app shows you different things depending on whether it’s in landscape or portrait mode. Now you can look like an utter nob on the Tube rotating your iPad around so that you can zoom further into the Page 3 stunna’s tits.
  11. Not be able to share the story with your friends because it’s not a page on the web with a Uniform Resource Indicator. Because why do you need universal addressability when you’ve got shiny spinny touchy magical things to rub your sweaty greasy fingers all over?
  12. Take time to download updated binary files the next time the application is updated in the App Store, that’ll provide you “new functionality”, even though there is no fucking functionality you actually want other than reading the fucking news.
  13. If you are on Android, be sure to install some anti-adware software in case the app comes with some delightful bit of creepy privacy-intruding out-of-app advertising.
  14. Give up, go to newsagent, buy paper edition, throw smartphone off a fucking cliff and start a letterbomb campaign against all the idiots who thought that turning newspapers into “apps” was a good idea.

In the “web vs. apps” war, I think you can infer which side I’m on. I wouldn’t download a BBC app or an NPR app for my computer. Why would I want one on my phone? Do I buy a separate radio to listen to different stations? No. The functionality is the same, the only thing that differs is the content. Apps ought to provide some actual functionality, not just blobs of content wrapped up in binary files.

Maintenant disponsible en version française