tommorris.org

Discussing software, the web, politics, sexuality and the unending supply of human stupidity.


A tale of two blog posts: how to be as glassy as GitHub or as murky as Microsoft

So, the big news this week is that Microsoft bought GitHub for $7.5 billion. This has been welcomed, lamented and, well, meme-ified in about equal measure.

What I found quite interesting was the language of the two blog posts announcing the deal rather clearly demonstrate the contrast between the two companies and how they operate and are perceived. I should first be clear that I have a fairly high regard for Satya Nadella, Microsoft’s CEO. He seems to be fairly thoughtful and deliberate in how he has run Microsoft and under his leadership, the company has made some rather big changes mostly for the better.

Those disclaimers duly made, Nadella’s post falls clearly into corporate jargon repeatedly. Every industry is being impacted by technology, and thus the workflows of developers will affect value creation, by empowering communities. If you spotted ideation, feel free to shout “bingo!”

There’s also some strange bits of Microsoftie waffle, like the “Intelligent Edge”. I had no idea what an “Intelligent Edge” is, so I read a post entitled “Advancing the future of society with AI and the intelligent edge” published on Microsoft’s blog by Frank X. Shaw, Corporate Vice President for Communications. I still have no idea what it is, except it involves drones and IoT devices and nifty machine learning algorithms that predict diseases. The fact that I’m unable to understand it after reading a blog post written by someone who has ‘Communication’ in their job title is rather telling.

(My best stab after reading it is when Microsoft talk about “intelligent edge”, what they mean is they want to make sure the computers that aren’t out there in the clouds—i.e. in a data centre—also have the facility to do vaguely intelligent things like doing machine learning and classification. As a concept, it seems to exist as a way to try and rhetorically rebalance back from the focus on the cloud.)

Anyway, I don’t mean to bash Microsoft, or Nadella, but merely to point out the difference in wording between the Nadella blog post and the post up on GitHub’s blog by Chris Wanstrath: “A bright future for GitHub”.

Microsoft’s post is full of generality and jargon, plus references to futuristic, unproven or speculative technologies like precision medicine and personalized banking. GitHub’s blog post is clear and specific: it tries to head off concerns about developer trust in Microsoft by pointing to clear examples of Microsoft’s contributions to projects like git-lfs, Electron and the open source release of VS Code. GitHub’s post talks in terms of human emotions like pride and excitement. There are some slight flaws with the GitHub post—mixing “as well as oil and water” is a metaphor well beyond the grave at this point, Minecraft probably wouldn’t be what I’d put at the top of my list of successful Microsoft acquisitions, and it repeats Microsoft’s impenetrable edge-iness: Code to Cloud and Code to Edge.

But in general, Wanstrath’s post is a reasonably good piece of writing and communicates the values of GitHub. It is short, uses straightforward language and communicates the message that both companies likely wished to send. As a GitHub user, Microsoft’s post is a clear example of the fear of big company takeovers of products one uses and broadly likes that many will have. Big companies like Microsoft, even those blessed with intelligent and technically sensible leaders like Nadella, communicate in this kind of strange mixture of jargon, marketese and speculative Big Picture diversions from the day-to-day reality that users care about. As a user, Wanstrath’s post reassures me that GitHub are looking after my interests while Nadella’s post mostly tells me about the continuing dysfunctions of big companies like Microsoft.

If you have anything to do with communicating with technical people or about technical products, both of these posts are worth reading for showing a very clear example of the values of clarity and precision vs. jargon-riddled haziness.



The British government have put out a draft bill for regulating self-driving cars.

The basic thrust of it is insurance companies will be required to cover accidents caused by self-driving vehicles just as they would with traditional vehicles, but they can then recover the losses from the manufacturer through bringing a civil suit. There’s also provisions in the bill related to electric vehicle charging.


ROCA is an interesting vulnerability for some RSA keys. It’s less panic-inducing than you might initially think, but it’s interesting to look into.

They’ve made a key checking tool available so you can test your keys with it. (I had a brief look at the code and it doesn’t look—or act—scary, but you know, you should exercise some discretion with software that’s going to inspect your private keys…)



LibreOffice: a case study in frustration and usability shortcomings

I don’t really use LibreOffice. I have it installed because it’s marginally better than the alternatives—namely, paying for Office 365, using OpenOffice (which is basically LibreOffice without any pesky developers actually working on it), or figuring out how Apple have broken Pages since I last used it. I use text/plain, Markdown, and if I’m feeling particularly masochistic, LaTeX. Or Google Docs because of social obligation.

Recently, I had some docx files I needed to tidy up, edit a bit and convert to PDF. Finally, a perfect use case for LibreOffice to shine.

I started to report these issues on LibreOffice’s bug tracker but life is too short for Bugzilla.

Keyboard shortcuts didn’t work

I opened up the first document and it was too big for my screen. I’m using a Mac, so let’s use the universal zoom out command: Cmd -

That didn’t work. It just put a - into the document. Unhelpful.

How about zoom in? Cmd =. Nope, that didn’t work either.

So that’s problem 1: Cmd - and Cmd = don’t work for zooming in/out even though that’s the keyboard shortcut in every other Mac app.

What is the keyboard shortcut for it?

Finding keyboard shortcuts

I clicked ‘Help’ on the menu bar. I typed in ‘keyboard shortcuts’ in the search box. Nada.

LibreOffice’s help menu doesn’t integrate with the Help menu search box in macOS. But it still has a search box. The point of the Help menu search box is that it searches both the menu options of the application and the topics listed in the documentation. In LibreOffice it does only the former (which I believe is built into the operating system) but not the latter. This is misleading: it’ll lead users to think there isn’t documentation there when there is.

So, problem 2: The Help menu has a search field that doesn’t work.

Okay, let’s look at the Help menu. Many Mac apps have an item on the Help menu called Keyboard Shortcuts. Nope.

Problem 3: If your app has keyboard shortcuts, you should have a menu item in the Help menu telling people what they are. Undiscoverable keyboard shortcuts need to be made discoverable.

The first entry in the help menu is called ‘LibreOffice Help’ and has the distinctly Windowsy shortcut F1. I open that up.

Problem 4: Don’t use Function key shortcuts on macOS like you would on Windows. macOS is not Windows.

It then brings up a window called LibreOffice Help and it is focussed in a search box so I can search the index. You know, the thing I couldn’t search using the in-menu search box.

So I search for “keyboard shortcuts”. Nope. There is “keyboard; general commands” which didn’t have a cross-reference for “keyboard shortcuts”. Okay, let’s take a look at that.

Problem 5: If you have a document containing keyboard shortcuts, you should be able to find it by typing ‘keyboard shortcuts’ into the search field.

Nothing in that document about zooming in or out. Okay, let’s search for something else. I focus back in the search field. I type Cmd+A to select all the text in the search field to delete it and… I get a Database Wizard window.

Do I want to create a new database? No. I want to select all in the window I was focussed in, so I could delete it and type something else, just like I would type Cmd+A in any other Mac app.

I’m pretty sure if I wanted to create a database, the route I would take wouldn’t be: open LibreOffice, open a word processing document, edit it a bit, want to zoom out, open up the help centre, type something, not find what I’m looking for, then try to change my search, then create a database. Makes perfect sense.

Problem 6: Why would I want to create a database when I’m looking at help files for the word processor?

The help centre is not going to help, evidently. Maybe there’s a menu option to zoom the document.

View menu, Zoom! Okay, no keyboard shortcuts listed in there. There’s keyboard shortcuts for a whole bunch of things which aren’t listed in the help document, but whatever.

I change the zoom level by going to the submenu. You can also change the zoom by using the slider in the bottom right hand corner of the window. This has zoom in and zoom out buttons (marked ‘-‘ and ‘+’). That function exists… but isn’t accessible using the menu or through a keyboard shortcut.

As an aside: the database thing

When I accidentally ended up in the ‘Database Wizard’ while trying to find keyboard shortcuts in the help section, I noticed it asked me what sort of database I wanted to create, with a drop-down menu.

The only option in the drop-down menu is ‘HSQLDB Embedded’. As someone who knows a bit about databases, I know what this is. But:

Problem 7: You are offering me a choice of databases but there’s only one. Why offer me a drop-down box—suggesting some choice—if there is no choice?

Problem 8: Why tell me what type of database it is? This gives me no meaningful information.

Back to my document

Alright, so zooming in and out of a document using system-wide keyboard shortcuts isn’t going to be possible. Oh well. Let’s work on the document.

First of all, there was a table I wanted to delete. I did what many years of Microsoft Word—and natural intuition—taught me. Go to the line after the table finished. Press delete a bunch of times until the table goes away. Nope.

How about create a selection that starts after the table finishes and go back before the table starts then press delete? Nope. That doesn’t work either.

The way you delete a table is you right click on the table and choose “Delete > Table”. Or you do the same from the drop-down menus.

Problem 9: Why not let me delete a table by backspacing over it, or range-selecting it and pressing delete?

I delete some stuff from the document. It has a table of contents that’s auto-generated. So I go to it and “update” it. It loses all the formatting from the table of contents. This is bad.

Problem 10: Updating a table of contents in a docx file lost the formatting of the table of contents.

Now, time to save my document. I save a copy as ODT. I want to export it as PDF too.

Problem 11: The Export as PDF window has ‘Create PDF form’ pre-selected even though my document doesn’t have any forms in it. Surely, it’d be sensible to work out whether the document has any forms in it then only offer me the option of whether to create a PDF form if there’s forms that actually need creation? Otherwise, you are asking me to make a wholly redundant decision (and, boy, are there a lot of decisions in the LibreOffice Export as PDF panel: it’s a six-panel tabbed interface with 75 different interface components including radio buttons, drop-downs, checkboxes, numeric selectors, buttons and a file selector). If it’s irrelevant, grey it out.

I’m not trying to be shitty to LibreOffice’s developers: I’m glad LibreOffice exists and want it to succeed, even though it’s not really for me. It still feels very unpolished and unloved. I get why some of the above issues exist: they are trying to make a system that works across platforms, they’ve got a whole bunch of different products to work on.

I have reported one of these issues on Bugzilla, and it was closed as a duplicate of an issue that has existed untouched since 2012. I wish the LibreOffice developers all the best, but it does seem like an unmanageably large monolithic blob of an application with an enormous amount of technical debt with a small, overstretched team.


git: redo a failed commit with the message you already typed

I bet you’ve been in this scenario: you are committing something to a Git repository and it fails.

Perhaps a pre-commit hook fails. Perhaps you are using a secure smartcard (including a YubiKey) to sign your commit, but it isn’t plugged in. Or you type in your PIN or GPG passphrase wrong. Or you’ve accidentally got an .git/index.lock file in the way and you need to clear it before committing. Or maybe your EDITOR environment variable is set wrong. You’ve attempted to commit. It fails.

You’ve fixed whatever makes it not fail. You want to commit again and use the beautiful commit message you wrote a few minutes ago when it failed.

Easy.

git commit -F .git/COMMIT_EDITMSG

When your commit fails, it stores the message in a file called .git/COMMIT_EDITMSG. This merely commits with the contents of that file as your commit message. -F/--file takes a file name and uses that as the commit message, similarly to how -m takes a message on the command line.

On my computer, I’ve set it up as an alias (git retry-commit). Which I probably won’t remember.


My blog has allowed me to post over 140 characters for a lot longer than Twitter.

I can also correct my spelling, add links and formatting and I own it. I like this. Writing without limitations. Also, no Nazis on my website. Guaranteed.


How to use a YubiKey to secure all the things

For a while I’ve been using YubiKey’s blue FIDO key. It enables you to use FIDO U2F for web services. Currently, the main sites that support Universal 2nd Factor (U2F) login are Google, Facebook, GitHub, Dropbox, Fastmail and the password manager Dashlane. It’d be great to see more sites added to this list, but this is a good start.

I upgraded recently from the blue FIDO key to the black YubiKey 4. The full YubiKey can be used for a few things

  • website 2FA using FIDO U2F
  • login to your computer using PIV
  • GPG
  • SSH

It took a while before I got it setup for each of these things.

U2F

This is the easy bit. You go to the relevant websites, you activate the U2F mode and you put the key in as required. This works the same whether you use the blue FIDO U2F-only key or the black YubiKey 4.

Generally, the way to use U2F is in addition to TOTP tokens. U2F is a nicer experience than TOTP on laptops and most desktops. (Really, some USB ports are really poorly designed for actually using a YubiKey with. The USB ports on either side of the old Mac keyboards are bad. The two USB ports on my Das mechanical keyboard are perfect.)

Computer login using PIV

To set your Windows or Mac computer up to login, you can do that using the YubiKey PIV Manager app. You can find instructions on the YubiKey website on how to set up key-based authentication on Linux.

The way I’m using the PIV mode is like this. I have a long password for my computer. I can type this in or I can use the YubiKey with a six-digit PIN. If you’ve got a good 20+ character password, you can always use that. But if you also have the USB key, you can use the PIN instead.

GPG

With the YubiKey, you can put a number of GPG keys on to it. As with the PIV logins, these are protected with a PIN. My main use case for GPG at the moment is code signing for GitHub. (Alas, nobody seems too bothered about email signing.)

This blog post by Simon Josefsson explains the process of setting up a YubiKey for GPG. The broad approach is you create a GPG key, then you create three subkeys—one for encryption, one for signing, one for authorisation. The private key for each is stored on the YubiKey: once you’ve pushed the private key to the YubiKey, there’s no way to get it back out again. (So, it’s sensible to make a backup of the keys and store them offline on, say, a USB key in a safe.)

If you’ve already got your public key on GitHub, you’ll need to export a new public key containing the subkeys you’ve stored on the YubiKey and paste that up on GitHub. (If you use GitLab, this whole approach doesn’t work… because of this bug I’ve reported.)

Once you’ve got it all set up, when you commit some new code to a Git repository, it’ll ask you for your YubiKey’s six-digit PIN. That is cached for a bit, so it should only interrupt you occasionally. If you remove the YubiKey, you’ll obviously have to put it back in and re-enter the PIN to commit new code.

SSH

Now the fiddly bit. The way you use SSH with the YubiKey is to convert your GPG key into an SSH public key. There’s a utility called gpgkey2ssh that does just this for you: you point it to your GPG public key and it’ll turn that into an SSH public key (remember, other than an offline—and hopefully encrypted—backup, you don’t have the private key: it’s stowed away inside the YubiKey).

The problem occurs with the version of GPG. You need to ensure you are running GPG 2.1. You can now get it from Homebrew on the Mac, so do that. You may need to ensure that symlinks are going to GPG 2.1 if you’ve got multiple installs.

This tutorial is the best one for getting gpg-agent setup for doing SSH. I found that it’s a bit of a faff keeping track of which keys are doing what during the process, so I ended up noting them down on a bit of paper.

So, what to do?

If you are an ordinary user and you don’t mind the tradeoffs of having to occasionally plonk a USB key in the side of your computer, get a blue YubiKey FIDO U2F. They’re fantastic.

There are issues: on desktop Macs, you need to ensure you have a USB socket that’s actually available and not a faff to use (i.e. not on the back of a Mac Mini, or tucked away under the edge of a keyboard so you can put the key in… but can’t actually push the button). You may need to get a little bit of USB extension cable to give you somewhere to press, or you might consider getting the YubiKey Nano.

If you are a geek and write software (or software-adjacent products including documentation), and you don’t mind jumping through quite a few more hoops to get the GPG and SSH stuff set up, you should do that too. If you don’t use GPG or SSH, you probably don’t need the black YubiKey and can stick to the blue FIDO U2F device.

You can buy the YubiKey 4 from Amazon UK, or if you prefer, you can get the YubiKey FIDO U2F ‘blue’ key.

Further reading


How to no longer fuck up symbolic links with lns

lns is an old-school Perl utility that unbreaks the command order of ln -s.

If you’ve got a file called a.txt and you want to make a symbolic link b.txt, you can type lns a.txt b.txt or lns b.txt a.txt and it’ll do the intelligent thing. This is really handy. It won’t overwrite anything. It won’t break shit. It’s in my ~/bin/ forever now.

Since that site is looking a bit old and shaky, here’s archive versions: Internet Archive 2016-10-30, GitHub Gist for source code 2017-08-30


Things I learned this week: #EuroPython

This week, I’ve been at EuroPython in Rimini, Italy. I’ve been tweeting some sessions on the @Nexmo Twitter account. Here, I’m link-dumping a bunch of stuff I learned that you may or may not find of interest. It’s less a methodical writeup of talks, more links and stuff I found that are interesting, some of which were discussed at the conference (others have nothing to do with Python, programming or the conference).

Sourcelift is a subsidised travel programme for open source developers to go to conferences and events. They support mostly Python, Node.js and PostgreSQL-related projects, plus language-agnostic tools like text editors and package managers.

World’s Biggest Data Breaches: a rather depressing infographic of big hacks and data breaches.

awesome-security is a collection of interesting links related to security.

Bandit is a static analyzer for Python that inspects the abstract syntax trees (ASTs) of Python code looking for common security issues. Some of the checks are pretty straightforward and could be achieved without the need for an AST parse: grep could tell you whether the code uses stuff like the ftplib, telnetlib, xmlrpclib or pickle modules, eval, or MD5. But checking for empty exception parsing (i.e. except: pass) is useful.

Static analysis seems to have been a theme at EuroPython this year, with one mention of SonarQube, a hosted SaaS application that scans for bugs and potential security vulnerabilities in a bunch of languages including C, C++, C#, PHP, Java, Python, JS, VB (6 and .NET), Swift and Objective-C. I haven’t tried it yet but it looks interesting.

Even more accessible is Semmle’s LGTM1. It’s blissfully simple: you add a repo, it scans it for bugs and alerts them to you in a dashboard. I’ve fixed a very minor one already. If you are maintaining an open source project, adding this kind of stuff is a no brainer. Robots finding bugs automatically is better than humans finding them.

Every year, there’s a games design contest called PyWeek. The idea is simple: you have one week to design and build a game, in Python, based on a theme. This is a really cool idea.

PyDatalog is an implementation of Datalog. Datalog is a simplified subset of Prolog. It is very easy for one to forget the existence of logic-based programming languages like Prolog/Datalog, but they still exist and still help solve real world problems.

objgraph is a Python object graph visualiser (it uses GraphViz). It’s quite helpful for finding memory leaks.

sanest is a library for nested dictionaries and lists. It allows you to do nested operations, and more intelligently handle errors.

Two web smaller-than-micro frameworks: Firefly and Pico.

I’m going to write more about data science, specifically machine learning soon.

  1. I’ve already mistyped it ‘LGBT’ at least three times. 🏳️‍🌈




Chatbots are the future and always will be

I read a lot online about how chatbots are the future. Ultimately, the proof of chatbots is in the eating—I mean, err, talking (this really is a crap metaphor). Let’s try one out to do something like… booking a flight.

The “domain model” of commercial passenger aviation is familiar to most people, but still there are a lot of choices to be made. Which airline? Which day do I want to fly? Some days are a bit cheaper than others. What seat class do I want? What ticket class do I want? (They are different things.) Do I want to book using my frequent flyer miles or with cash? Do I get checked baggage or just hand baggage? It might be slightly cheaper if I fly from London Gatwick rather than Heathrow. Do I want to land at Newark or JFK? Am I logged into my frequent flyer account? Do I need to supply Advance Passenger Information now or later? What about if I need to change flights? If I book business class for the long-haul segment, am I going to get business class on the short-haul codeshare segment? What are the cancellation conditions?

Apps like those provided by British Airways do their best to hide away the complexity of this. Within the confines of “I know I want to fly with BA”, rather than “I want to compare different airlines”, the app from BA provides a pretty good experience at managing this matrix of different options.

Now let’s try and book a flight using a chatbot.

First stop, Facebook Messenger. After all, Mark Zuckerberg said:

We think that you should just be able to message a business in the same way you message a friend

Well, I know that I like messaging friends at 8:30 on a Saturday morning demanding that they look up flights for me. So let’s go!

I open up Facebook. ‘New Message’. I type in British Airways. And I get… nothing.

Okay, this is just the Messenger thing they build into the default Facebook view. I guess I should open up the actual Facebook Messenger page. Let’s have a chat with British Airways.

I found, well, British Airways i360. Which is a giant levitating dougnut on Brighton beach. I thought about messaging it and asking whether or not the doughnut is currently levitating or not. But then I realised that there’s probably someone being paid not very much money to answer stupid questions from people on the Internet and left it be.

Maybe British Airways haven’t climbed on board the chatbot train yet. I’m sure some enterprising developer must have built a chatbot that lets me search and book flights. Let’s search for one, on Facebook Messenger. I had an insightful conversation with ‘Flight Bot’ that resulted in me… not being able to search for a flight. That went well.

I mean, I get it. It seemed like a good idea when they built it, maybe not now.

I still want to find a chatbot that can help me fly from London Heathrow to Newark Liberty International. That shouldn’t be too hard. So, as with all problems, let’s turn to Google. Oh, Google, find me a travel chatbot.

Huzzah! There are plenty to choose from. The popular travel booking sites Expedia, Skyscanner, Kayak and Cheapflights all have chatbots. On Facebook! Okay, let’s test Expedia.

That didn’t work out. I’m now feeling guilty about wasting some poor human’s time with a test query when I could just search on the web. Let’s give Skyscanner a go. This time, instead of just being a chat window, it asks me to “Get Started”. This means it’s actually a bot, I guess.

The fact that it’s taken me this long to find a bot to help me book a flight, when the apps and websites are already to hand is… kind of telling. But whatever, let’s play along.

Hi Tom! 🙂 Welcome to the Skyscanner bot. We’ll help you find your next trip ✈. What would you like to do?

  • Find a flight deal
  • Tell me more

I guess I should “Find a flight deal”.

OK, let’s start again… where would you like to go?

“New York City”. If you can get me there, you can get me anywhere.

A flight from New York, United States

  • Change origin city
  • Set as home airport

Where are you going?

  • Don’t know
  • Show weekend trips

Err, no, hold on, you asked me where I would like to go. I’m in London right now. I want to go from London. Click, tap, London.

A flight from London, United Kingdom

  • Change origin city
  • Set as home airport

Where are you going?

“New York City.”

A flight from New York City.

  • Change origin city
  • Set as home airport

Where are you going?

  • Don’t know
  • Show weekend trips

Oh Jesus, I’ve managed to get this piece of shit to go into an infinite loop just by asking it to go from London to New York.

I’m not kidding.

I was going to write a piece about how chatbots necessarily have to provide an experience that doesn’t allow the user to control the details of a somewhat complicated, but ordinary, process like booking an international flight. That compared to using the website or the app, you’d be less able to control some of the complex matrix of choices you make when booking a flight. I was also going to point out that chatbots do not allow for comparison shopping in a way that, say, tabbed browsing or multiple windows does. Being able to open up three different airlines in three different tabs and search their websites to compare and contrast.

Instead, I ended up talking to a chatbot created by a major travel comparison site that failed to grasp the difference between the city I wished to depart from and my destination. I’d understand if this were a weekend hackathon project built by some people dosed up on energy drinks. But this is a product from one of the companies in the top five of comparison sites.

While faffing around with this ridiculous bot, I did get a nice message from a human at Expedia directing me to their website, their phone lines and wishing me a nice weekend. All of which is much more useful than a bloody chatbot that can’t decide whether I’m talking about whether I’m going to New York or from New York.

From this experience, chatbots are clearly the future of human-computer interaction. On behalf of programmers everywhere, I implore you to waste a few more billions on building them, even if they don’t actually help your customers achieve what they want to do.




The Chrome plugin Magnetify does what Spotify ought to already do: make it so it just opens the desktop app when you click a Spotify link in your browser.

The small army of browser extensions one has to install to keep using the web a tolerable experience is pretty telling.


URL Tracking Snipper is a Chrome extension that removes all those nasty utm_source parameters from the query strings of URLs. Works very nicely. I already try to remove these parameters by hand, but automating the process is helpful.

Beyond being potentially creepiness-enabling, tracking parameters make things like bookmarks less useful. If I bookmark URL x, and then I bookmark URL x1 which is the same content as URL x, but has a load of tracking crap on the end of the URL, my bookmarking tools (Pinboard in my case) may not tell me I have already bookmarked it.

These parameters also misleading. If you post, say, a link with UTM parameters saying that the link came from Twitter and then I share that link via another channel, your analytics data now says you’ve got lots of people visiting via Twitter. But they didn’t come from Twitter—they came from someone who shared a link from Twitter in another channel. “Cool URIs don’t change” (said Tim Berners-Lee), but cool URIs also don’t replicate unnecessarily. In an ideal world, we’d have one URL for each piece of content.

These irritating query parameters make the lives of web users more frustrating solely to benefit somewhat creepy ad-tech people. Snapping these nasty protrusions off URLs benefits everyone. Except marketers. Oh well. I guess we’ll have to live with that.