A little Nix fix

I’ve been looking at Nix for the last few years with some curiosity. The first time I heard of it, I was at EMFCamp and downloaded it on my Mac while in a talk (yup, great decision). I installed it only for it to create some strange file system and want to modify my fstab which is vaguely terrifying for a random thing I’ve downloaded off the Internet. I meticulously copied down the results of the install script worried that like previous packaging systems on macOS (I’ve used Fink, MacPorts and Homebrew, and all of them have found ways to irritate me), it’d cause a whole load of mess that I would have to tidy up, which would take an enormous amount of time and effort.

I didn’t particularly want to delve through that big pile of complexity and so on the backburner it went. And “backburner”, I mean “list of things I’d like to know one day”. Recently, I decided to give it a proper try, and then share what I’ve learned.

The pitch

When I first used it, I wrote in my notes: “Brazil Nix is the country package manager of the future, and always will be”. The problems: documentation is sparse and often outdated, and the documentation which exists usually contains omissions or assumes knowledge that a novice lacks. Often simple things like “where exactly is this file supposed to go?” These are fixable, though, and the promise of what Nix offers is compelling enough.

I’ve been stuck in situations in the past where I’ve needed to deploy two different packages to a server that both rely on, say, Python, but require different CPython versions and build against different versions of the same C libraries. You then add the abstractions of unpleasant YAML-as-poorly-specified-programming-language DevOps tools like Ansible to the mix, and deploying and managing a simple Python script suddenly becomes a lot of work. The reaction to this unpleasantness has been to shift towards language-specific package platforms, which then breed a cacophony of CLI tooling (pip, pipx, virtualenvwrapper, setuptools, venv, pip-tools, conda, poetry, PDM, hatch) all with cognitive overhead.

When that got too much, there was a shift towards virtualisation-based abstractions (remember Vagrant?) and then on to containers, where it’s considered totally normal to wrap a few hundred lines of Rust that compiles into a megabyte or so of executable in a gigabyte of Linux dependencies. Suddenly, inter-process communication becomes a cross-container networking problem, which in turn means a whole lot of YAML code to so that container B can expose a network port to container A, and container A can discover the IP address of container B.

(Great, we’ve turned writing and running a few hundred lines of Python into DNS management problem, plus you get to download Debian over and over—truly, bravo!)

You then also waste a lot of time trying to get language servers to talk to the container, especially if you’d rather use something that isn’t Visual Studio Code. Plus you also have the constant fun of there being a mismatch between your development environment and the container—good luck if you’re trying to write code that relies on, say, your GPU, or have fun fixing your Home Assistant setup when Debian randomly breaks the way it mounts serial devices so that Docker stops exposing a USB device from the host to the container. Also, if you’re doing this in return for money rather than recreationally, you get to add “container security monitoring” to the list of things that’ll go wrong in annoying ways.

I guess the next step after the container revolution is a shift to developers using thin clients and cloud dev environments, with surveillance baked right in to make tech even more corporate and joyless.

Enter the pitch for Nix: reproducible package and environment management that works across platforms and with multiple languages, and without having to containerise absolutely everything. After all that waffle, let’s give it a spin.

As a Homebrew (not quite) replacement…

I installed Nix on a Mac that already had Homebrew, but which did not have a lot of packages installed. I followed the standard guide for a multi-user installation and it pretty much Just Worked.

In ~/.config/nix/nix.conf I added:

experimental-features = nix-command flakes

nix-command gives you a single nix CLI command. It’s still new, according to the wiki page. flakes enables Nix Flakes. They’re a thing I wanted to play around with. I am aware of the risk of filling the Internet with words of dubious accuracy about things I don’t understand, so I refer you to people who actually know.

I ensured that the Nix paths were added to my $PATH: ~/.nix-profile/bin and /nix/var/nix/profiles/default/bin.

Then let’s install a package.

nix profile install nixpkgs#wget

Now look at an example of the chain of symlinks starting at ~/.nix-profile/bin/wget. Here they are on my computer (after installing a load of other packages).

~/.nix-profile links to ~/.local/state/nix/profiles/profile which links to ~/.local/state/nix/profiles/profile-16-link which links to /nix/store/5arz7ajiip0fgcbk28hgp5cw1qkvadac-profile.

~/.nix-profile/bin/wget links through that chain of directory links to /nix/store/hhldp21b1i4yj4nsb7ghr5115q0vvb76-wget-1.21.4/bin/wget.

Notice profile-16-link? This is version 16. If I type in nix profile history it shows me all 16 versions and what changed in each one - the adding, upgrading or removing of each package. I can rollback to the previous version—it’s not uninstalling the package, it’s repointing the symlink to a profile that does not contain it anymore. The package can still lurk there so other profiles can point to it, until you run the garbage collector which removes objects that are no longer in use.

Migrating packages over from Homebrew then is fairly straightforward. Run brew leaves (which shows the packages you’ve installed without dependencies), look up each package in the Nix package repository (web version or nix search nixpkgs neovim) and install it.

Problems: Casks, Emacs and other limitations

The first real problem I had was after doing a macOS point update. /etc/zshrc broke. The is a known problem, but an annoyance nonetheless. I personally think the best way is to assume the system-level shell config will break on update and instead just put the config in a user-level config.

Beyond that irritations, there are some limitations on going full Nix on macOS. The big thing that you can’t get through Nix packages are Homebrew casks. For macOS GUI app, probably stick with Homebrew casks.

There are good reasons for this: Mac GUI applications have a habit of self-updating, possibly through something like Sparkle or equivalent. There’s also the fun and joy of macOS code signing. Applications magically self-updating are something of a mismatch with reproducible builds.

Also, if you’re an Emacs user on macOS and you want an Emacs.app in your /Applications folder, getting it from Homebrew through a tap is about the best I can suggest. My preferred poison:

$ brew tap railwaycat/emacsmacport
$ brew install railwaycat/emacsmacport/emacs-mac --with-native-comp --with-natural-title-bar
$ cp -a $(brew --prefix)/opt/emacs-mac/Emacs.app /Applications

The other thing you’ll need to do: make sure the Nix paths are respected by any GUI-based editors or IDEs.

Throwaway environments and scripts

Another very quick easy win I found having Nix there - quick scripts with nix-shell.

I had to write a tiny little one-off script today to parse some data out of a webpage. My go-to tools for this are Python and bs4. My usual approach would be to create a directory, poetry init, then poetry add some packages.

Instead, I used nix-shell to get a Python environment with the packages I wanted.

nix-shell -p python311 python311Packages.beautifulsoup4 python311Packages.httpx

I can now program in the REPL by typing python3 and bs4 and httpx are there to use. I noodle about with the REPL, and copy bits I like into a .py file. Once I’m happy, I add the following to the top…

#! /usr/bin/env nix-shell
#! nix-shell -i python3 -p python311 python311Packages.beautifulsoup4 python311Packages.httpx

Now, when I run the script, it creates an environment with python311 and the two Python packages, then runs the script through python3.This is broadly equivalent to pipx run, but with the advantage that it’s not just limited to Python. The Nix wiki lists a bunch of other examples.

What next?

This is a first little taste. The Nix community has built a lot of stuff, and it’s quite easy to look at it and think “there’s way too much here, I’m never going to learn all this” and then give up. I rather like that you can mostly just use it as a package manager that works across both macOS and Linux (on a constrained but modern set of platforms—x86-64 and ARM), and get most of the benefits of isolated, versioned, reproducible development environments without the overhead of containers.

Some people use home-manager to manage their home directory and dotfiles. It looks useful, but if you’ve already got a dotfiles management tool like chezmoi you are almost certainly going to spend a bunch of time porting over stuff from your existing solution, perhaps for only minimal benefit.

Writing your dotfile configs in the Nix language (rather than plain text files with some kind of templating system like chezmoi and friends do) promises type checking—maybe editor support, eventually. Which would be nice compared to the TOML/YAML+templating preprocessor logic that’s normal in both config files and the DevOps world. In practice, I found myself trying to move my .gitconfig over having to go online to work out what the equivalent is, then decided this wasn’t worth the time and effort.

The rough conclusion I’ve come to: it’s pretty good. As a replacement for most of Homebrew (with the exception of Mac GUI apps through casks and other oddities like Emacs), it’s decent. As a server OS—why not? It’s fun to play around with in a VM, and I may try sticking it on a Raspberry Pi soon.

The problems that I started with are still there though. The documentation is sort of there, but you do have to wander off a bit into random blog posts and forum posts to really find stuff. I still don’t quite grasp the Nix language. The CLI tooling is improving, but it is still a bit alien compared to familiar friends like apt or brew. There learning curve and complexity is still daunting. Consider though, the status quo: for development environments, we have language-specific tooling (and way too much of it). And then there’s containerising everything (and seemingly consensus towards nasty templated YAML files for config). Both of those are layers of complexity that we may not recognise as such due to familiarity. Nix may always remain the future, but it is solving the right problems, and it is pretty fun to play with.

Disclaimer: this is not documentation. It is a blog post in the old school journal way where one plays with stuff and shares it. I do not guarantee I’m doing it the right way. For what it’s worth, it also does not constitute legal, medical or financial advice. Or life advice.