A little Nix fix
I’ve been looking at Nix for the last few years with some curiosity. The first time I heard of it, I was at EMFCamp and downloaded it on my Mac while in a talk (yup, great decision). I installed it only for it to create some strange file system and want to modify my fstab which is vaguely terrifying for a random thing I’ve downloaded off the Internet. I meticulously copied down the results of the install script worried that like previous packaging systems on macOS (I’ve used Fink, MacPorts and Homebrew, and all of them have found ways to irritate me), it’d cause a whole load of mess that I would have to tidy up, which would take an enormous amount of time and effort.
I didn’t particularly want to delve through that big pile of complexity and so on the backburner it went. And “backburner”, I mean “list of things I’d like to know one day”. Recently, I decided to give it a proper try, and then share what I’ve learned.
The pitch
When I first used it, I wrote in my notes: “Brazil Nix is the
country package manager of the future, and always will
be”.
The problems: documentation is sparse and often outdated, and the documentation
which exists usually contains omissions or assumes knowledge that a novice lacks.
Often simple things like “where exactly is this file supposed to go?” These are
fixable, though, and the promise of what Nix offers is compelling enough.
I’ve been stuck in situations in the past where I’ve needed to deploy two different packages to a server that both rely on, say, Python, but require different CPython versions and build against different versions of the same C libraries. You then add the abstractions of unpleasant YAML-as-poorly-specified-programming-language DevOps tools like Ansible to the mix, and deploying and managing a simple Python script suddenly becomes a lot of work. The reaction to this unpleasantness has been to shift towards language-specific package platforms, which then breed a cacophony of CLI tooling (pip, pipx, virtualenvwrapper, setuptools, venv, pip-tools, conda, poetry, PDM, hatch) all with cognitive overhead.
When that got too much, there was a shift towards virtualisation-based abstractions (remember Vagrant?) and then on to containers, where it’s considered totally normal to wrap a few hundred lines of Rust that compiles into a megabyte or so of executable in a gigabyte of Linux dependencies. Suddenly, inter-process communication becomes a cross-container networking problem, which in turn means a whole lot of YAML code to so that container B can expose a network port to container A, and container A can discover the IP address of container B.
(Great, we’ve turned writing and running a few hundred lines of Python into DNS management problem, plus you get to download Debian over and over—truly, bravo!)
You then also waste a lot of time trying to get language servers to talk to the container, especially if you’d rather use something that isn’t Visual Studio Code. Plus you also have the constant fun of there being a mismatch between your development environment and the container—good luck if you’re trying to write code that relies on, say, your GPU, or have fun fixing your Home Assistant setup when Debian randomly breaks the way it mounts serial devices so that Docker stops exposing a USB device from the host to the container. Also, if you’re doing this in return for money rather than recreationally, you get to add “container security monitoring” to the list of things that’ll go wrong in annoying ways.
I guess the next step after the container revolution is a shift to developers using thin clients and cloud dev environments, with surveillance baked right in to make tech even more corporate and joyless.
Enter the pitch for Nix: reproducible package and environment management that works across platforms and with multiple languages, and without having to containerise absolutely everything. After all that waffle, let’s give it a spin.
As a Homebrew (not quite) replacement…
I installed Nix on a Mac that already had Homebrew, but which did not have a lot of packages installed. I followed the standard guide for a multi-user installation and it pretty much Just Worked.
In ~/.config/nix/nix.conf
I added:
experimental-features = nix-command flakes
nix-command
gives you a single nix
CLI command. It’s still new, according
to the wiki page. flakes
enables Nix
Flakes. They’re a thing I wanted to play
around with. I am aware of the risk of filling the Internet with words of
dubious accuracy about things I don’t understand, so I refer you to people who
actually know.
I ensured that the Nix paths were added to my $PATH
: ~/.nix-profile/bin
and
/nix/var/nix/profiles/default/bin
.
Then let’s install a package.
nix profile install nixpkgs#wget
Now look at an example of the chain of symlinks starting at
~/.nix-profile/bin/wget
. Here they are on my computer (after installing a
load of other packages).
~/.nix-profile
links to ~/.local/state/nix/profiles/profile
which links to
~/.local/state/nix/profiles/profile-16-link
which links to
/nix/store/5arz7ajiip0fgcbk28hgp5cw1qkvadac-profile
.
~/.nix-profile/bin/wget
links through that chain of directory links to
/nix/store/hhldp21b1i4yj4nsb7ghr5115q0vvb76-wget-1.21.4/bin/wget
.
Notice profile-16-link
? This is version 16. If I type in nix profile
history
it shows me all 16 versions and what changed in each one - the adding,
upgrading or removing of each package. I can rollback to the previous
version—it’s not uninstalling the package, it’s repointing the symlink to a
profile that does not contain it anymore. The package can still lurk there so
other profiles can point to it, until you run the garbage collector which
removes objects that are no longer in use.
Migrating packages over from Homebrew then is fairly straightforward. Run brew
leaves
(which shows the packages you’ve installed without dependencies), look
up each package in the Nix package repository (web
version or nix search nixpkgs neovim
) and install
it.
Problems: Casks, Emacs and other limitations
The first real problem I had was after doing a macOS point update. /etc/zshrc
broke. The is a known problem, but
an annoyance nonetheless. I personally think the best way is to assume the
system-level shell config will break on update and instead just put the
config in a user-level config.
Beyond that irritations, there are some limitations on going full Nix on macOS. The big thing that you can’t get through Nix packages are Homebrew casks. For macOS GUI app, probably stick with Homebrew casks.
There are good reasons for this: Mac GUI applications have a habit of self-updating, possibly through something like Sparkle or equivalent. There’s also the fun and joy of macOS code signing. Applications magically self-updating are something of a mismatch with reproducible builds.
Also, if you’re an Emacs user on macOS and you want an Emacs.app
in your
/Applications
folder, getting it from Homebrew through a tap is about the
best I can suggest. My preferred poison:
$ brew tap railwaycat/emacsmacport
$ brew install railwaycat/emacsmacport/emacs-mac --with-native-comp --with-natural-title-bar
$ cp -a $(brew --prefix)/opt/emacs-mac/Emacs.app /Applications
The other thing you’ll need to do: make sure the Nix paths are respected by any GUI-based editors or IDEs.
Throwaway environments and scripts
Another very quick easy win I found having Nix there - quick scripts with
nix-shell
.
I had to write a tiny little one-off script today to parse some data out of a
webpage. My go-to tools for this are Python and
bs4. My usual approach
would be to create a directory, poetry init
, then poetry add
some packages.
Instead, I used nix-shell
to get a Python environment with the packages I wanted.
nix-shell -p python311 python311Packages.beautifulsoup4 python311Packages.httpx
I can now program in the REPL by typing python3
and bs4
and httpx
are
there to use. I noodle about with the REPL, and copy bits I like into a .py
file. Once I’m happy, I add the following to the top…
#! /usr/bin/env nix-shell
#! nix-shell -i python3 -p python311 python311Packages.beautifulsoup4 python311Packages.httpx
Now, when I run the script, it creates an environment with python311
and the
two Python packages, then runs the script through python3
.This is broadly
equivalent to pipx run
, but with the
advantage that it’s not just limited to Python. The Nix
wiki lists a bunch of other
examples.
What next?
This is a first little taste. The Nix community has built a lot of stuff, and it’s quite easy to look at it and think “there’s way too much here, I’m never going to learn all this” and then give up. I rather like that you can mostly just use it as a package manager that works across both macOS and Linux (on a constrained but modern set of platforms—x86-64 and ARM), and get most of the benefits of isolated, versioned, reproducible development environments without the overhead of containers.
Some people use home-manager to manage their home directory and dotfiles. It looks useful, but if you’ve already got a dotfiles management tool like chezmoi you are almost certainly going to spend a bunch of time porting over stuff from your existing solution, perhaps for only minimal benefit.
Writing your dotfile configs in the Nix language (rather than plain text files
with some kind of templating system like chezmoi and friends do) promises type
checking—maybe editor support, eventually. Which would be nice compared to the
TOML/YAML+templating preprocessor logic that’s normal in both config files and
the DevOps world. In practice, I found myself trying to move my .gitconfig
over having to go online to work out what the equivalent is, then decided this
wasn’t worth the time and effort.
The rough conclusion I’ve come to: it’s pretty good. As a replacement for most of Homebrew (with the exception of Mac GUI apps through casks and other oddities like Emacs), it’s decent. As a server OS—why not? It’s fun to play around with in a VM, and I may try sticking it on a Raspberry Pi soon.
The problems that I started with are still there though. The documentation is
sort of there, but you do have to wander off a bit into random blog posts and
forum posts to really find stuff. I still don’t quite grasp the Nix language.
The CLI tooling is improving, but it is still a bit alien compared to familiar
friends like apt
or brew
. There learning curve and complexity is still
daunting. Consider though, the status quo: for development environments, we
have language-specific tooling (and way too much of it). And then there’s
containerising everything (and seemingly consensus towards nasty templated YAML
files for config). Both of those are layers of complexity that we may not
recognise as such due to familiarity. Nix may always remain the future, but it
is solving the right problems, and it is pretty fun to play with.
Disclaimer: this is not documentation. It is a blog post in the old school journal way where one plays with stuff and shares it. I do not guarantee I’m doing it the right way. For what it’s worth, it also does not constitute legal, medical or financial advice. Or life advice.