This article was written by guest author – and GitKon speaker – Théophane Huffschmitt.
By day, Théophane leads the Nix group at Tweag/Modus Create, and by night, he’s a NixOS Foundation board member. For the last several years, Théophane has worked as a consultant in functional programming before turning to build systems, then Nix and the Nix community. He’s active at both the technical level (active contributor and maintainer of Nix) and the organizational one through the Foundation.
Hear more about this topic in Théophane’s GitKon session, Intro to Nix: Keep Your Environment in Control, Wherever You Are. Watch on-demand here (expand the video description and use the chapters to jump to his talk).
Limitations of Traditional Development Tools
Like many other kids of my generation, my very first experience of programming came out of boredom during some math classes in high school, where I couldn’t do anything more than play with my calculator (a TI-89 titanium, if I recall correctly).
Thus, I learned TI-BASIC, the dialect of BASIC used to program it and started toying with it, building small games, animations or utility programs to solve some common math problems. This was great, particularly because I could directly program on the calculator itself without any setup.
However, I eventually got frustrated by the limitations of TI-BASIC, and in particular, its incredibly slow and limited graphical abilities. It turns out that these can be overcome by bypassing the BASIC layer and directly programming the calculator in C (or assembly for those brave and foolish).
Unfortunately, programming on the calculator wasn’t really a viable option anymore, and instead, I had to install the proper toolchain on my computer, along with the tooling to send the programs to my calculator, maybe an emulator to speed things up and so on. For some reasons that I can’t recall, installing these turned out to be fraught with challenges, to the point that I eventually gave up and never wrote a single C program for my dear calculator.
This is but a small anecdote with zero consequence (except maybe me getting mildly bored for a few hours a week). However, this story repeats itself every day whenever a new team member joins and spends hours or even days setting up the right environment to build the project. Or whenever a potential contributor never shows up because he couldn’t even manage to build this open-source project he wanted to contribute to.
A similar story also unfolds whenever someone changes a bit the requirements of a program, and it suddenly doesn’t build anymore on their coworker’s laptop or the CI machines. Or worse, it still builds and runs but behaves improperly under obscure conditions because it uses a slightly different library or runtime.
A New Approach to Development Environments with Nix
It would be so much better if it were possible to define the required environment in a way that the machine can understand. That way, new joiners could just get a program to read these requirements and set up the environment automatically. Changing the project’s dependencies wouldn’t break other people’s setups as the tool would automatically adapt them.
This is a relatively old idea in computer science that got popularized nearly 20 years ago, with tools like python’s easy_install or nodejs’s npm offering a declarative description of your project’s dependencies. Then tools like terraform or Docker solved these at another layer by offering you to describe and distribute fully isolated environments for your software to run on.
None of these solutions is totally satisfying, though. Language-specific package managers fall short as soon as the project requires something outside of their ecosystem (even the so-called “native dependencies” that everyone relies on can be the source of many troubles).
Containers or VM solutions tend to be both too heavy to use (fully isolated environments is generally not what you want for iterative development) and hard to maintain in the long run (complex Dockerfiles can quickly become a very daunting beast). They can be used in concert to alleviate some of these, but this still leaves a number of unsatisfying flaws.
Nix provides a third approach, bridging the gap between the simplicity and effectiveness of language-specific package managers and the genericity of containers. It can do that by combining three key properties:
- It is totally agnostic to the underlying ecosystem, meaning it can equally handle a C dependency, a javascript one, a COBOL one, or even a mixture of the three. And being backed by Nixpkgs, which is the largest generalist software collection in the world, it will, in practice, handle most of these out of the box. And while Nixpkgs won’t know about the whole of Npm or crate.io, it can complement your favourite package manager to give you the best of both worlds.
- It is totally declarative. Unlike Dockerfiles or Vagrantfiles, which rely on executing imperative shell scripts on the machine and on the user to make these reliable and reproducible, Nix just lets you declare the dependencies that you want, making sure that they will each be installed independently from each other. This also means that changing something in the environment (like adding or removing a dependency) is guaranteed to only affect what it’s supposed to affect and not touch anything else.
- It knows when to be strict and when to give the users some slack. While building the environment is done in very strict sandboxes to guarantee the reliability of the result, this environment itself is merely tweaking some environment variables to make the required dependencies available alongside whatever was already on the machine so that developers can keep using their tools and configuration. When needed (on the CI, for instance), it can be strengthened up to the same rigid sandbox used for the builds or have it produce OCI images to run your build in a container.
In Conclusion
There’s no silver bullet in this world, and I wouldn’t pretend that Nix is an exception to that rule. However, I’ve found it increasingly relevant, both in industrial and open-source settings, as one of the most impactful tools for improving the developer’s life.
Like most innovative tools in the history of computing, from Lisp to Docker through Unix and Make, Nix represents by itself a whole galaxy of use cases and solutions, and this is barely scratching the surface of what Nix offers. But this ability to declare reliable environments alone makes it a tool worth checking out and embracing. Click here to watch my on-demand talk from this year’s GitKon for more on this topic.
