Overview
Contents
Okay, before we get started:
NO, NIX IS NOT JUST AN OPERATING SYSTEM. I'm going to talk about what it is and what you can use it for, but nixos is merely a project that uses the nix language/tooling. You do not need to even use linux to use nix. Okay, now that we have that out of the way let's get started.
Why Nix?
Before even grappling with what it is, I think it's good to understand what it can do. Some awesome things that nix does:
- Consistent developer environments that are decoratively specified. This means everyone working on a project can have the same LSP, binaries in their $PATH, etc.
- Pure, reproducible project builds. Nix "wraps" onto tooling you already use for builds, and does the build in a sandbox. This means that there's no internet access, the time is set to
0(UTC) (which means things like Latex "builds", which inject timestamps, will be reproducible too!), and more. - Let you configure your home directory using a project called "home manager." You can specify how to set up various programs and their configurations fully declarativly with
nixcode. - Create very minimal docker images, that don't have a
FROM(that areFROM SCRATCH), and only have exactly the things you need to dockerize your project. - Get an android emulator going in 4 lines of code. And a million other things.
Where nix really shines is reproduability. If it works once, it will probably work again.
Some Terms
What is purity? Purity just means when you put X, Y, and Z in, you'll get W out. If you put X, Y, and Z in two weeks later, on a Mac, in a different time zone, you'll get W out. If you install a different C compiler, you'll still get W out. Nix can guarantee that your builds are pure.
What is lazy?
Lazy just means that the language won't try to evaluate anything it does not absolutely need to evaluate. If you import every package in existence, nixpkgs, nix will not literally build everything. But if you reference foobar from nixpkgs then it will build it.
What is Nix?
The question with so many answers. Nix is at least 4 totally different things.
- It's a programming language. It's functional, lazy, and can be pure. You can do basic things like
{ a, b }: a + b! You can also use it to do more advanced tasks like mapping over arrays or attrsets (dictionaries), and everything you'd want from a programming language. - It's a package manager! The nix repository has more packages than any other package manager (
apt,pacman, etc), and more fresh (new) packages too! All the packages are not actual binary blobs, butnixcode (remember, it's a language!) that defines build instructions for producingderivations(more on this later) containing over 100k different projects. This even includes pure build instructions for things like buildingchromefrom source and packaged patched binaries like jetbrains IDEs (ikk). Your packages are all nicely managed in anix store, again, more on this later. - It's a utility library.
nixpkgsis a library that contains a bunch of utilities that let you do super cool things, like defining pure builds for minimalFROM SCRATCHdocker images. It fixes all the bloat ofdocker! - It's an operating system. Because it gives you so much power in specifying an exact state of a build output (a "derivation"), people have used it to create nixos. NixOS is a completely decoratively specified Linux distribution where your entire computer configuration lies in
.nixtext files. I have configured my computer this way, and, though it was a long process, the declarativity/reproducitivity is really nice.
Derivations
The "nix store"
There is this thing called the "nix store". It's just a directory that's read only, usually at /nix/store. You can write nix code, using the derivation keyword, to create them, but that's very low level. I'll let you read more about them here, but in this post we'll just use pkgs.stdenv.mkDerivation, along with trivial builders like pkgs.writeShellScriptBin. They are handy helper functions from the nixpkgs utilities (remember when I said nix was a utility library!). Everything in the nixpkgs registry outputs derivations. These "derivations" are resulting directories in this "nix store", that look like «derivation /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv» -- they are just subdirectories of the store. The outputs are read only because they are in the store, and we know that everything that made its way to the store did so in a way that was pure.
If you're interested in how derivations actually work, there's some intermediate steps, including producing a drv file with instructions to nix on how to create it. But I'm going to skip over that.
Flakes
Flakes are really just entrypoints. The older way to do things with nix was to use channels. I don't want to go too far into those, since they are lame (sorry, it's true), but basically, they are global references to sources. They'd let you specify internet inputs to projects with <> syntax:
1{ pkgs ? import <nixpkgs> { }, }: 2...
Where we by default get nixpkgs from the global "channel" (iickk)! That's impure. What if nix decides to update their unstable branch (nixpkgs is just a git repo!) and your package references break?
So from this point on I'm going to pretend channels don't exist, and completely stop using them (mostly). Flakes are nix's solution to the problem. They guarantee real purity.
A flake has this basic structure:
1# flake.nix 2{ 3 inputs = {}; # Specify inputs with URIs 4 outputs = {self, ...}: {}; # A function! 5}
It lives in a flake.nix
Where we usually make inputs at least have nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";, the nix package registry, with all the packages we may ever want. self is a reference to the project at its previous commit, and flakes require you to use version control to ensure purity of file inputs.
You might be thinking, "hey, github:nixos/nixpkgs/nixos-unstable doesn't sound very pure!" True. flakes generate lockfiles with reversions and sha256 hashes. When we run nix flake lock we get something like this.
1// flake.lock 2{ 3 "nodes": { 4 "nixpkgs": { 5 "locked": { 6 "lastModified": 1719690277, 7 "narHash": "sha256-0xSej1g7eP2kaUF+JQp8jdyNmpmCJKRpO12mKl/36Kc=", 8 "owner": "NixOS", 9 "repo": "nixpkgs", 10 "rev": "2741b4b489b55df32afac57bc4bfd220e8bf617e", 11 "type": "github" 12 }, 13 "original": { 14 "owner": "NixOS", 15 "ref": "nixos-unstable", 16 "repo": "nixpkgs", 17 "type": "github" 18 } 19 }, 20 "root": "root", 21 "version": 7 22}
Flakes let us define ways to get to derivation (the things that "build their way into" the nix store).
A C hello world
A grain of salt: I don't do much C. But I'm going to kick off this section with a C hello world, specifically because C is pretty easy to compile, and has a build time requirement: a compiler like gcc.
This is a bit boring, but the way we package this will be helpful for some more fun stuff later.
First, we make a hello.c with a hello world...
1#include <stdio.h> 2 3int main() { 4 printf("Hello, World!\n"); 5 return 0; 6}
Okay, we've created the hello world. Now time to define the build with nix.
To start, a tiny bit more on flakes.
With flakes, there's some special outputs that nix will look for.
packages.${system}.default- If we're running linux (this works on mac too though), this might be
packages.x86_64-linux.default. This is basically the "default" thing that gets build when we donix build, but if we specify something else, likepackages.${system}.foobar, then we can build the derivation with thenixcli usingnix build .#foobarinstead.
- If we're running linux (this works on mac too though), this might be
This means that a flake often will look something like
1{ 2 inputs = { ... }; 3 outputs = { self, ... }: 4 { 5 packages.x86_64-linux = { 6 default = pkgs.writeShellScriptBin "hello" "echo 'hello!'" 7 }; 8 }; 9}
Having to define an export for every system when our project isn't really system specific is annoying, so nixers often use something called flake-utils, which provides a helpful eachDefaultSystem utility to generate the different outputs for different systems for us. Nix is lazy, so it'll only build for the system we need to build for when we run nix build.
1... 2 outputs = 3 { 4 self, 5 nixpkgs, 6 flake-utils, 7 }: 8 flake-utils.lib.eachDefaultSystem ( 9 system: 10 let 11 pkgs = import nixpkgs { inherit system; }; 12 in 13 { 14 packages = { 15 default = # (The derivation / builder) 16 }; 17...
eachDefaultSystem is a function that intakes a function that takes an argument system, that outputs the regular outputs based on it, and then provides outputs for all systems supported by nix (but only evaluates when you need to).
Returning to our C hello world, we will use a helper function called pkgs.stdenv.mkDerivation. nix ships with a derivation keyword, but that takes a binary that expects to make the derivation, and is super raw. This helper will do a lot of the heavy lifting by breaking things up into stages -- the unpack phase, build phase, and install phase (and a ton others -- see here).
buildInputs specifies what should be available in the path of our builder's environment. We can use nativeBuildInputs since our buildInputs are only needed during building -- native implies that the program itself doesn't need them.
The Sandbox
All nix builds happen in a special sandbox. The src argument specifies what should exist in the sandbox. We need to move the source code over!
The sandbox does a lot to ensure that we are declarative and the output is pure.
When sandbox builds are enabled, Nix will set up an isolated environment for each build process by constraining build inputs to improve reproducibility.
It is achieved by isolating build jobs from input sources whose contents are prone to change dynamically and without notice. For example, the main file system hierarchy is completely bypassed to prevent depending on files in global directories, such as
/usr/bin, where a reference to an executable may point to different version as time goes by.
Also, it does things like disallow networking and sets the timestamp to UNIX 0 (even time is dynamic and could lead to impurity!).
Okay. So let's look at the flake...
1{ 2 description = "C Hello world"; 3 4 inputs = { 5 flake-utils.url = "github:numtide/flake-utils"; 6 nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; 7 }; 8 9 outputs = 10 { 11 self, 12 nixpkgs, 13 flake-utils, 14 }: 15 flake-utils.lib.eachDefaultSystem ( system: 16 let 17 pkgs = import nixpkgs { inherit system; }; 18 in 19 { 20 packages.default = pkgs.stdenv.mkDerivation { 21 pname = "hello"; 22 version = "1.0"; 23 src = self; # code at the last commit 24 nativeBuildInputs = [ pkgs.gcc ]; 25 buildPhase = '' 26 gcc -o hello ./hello.c 27 ''; 28 installPhase = '' 29 mkdir -p $out/bin 30 cp hello $out/bin 31 ''; 32 }; 33 } 34 ); 35}
We're not quite ready. Remember when I said that we need to use version control to help our flakes guarantee purity?
1git init 2git add *.c *.nix 3git commit -m "Initial commit"
Okay, finally we can build and run it with nix.
nix run
# OR
nix build && ./result/bin/hello
And we get a "hello world"!
pkgs.stdenv.mkDerivation resolves to a string (that's right, a piece of text). It's a path to a result in the nix store. nix build helps us out by making a symlinked folder ./result that directs to that directory in the nix store (that is read only, because it's in the nix store).
Okay, enough boring hello worlds in C nonsense. Here's something fun -- binary runtime dependencies, and Python!
A simple Python Script
Let's consider a simple Python script that uses selenium to navigate to https://example.com. Usually this would be a little annoying since there's a dependency on chrome and chromedriver, but we can bundle those things with nix pretty easily.
Here's our basic script that goes to example.com and yoinks the tab's title, printing it to stdout.
1#./main.py 2 3from selenium import webdriver 4 5if __name__ == "__main__": 6 driver = webdriver.Chrome() 7 driver.get('https://example.com') 8 print(driver.title) 9 driver.quit()
Here we have one dependency, selenium, and two secret ones, chromedriver, and chromium. Usually, we'd do something like
1sudo apt-get install chromium-browser 2sudo apt-get install chromium-chromedriver
God forbid we're on Windows, and it's so much worse. We might have to add
1options = webdriver.ChromeOptions() 2options.binary_location = '/usr/bin/chromium-browser'
So let's package it with nix, which will work on ALL systems (including native macos, and windows with wsl)...
I have a collection of Project Templates for various different languages that use popular nix tools to create derivations for projects. I'm using Cookiecutter, which is a bit of an aside, but lets you template entire projects using Jinja, so you can specify what a "directory" of a, say, Python project would look like. We're going to use my python script template here, which you can get at that repo if you want, but we'll build up to it slowly.
pkgs.writeShellScriptBin
writeShellScriptBin is a nice little helper function
It is so simple, here's the source code...
1writeShellScriptBin = name: text: 2 writeTextFile { 3 inherit name; 4 executable = true; 5 destination = "/bin/${name}"; 6 text = '' 7 #!${runtimeShell} 8 ${text} 9 ''; 10 checkPhase = '' 11 ${stdenv.shellDryRun} "$target" 12 ''; 13 meta.mainProgram = name; 14 };
runtimeShell is just /nix/store/blahblahShaHashblahblah/bin/bash. It literally just makes an executable bash script.
Okay, so let's use it to package our python program...
We'll first write a bash script, that's roughly similar to what we want at the end of the day, but without binary dependencies
1export PATH=$PATH:${pkgs.chromedriver}/bin:${pkgs.ungoogled-chromium}/bin; 2export PYTHONPATH= ??? 3${python}/bin/python3 main.py
Something like that. nix can handle the python packages for us. I'm not going to go too far into that for now. Notice the python3.withPackages. Most popular packages are already packaged with nix, and it's not hard to package ones that aren't (usually, some require messy runtime deps).
1{ 2 description = "Some python script"; 3 4 inputs = { 5 flake-utils.url = "github:numtide/flake-utils"; 6 nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; 7 }; 8 9 outputs = 10 { 11 self, 12 nixpkgs, 13 flake-utils, 14 }: 15 flake-utils.lib.eachDefaultSystem ( 16 system: 17 let 18 pkgs = import nixpkgs { inherit system; }; 19 python = pkgs.python3.withPackages (python-pkgs: [ python-pkgs.selenium ]); 20 in 21 { 22 packages = { 23 default = 24 pkgs.writeShellScriptBin "main" /* bash */ # (treesitter directive) 25 '' 26 export PATH=$PATH:${pkgs.chromedriver}/bin:${pkgs.ungoogled-chromium}/bin; 27 ${python}/bin/python3 main.py 28 ''; 29 }; 30 } 31 ); 32}
Remember to git init, git add and git commit (I didn't say this before, but commiting isn't actually necessary, if you don't nix will still build but complain that your codebase is dirty. Okay, now we're ready.
nix run
It works! Chromedriver should open up, the tab should load example.com, and then it should grab the tab name and close.
This still is slightly impure (although, I should note that nix guarantees pure builds, not pure executions, so we may never be able to be 100% sure that the program will run the same way) though because we're appending to our $PATH, which means that we're expecting things to exist in our PATH that may not. I'd like to avoid "generating" bash scripts, so instead I'm going to use a utility called wrapProgram to set the PATH to whatever we have at build time as our path, which has the necessary deps and is pure (because of nix), and only run the python script with a bash script (to make it executable -- we could use a python shebang too).
Here's what that would look like...
1{ 2 description = "Python with Selenium example"; 3 4 inputs = { 5 flake-utils.url = "github:numtide/flake-utils"; 6 nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; 7 }; 8 9 outputs = 10 { 11 self, 12 nixpkgs, 13 flake-utils, 14 }: 15 flake-utils.lib.eachDefaultSystem ( 16 system: 17 let 18 pkgs = import nixpkgs { inherit system; }; 19 python = pkgs.python3.withPackages (python-pkgs: [ python-pkgs.selenium ]); 20 in 21 { 22 packages = rec { 23 default = pkgs.stdenv.mkDerivation { 24 name = "main"; 25 src = self; 26 dontUnpack = true; 27 buildInputs = [ pkgs.makeWrapper ]; 28 installPhase = '' 29 mkdir -p $out/bin 30 cp ${script}/bin/* $out/bin 31 ''; 32 postFixup = '' 33 wrapProgram $out/bin/main \ 34 --set PATH $PATH:${ 35 pkgs.lib.makeBinPath [ 36 pkgs.chromedriver 37 pkgs.ungoogled-chromium 38 ] 39 } 40 ''; 41 }; 42 script = pkgs.writeShellScriptBin "main" "${python}/bin/python3 main.py"; 43 }; 44 } 45 ); 46}
Bash scripts with ANY binary!
If we're okay with only kinda-pure stuff though, generating bash scripts is really fun with nix. We can write a script using literally any dependency we want.
I'll provide a neat script I made to take screenshots of regions of my screen.
Usually it'd look like this...
1FILEPATH=/tmp/$(uuidgen) 2grim -g "$($slurp)" "$FILEPATH.png" 3wl-copy --type "text/uri-list" "file://$FILEPATH.png" 4notify-send "Successfully saved screen capture!" "The png has been saved to $FILEPATH"
But that means I'd need to add grim (a utility to select a region of your screen on Wayland), wl-copy (a Wayland clipboard cli utility), and notify-send (a notification daemon connector cli) to my global $PATH. That's gross, and not nixy at all. What if I want 50 more random binary dependencies? If you say that's a bad idea, it's probably because you don't want to have to have people install so many things to their global /usr/bin to install it.
I really like this paradigm my friend Trevor showed me...
1partial-screenshot = pkgs.writeShellScriptBin "partial-screenshot.sh" '' 2 slurp=${pkgs.slurp}/bin/slurp; 3 grim=${pkgs.grim}/bin/grim; 4 wl_copy=${pkgs.wl-clipboard}/bin/wl-copy; 5 notify=${pkgs.libnotify}/bin/notify-send; 6 ${builtins.readFile ./scripts/partial-screenshot.sh} 7'';
The various binary dependencies will resolve to their nix store paths, and since nix is lazily evaluated we can use ANY binary, and it will poof into existence at the right store path at build time. builtins.readFile will then inject our regular bash script that uses the stuff...
1FILEPATH=/tmp/$(uuidgen) 2$grim -g "$($slurp)" "$FILEPATH.png" 3$wl_copy --type "text/uri-list" "file://$FILEPATH.png" 4$notify "Successfully saved screen capture!" "The png has been saved to $FILEPATH"
And if we do a nix build, we'll get
1#!/nix/store/agkxax48k35wdmkhmmija2i2sxg8i7ny-bash-5.2p26/bin/bash 2slurp=/nix/store/hfii9xxi8vwmlq86vh2j9dl73zzy7s1w-slurp-1.5.0/bin/slurp; 3grim=/nix/store/jkv33a361c8nlgp2kcx1azncipxdn4nh-grim-1.4.1/bin/grim; 4wl_copy=/nix/store/18rwzxp6m29m8c5bxgpxci1ad1q4kl94-wl-clipboard-2.2.1/bin/wl-copy; 5notify=/nix/store/w141cbf1p9mcyp7vqv6a4fw4hm093qb5-libnotify-0.8.3/bin/notify-send; 6FILEPATH=/tmp/$(uuidgen) 7$grim -g "$($slurp)" "$FILEPATH.png" 8$wl_copy --type "text/uri-list" "file://$FILEPATH.png" 9$notify "Successfully saved screen capture!" "The png has been saved to $FILEPATH"
A nice, executable script, with absolute references to packages that are NOT in our path. This is one of my favorite nix things to do.
Reproducible Developer Environments
Another nice thing nix can do is let you create developer environments very easily. You use pkgs.mkShell, which creates a derivation (remember those still?), and then enters the environment of the derivation.
You can specify buildInputs (although you should use packages) to add all the things you'd want for developing the project.
1devShells = { 2 default = pkgs.mkShell { 3 packages = [ 4 python 5 pkgs.pyright 6 pkgs.black 7 ]; 8 }; 9 };
And then you can plop it in your flake.nix (here's the one we worked on earlier)
1{ 2 description = "{{ cookiecutter.description }}"; 3 4 inputs = { 5 flake-utils.url = "github:numtide/flake-utils"; 6 nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; 7 }; 8 9 outputs = 10 { 11 self, 12 nixpkgs, 13 flake-utils, 14 }: 15 flake-utils.lib.eachDefaultSystem ( 16 system: 17 let 18 pkgs = import nixpkgs { inherit system; }; 19 python = pkgs.python3.withPackages (python-pkgs: [ python-pkgs.selenium ]); 20 in 21 { 22 packages = {}#snip ... 23 devShells = { 24 default = pkgs.mkShell { 25 shellHook = '' 26 # run when they enter 27 echo "Welcome to the dev environment!" 28 ''; 29 packages = [ 30 python 31 pkgs.pyright 32 pkgs.black 33 ]; 34 }; 35 }; 36 } 37 ); 38}
This will give you access to what our output had access to -- it updates our $PYTHONPATH so that our LSP can see the dependencies, so we can get nice red squiggles and all the good language server support. We can add any dependencies we want, and even add shell hooks to set up the developer environment further.
To enter the devshell, you just do nix develop. You can also use direnv, by installing it, and then creating a .envrc with the contents
1use flake
And then typing direnv allow. It's a neat utility made so that when you cd into directories it automatically enters their respective devshell. This is great because you don't need your python and rust and android and javascript and typescript bun node encryption cli tools and so much other crap in the global path. It reduces conflicts, and all developers working on your project can have the same environment.
When you're ready for a real, pure build, you can then just slap in a packages.default, and then you're set.
Some really cool builders
There's a lot of nice nix abstractions out there. This includes 3rd party builders and builtin ones. Some cool ones are
- Poetry2nix to build poetry python projects easily
- buildMavenPackage to build java projects
- buildNpmPackage for building maven packages
- buildGoModule for building go modules
- And builders for most other languages too!
One of the issues with packaging with nix is that the sandbox that the building happens in must use nix's primitive fetchers like fetchURL and fetchTAR before unpacking, and there's no internet during the build step. This poses a challenge since you can't do things like pip install during the build. These fancy builders basically let you do the downloads and specify hashes before the build, using the hashes to guarantee purity (a common nix technique)
Living the Nix Life (NixOS)
Here is a pure, complete, declarative, plug and play NixOS configuration to describe an entire linux system.
1{ 2 description = "An entire system configuration"; 3 4 inputs = { 5 nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; 6 }; 7 8 outputs = { self, nixpkgs, ... }@inputs: { 9 nixosConfigurations.my-nixos = nixpkgs.lib.nixosSystem { 10 system = "x86_64-linux"; 11 modules = []; 12 }; 13 }; 14}
Just a plain old super super minimal linux setup. But this isn't really useful. There's no shell, no packages, no users, nothing useful.
So let's add some user and set up ssh...
1# Credit (inspiration): https://nixos-and-flakes.thiscute.world/nixos-with-flakes/get-started-with-nixos 2{ 3 description = "An entire system configuration"; 4 5 inputs = { 6 nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; 7 }; 8 9 outputs = { 10 self, 11 nixpkgs, 12 ... 13 } @ inputs: { 14 nixosConfigurations.my-nixos = nixpkgs.lib.nixosSystem { 15 system = "x86_64-linux"; 16 modules = [ 17 { 18 users.users.wolf = { 19 description = "wolf"; 20 openssh.authorizedKeys.keys = [ 21 "ssh-ed25519 %3Csome-public-key%3E wolf@wolf-pc" 22 ]; 23 packages = with pkgs; [firefox]; 24 }; 25 services.openssh = { 26 enable = true; 27 settings = { 28 PermitRootLogin = "no"; 29 PasswordAuthentication = false; 30 }; 31 openFirewall = true; 32 }; 33 } 34 ]; 35 }; 36 }; 37}
Other Cool Things I've Come Accross
There's a million neat nix things that I come across each week. Here's a list of some cool ones that might be worth checking out...
Nix Dev Containers on Windows w/Nix WSL
Thanks to this Nix on WSL you can set up developer containers with nix devshells, defined with flakes, on Windows. You can also configure an entire NixOS configuration on Windows. This is much better than using docker dev containers!
Credits
Thanks to Paolo Holinski for inspiration from a Recurse Center nix presentation and Trevor Nichols for getting me into nix in the first place.