How does nix-env knows the store path in advance?

If I query for available packages for Go:

nix-env -qa go -b --put-path --json

I get:

{
  "nixpkgs.go_1_16": {
    "name": "go-1.16.15",
    "pname": "go",
    "version": "1.16.15",
    "system": "x86_64-linux",
    "outputs": {
      "out": "/nix/store/nqi39ksavkfrxkrz3d0797n5wmzi9r30-go-1.16.15"
    }
  },
  "nixpkgs.go": {
    "name": "go-1.17.7",
    "pname": "go",
    "version": "1.17.7",
    "system": "x86_64-linux",
    "outputs": {
      "out": "/nix/store/3v2l94h7pllq6za9km3388cyd5agrln7-go-1.17.7"
    }
  },
  "nixpkgs.go_1_18": {
    "name": "go-1.18",
    "pname": "go",
    "version": "1.18",
    "system": "x86_64-linux",
    "outputs": {
      "out": "/nix/store/7jyfpb96xv3hr8dpfhnbb0f7zscwm7sr-go-1.18"
    }
  }
}

I'm curios, how does nix-env knows the store path in advance? As far as I'm aware, the store path is generated by nix on build time, but nix-env already knows it without building(i see nothing gets downloaded from network for example).

Solution

The build process doesn't directly take .nix code (as downloaded from channels like https://nixos.org/channels/nixos-21.11) as inputs. Instead, it takes derivations as inputs; those derivations are generated by evaluating code written in the Nix language, and then are written into the Nix store as .drv files (which can be readily decoded to JSON for reading; see nix show-derivation). These .drv files are what is used as input by the actual build process (run by the nixbld sandboxed user accounts).

Those derivations fully specify what the build process will do -- all its inputs, and all its outputs. Because the outputs (except for those of fixed-output derivations) are addressed by the hash of the build process used to generate them, it's not necessary to actually run a build before the hash is known.

The exception I mentioned above is a fixed-output derivation; these are used for downloading resources that will be used by the build process off the network. For a Nix build step to be allowed network access, it needs to assert ahead-of-time what the output of that build step will be -- and the step is considered failed if it produces any output that does not match what was stated. In these, the hash of the content itself is used for the store location, so changes to the download process don't require re-downloading content.

The ability to predict the output location pre-build is critical because packages are pulled down from the Hydra binary cache by their hashes. If we didn't know the hashes until we had already built the package, the cache would be useless: We wouldn't know what name to look for the binary under unless we already had that binary in our posession! (And similarly, we wouldn't be able to plan more than one step ahead if we didn't know the names and hashes of our inputs without actually building them).

Let's take a practical example. Take it as given that as of the time that I'm writing this, I don't have GCC installed via Nix.

$ nix repl
Welcome to Nix version 2.3.16. Type :? for help.

nix-repl> :l <nixpkgs>
Added 15472 variables.

nix-repl> gcc8
«derivation /nix/store/9fpas3flqf424g46b8ldkbz7sgd9r7qk-gcc-wrapper-8.5.0.drv»

nix-repl> :b gcc8
...and either a download or a long build process runs here.

Notice how evaluating gcc8 gives us a /nix/store/*-gcc*.drv file. That file contains a plan describing how Nix would build gcc 8 if we wanted it to.

We can demonstrate that that plan includes the output location:

$ nix show-derivation /nix/store/9fpas3flqf424g46b8ldkbz7sgd9r7qk-gcc-wrapper-8.5.0.drv | jq '.[].outputs'
{
  "info": {
    "path": "/nix/store/znasm5jz3pp57ivspw5ahgn7rzfk791w-gcc-wrapper-8.5.0-info"
  },
  "man": {
    "path": "/nix/store/j46y5mrppjw7nw9g8ckd3h438k8jjvkr-gcc-wrapper-8.5.0-man"
  },
  "out": {
    "path": "/nix/store/v9pv2w7qiw1cpbjn4wjdkxkzld7pfki4-gcc-wrapper-8.5.0"
  }
}