Granular builds of Rust projects for Nix
Bring Nix dependency management to your Rust project!
nix develop
or direnv allow
in this repo and see!With nix (with flake support) installed, generate a
Cargo.nix
for your project:
# Use nix to get cargo2nix & rust toolchain on your path
nix develop github:cargo2nix/cargo2nix#bootstrap
# In directory with Cargo.toml & Cargo.lock files (cargo generate-lockfile)
cargo2nix
# Or skip the shell and run it directly
nix run github:cargo2nix/cargo2nix
# You'll need this in version control
git add Cargo.nix
To consume your new Cargo.nix
, write a nix expression like that found in the
hello world example.
A bare minimum flake.nix:
{
inputs = {
cargo2nix.url = "github:cargo2nix/cargo2nix/release-0.11.0";
flake-utils.follows = "cargo2nix/flake-utils";
nixpkgs.follows = "cargo2nix/nixpkgs";
};
outputs = inputs: with inputs;
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [cargo2nix.overlays.default];
};
rustPkgs = pkgs.rustBuilder.makePackageSet {
rustVersion = "1.75.0";
packageFun = import ./Cargo.nix;
};
in rec {
packages = {
# replace hello-world with your package name
hello-world = (rustPkgs.workspace.hello-world {});
default = packages.hello-world;
};
}
);
}
For a more complete project with CI & CD mostly ready to go, check out Unixsocks or cargo2nix's own CI workflow.
# these must be in version control!
git add flake.nix Cargo.nix
nix build
...
...
...
./result-bin/bin/hello
hello world!
Check out our series of example projects which showcase how to use
cargo2nix
in more detail.
In this repo, simply use nix develop
or direnv allow
. Even if you are on
a bare NixOS system or fresh OSX environment with no dependencies or toolchains
installed, you will have everything you need to run cargo build
. See the
devShell
attribute in flake.nix
to see how to prepare this kind of shell.
The workspaceShell
function, created by makePackagSet
accepts all the same options as the nix mkShell
function.
In your flake, you can choose your cargo2nix version by changing the URL.
Flake URL | Result |
---|---|
github:cargo2nix/cargo2nix/ | latest release (check repo's default branch, release-0.11.0) |
github:cargo2nix/cargo2nix/release-0.11.0 | use a specific release |
github:cargo2nix/cargo2nix/unstable | latest features & fixes |
Only use unstable for developing with the latest features. PR's against old releases can be accepted but no active support will be done. The default branch for this repo is updated whenever a new release tag is made. Only specific release branches are "stable."
Update your flake lock with the latest or a specific version of cargo2nix:
nix flake lock --update-input cargo2nix
nix flake lock --update-input cargo2nix --override-input cargo2nix github:cargo2nix/cargo2nix/?rev=d45481420482fa7d9b0a62836555e24ec07d93be
If you need newer versions of Rust or the flake-utils inputs, just specify them using url instead of follows.
makePackageSet
The makePackageSet
function from the
overlay accepts arguments that
adjust how the workspace is built. Only the packageFun
argument is required.
Cargo2nix's own flake.nix has more information.
rustVersion
- is either a version string or YYYY-MM-DD date-string
rustChannel
- "nightly"
"beta"
"stable"
rustProfile
- "default"
or "minimal"
usually
extraRustComponents
- ["clippy" "miri"]
etc
workspaceSrc
- override where the source is supplied relative to the
Cargo.nix
rootFeatures
- a list of foo/feature
strings for workspace crate features
packageOverrides
- control over the individual crate overrides used to make
them compatible on some platforms, for example to tweak C lib consumption
target
- setting an explicit target, useful when cross compiling to obtain a
specific Rust target that doesn't align with the nixpkgs target
rustPkgs
contains all crates in the dependency graph and some extra
conveniences for development. The workspace crates are also exposed via a
workspace attribute.
rustPkgs.<registry>.<crate>.<version>
is an example of a crate function
path. Calling the function results in a completed derivation, which can be used
as a flake output. They support all the normal behaviors such as override
and
overrideAttrs
. See mkCrate.nix for the full set of
arguments the crate function supports.
rustPkgs.workspace.<crate>
are usually the packages you will use. The other
paths look like:
rustPkgs."registry+https://github.com/rust-lang/crates.io-index".openssl."0.10.30"
rustPkgs.workspaceShell
is a derivation using Nix's standard mkShell
,
embelished with information we learned from the dependencies and their
overrides, enabling vanilla cargo build
to work in a nix develop
shell.
This is for finished derivations, not for dependencies. Keep reading below for
using makeOverride
in the dependency tree.
workspaceShell
and crates both support override
and
overrideAttrs
like normal Nix derivations. This allows you to customize
the workspace shell or a build step in your workspace crate very easily. See
nix show-derivation
and nix show-derivation #devShell
for more information.
You can make overrides to packages in the dependency tree. See examples in
overrides.nix. Overriding the buildPhase
etc is
possible for a single crate without modifying mkcrate.nix
in cargo2nix
directly. The output of nix show-derivation
can be valuable when determining
what the current output result is.
The most important function in cargo2nix source is mkcrate.nix because it's how we store information in dependents and replay them back when building dependents. It is vital for building crates in isolation.
The cargo2nix
utility reads the Rust workspace configuration and
Cargo.lock
and generates nix expressions that encode some of the feature,
platform, and target logic into a Cargo.nix
The cargo2nix Nixpkgs overlay
consumes the Cargo.nix
, feeding it what you pass to makePackageSet
to
provide workspace outputs you can expose in your nix flake
Because we know all of the dependencies, it's easy to create a shell from those
dependencies as environment setup using the workspaceShell
function and
exposing the result in the devShell
flake output
Just like regular cargo
builds, the Nix dependencies form a DAG, but purity
means we only expose essential information to dependencies and manually invoke
cargo
. Communication from dependencies to dependents is handled by writing
some extra outputs and then reading those outputs inside the next dependent
build.
There's two broad categories of information that need to be transmitted when hand-building crates in isolation:
Global information
x86_64-unknown-linux-gnu
build
or test
Cargo.nix
expressionsThis information is known before any of the crates are built. It's used at
evaluation time to decide what will be built. See nix show-derivation
results.
Propagated information
Each dependency writes information such as linker flags alongside its rlib and other outputs. When the dependent is going to consume the dependency, it reads this information back.
Derivations are evaluated in Nix with global information available. During the
build, rlibs and dependency information are propagated back up the DAG. Each
derivation's build shell combines the linking, features, target, and other
information. You can see how it's used in
mkcrate.nix
Evaluation of nix derivations doesn't require building anything. If you want to
build a specific variant of a crate in a workspace with Nix, we would have to
know this when building all of its dependencies. This means certain behavior to
switch features and optional dependencies on or off depends on what else is
being built. By default cargo2nix
will build crates as if all other crates in
the workspace might be build. This can be somewhat controlled with the
rootFeatures
argument. (see rootFeatures
in Cargo.nix).
This actually improves caching but may rarely result in a long build for an
unneeded dependency (which your workspace should put behind a non-default
top-level feature). Cargo isn't any better at this aspect of caching vs
rebuilding.
Flakes require flake.nix
and Cargo.nix
to be in version control. git add flake.nix Cargo.nix
etc. Remember to keep them up to date! Before
building the examples, you will usually need to update their pin of
cargo2nix: nix flake lock --update-input cargo2nix
or nix may complane
about paths.
Old versions of the cargo2nix.overlay
usually cannot consume newer versions
of the Cargo.nix
that an updated cargo2nix will produce. Update your
inputs with nix flake lock --update-input cargo2nix
or nix build --update-input cargo2nix
When building sys
crates, build.rs
scripts may themselves attempt to
provide native dependencies that could be missing. See the
overlay/overrides.nix
for patterns of common solutions for fixing up
specific deps.
To provide your own override, pass a modified packageOverrides
to
pkgs.rustBuilder.makePackageSet
:
rustPkgs = pkgs.rustBuilder.makePackageSet {
# ... required arguments not shown
# Use the existing all list of overrides and append your override
packageOverrides = pkgs: pkgs.rustBuilder.overrides.all ++ [
# parentheses disambiguate each makeOverride call as a single list element
(pkgs.rustBuilder.rustLib.makeOverride {
name = "fantasy-zlib-sys";
overrideAttrs = drv: {
propagatedBuildInputs = drv.propagatedBuildInputs or [ ] ++ [
pkgs.zlib.dev
];
};
})
];
};
Each derivation function in rustBuilder.makePackageSet
has it's outputs
created within mkcrate.nix, using some bash
functions in mkcrate-utils.sh. We try not to
copy more than necessary to the outputs, but this also means sometimes
skipping a necessary file. Each derivation is multiple output, using bin
and out
. The default output is bin
and should contain just what's
necessary at runtime, possibly linked to other files in the Nix store. This
output si for installation into a Nix profile or shell. The out
output
contains all of this and extra information necessary for dependents to
consume the crate, usually linking information, which will collide if you
attempt to install several such derivations.
Non-deterministic rustc or linker behavior can lead to binary-incompatible
crates. Nix cannot protect from non-determinism, only impurity. Override
your builds with preferLocalBuild = true;
allowSubstitutes = false;
for
the affected package. This has been seen more often because of
nondeterministic macros. See #184 for more information.
Nixpkgs is a rolling release, and that means breakages occur but you have
many potential successful versions to choose from. View the CI logs and
check the flake.lock
for rev information from recent successes. Update to
a specific input version with:
nix flake lock --override-input nixpkgs github:nixpgks/nixpkgs?rev=a284564b7f75ac4db73607db02076e8da9d42c9d
Toml parsing / conversion issues Error: Cannot convert data to TOML (Invalid type <class 'NoneType'>)
jq
and remarshal
are used to read & modify toml files in some
cases. Lines of the form: [key."cfg(foo = \"a\", bar = \"b\"))".path]
could produce breakage when jq
output was fed back to remarshal
. There
are workarounds in place to catch many cases. See #149 for more information
and report any newly found breakage until a total solution is in place.
Git dependencies and crates from alternative Cargo registries rely on
builtins.fetchGit
to support fetching from private Git repositories. This
means that such dependencies cannot be evaluated with restrict-eval
applied.
Also, if your Git dependency is tied to a Git branch, e.g. master
, and you
would like to force it to update on upstream changes, you should append
--option tarball-ttl 0
to your nix-build
command.
You can load a nix shell for any crate derivation in the dependency tree. This
is the same environment the cargo2nix
overlay will build them in.
To do this, first find the .drv for your dependency by using, for example, nix show-derivation | grep colorify
nix show-derivation | rg -o "/nix.*crate.*colorify.*drv"
nix/store/whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gz
# ignore environment to remove your shell's impurities
nix develop --ignore-environment nix/store/whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gz
# the environment is now as it is when nix builds the package
echo $src
nix/store/whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gz
# If you are working on a dependency and need the source (or a fresh copy) you
# can unpack the $src variable. Through nix stdenv, tar is available in pure
# shells
mkdir debug
cp $src debug
cd debug
tar -xzfv whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gz
cd <unpacked source>
You will need to override your Cargo.toml
and Cargo.lock
in this shell, so
make sure that you have them backed up if your are directly using your clone of
your project instead of unpacking fresh sources like above.
Now you just need to run the $configurePhase
and $buildPhase
steps in order.
You can find additional phases that may exist in overrides by running env | grep Phase
echo $configurePhase
# runHook preConfigure runHook configureCargo runHook postConfigure
runHook preConfigure # usually does nothing
runHook findCrate
runHook configureCargo
runHook postConfigure # usually does nothing
echo $buildPhase
# runHook overrideCargoManifest runHook setBuildEnv runHook runCargo
runHook overrideCargoManifest # This overrides your .cargo folder, e.g. for setting cross-compilers
runHook setBuildEnv # This sets up linker flags for the `rustc` invocations
runHook runCargo
If runCargo
succeeds, you will have a completed output ready for the (usually)
less interesting $installPhase
. If there's a problem, inspecting the env
or
reading the generated Cargo.lock
etc should yield clues. If you've unpacked a
fresh source and are using the --ignore-environment
switch, everything is
identical to how the overlay builds the crate, cutting out guess work.
See Contributing for potentially more information.
The design for the Nix overlay is inspired by the excellent work done by James Kay, which is described here and here. His source is available here. This work would have been impossible without these fantastic write-ups. Special thanks to James Kay!
cargo2nix
is free and open source software distributed under the terms of the
MIT License.