ghostfox/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

push sheeet
Some checks failed
Periodic Merges (6h) / master → staging-nixos (push) Failing after 12m50s
Details
Periodic Merges (6h) / master → staging-next (push) Failing after 12m54s
Details
Periodic Merges (24h) / merge-base(master,staging) → haskell-updates (push) Failing after 11m54s
Details
Periodic Merges (6h) / staging-next → staging (push) Failing after 12m13s
Details
Periodic Merges (24h) / staging-next-25.05 → staging-25.05 (push) Failing after 13m24s
Details
Periodic Merges (24h) / release-25.05 → staging-next-25.05 (push) Failing after 14m28s
Details

Browse Source
This commit is contained in:
Dark Steveneq
2025-10-09 14:15:47 +02:00
commit 646b892680
49168 changed files with 5897842 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
lib/.version Normal file
View File

@@ -0,0 +1 @@
25.11

159
lib/README.md Normal file
View File

@@ -0,0 +1,159 @@
# Nixpkgs lib
This directory contains the implementation, documentation and tests for the Nixpkgs `lib` library.
## Overview
The evaluation entry point for `lib` is [`default.nix`](default.nix).
This file evaluates to an attribute set containing two separate kinds of attributes:
- Sub-libraries:
Attribute sets grouping together similar functionality.
Each sub-library is defined in a separate file usually matching its attribute name.
Example: `lib.lists` is a sub-library containing list-related functionality such as `lib.lists.take` and `lib.lists.imap0`.
These are defined in the file [`lists.nix`](lists.nix).
- Aliases:
Attributes that point to an attribute of the same name in some sub-library.
Example: `lib.take` is an alias for `lib.lists.take`.
Most files in this directory are definitions of sub-libraries, but there are a few others:
- [`minfeatures.nix`](minfeatures.nix): A list of conditions for the used Nix version to match that are required to evaluate Nixpkgs.
- [`tests`](tests): Tests, see [Running tests](#running-tests)
- [`release.nix`](tests/release.nix): A derivation aggregating all tests
- [`misc.nix`](tests/misc.nix): Evaluation unit tests for most sub-libraries
- `*.sh`: Bash scripts that run tests for specific sub-libraries
- All other files in this directory exist to support the tests
- [`systems`](systems): The `lib.systems` sub-library, structured into a directory instead of a file due to its complexity
- [`path`](path): The `lib.path` sub-library, which includes tests as well as a document describing the design goals of `lib.path`
- All other files in this directory are sub-libraries
### Module system
The [module system](https://nixos.org/manual/nixpkgs/#module-system) spans multiple sub-libraries:
- [`modules.nix`](modules.nix): `lib.modules` for the core functions and anything not relating to option definitions
- [`options.nix`](options.nix): `lib.options` for anything relating to option definitions
- [`types.nix`](types.nix): `lib.types` for module system types
## PR Guidelines
Follow these guidelines for proposing a change to the interface of `lib`.
### Provide a Motivation
Clearly describe why the change is necessary and its use cases.
Make sure that the change benefits the user more than the added mental effort of looking it up and keeping track of its definition.
If the same can reasonably be done with the existing interface,
consider just updating the documentation with more examples and links.
This is also known as the [Fairbairn Threshold](https://wiki.haskell.org/Fairbairn_threshold).
Through this principle we avoid the human cost of duplicated functionality in an overly large library.
### Make one PR for each change
Don't have multiple changes in one PR, instead split it up into multiple ones.
This keeps the conversation focused and has a higher chance of getting merged.
### Name the interface appropriately
When introducing new names to the interface, such as new function, or new function attributes,
make sure to name it appropriately.
Names should be self-explanatory and consistent with the rest of `lib`.
If there's no obvious best name, include the alternatives you considered.
### Write documentation
Update the [reference documentation](#reference-documentation) to reflect the change.
Be generous with links to related functionality.
### Write tests
Add good test coverage for the change, including:
- Tests for edge cases, such as empty values or lists.
- Tests for tricky inputs, such as a string with string context or a path that doesn't exist.
- Test all code paths, such as `if-then-else` branches and returned attributes.
- If the tests for the sub-library are written in bash,
test messages of custom errors, such as `throw` or `abortMsg`,
At the time this is only not necessary for sub-libraries tested with [`tests/misc.nix`](./tests/misc.nix).
See [running tests](#running-tests) for more details on the test suites.
### Write tidy code
Name variables well, even if they're internal.
The code should be as self-explanatory as possible.
Be generous with code comments when appropriate.
As a baseline, follow the [Nixpkgs code conventions](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#code-conventions).
### Write efficient code
Nix generally does not have free abstractions.
Be aware that seemingly straightforward changes can cause more allocations and a decrease in performance.
That said, don't optimise prematurely, especially in new code.
## Reference documentation
Reference documentation for library functions is written above each function as a multi-line comment.
These comments are processed using [nixdoc](https://github.com/nix-community/nixdoc) and [rendered in the Nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/#chap-functions).
The nixdoc README describes the [comment format](https://github.com/nix-community/nixdoc#comment-format).
See [doc/README.md](../doc/README.md) for how to build the manual.
## Running tests
All library tests can be run by building the derivation in [`tests/release.nix`](tests/release.nix):
```bash
nix-build tests/release.nix
```
Some commands for quicker iteration over parts of the test suite are also available:
```bash
# Run all evaluation unit tests in tests/misc.nix
# if the resulting list is empty, all tests passed
nix-instantiate --eval --strict tests/misc.nix
# Run the module system tests
tests/modules.sh
# Run the lib.sources tests
tests/sources.sh
# Run the lib.filesystem tests
tests/filesystem.sh
# Run the lib.path property tests
path/tests/prop.sh
# Run the lib.fileset tests
fileset/tests.sh
```
## Commit conventions
- Make sure you read about the [commit conventions](../CONTRIBUTING.md#commit-conventions) common to Nixpkgs as a whole.
- Format the commit messages in the following way:
```
lib.(section): (init | add additional argument | refactor | etc)
(Motivation for change. Additional information.)
```
Examples:
* lib.getExe': check arguments
* lib.fileset: Add an additional argument in the design docs
Closes #264537

100
lib/ascii-table.nix Normal file
View File

@@ -0,0 +1,100 @@
{
"\t" = 9;
"\n" = 10;
"\r" = 13;
" " = 32;
"!" = 33;
"\"" = 34;
"#" = 35;
"$" = 36;
"%" = 37;
"&" = 38;
"'" = 39;
"(" = 40;
")" = 41;
"*" = 42;
"+" = 43;
"," = 44;
"-" = 45;
"." = 46;
"/" = 47;
"0" = 48;
"1" = 49;
"2" = 50;
"3" = 51;
"4" = 52;
"5" = 53;
"6" = 54;
"7" = 55;
"8" = 56;
"9" = 57;
":" = 58;
";" = 59;
"<" = 60;
"=" = 61;
">" = 62;
"?" = 63;
"@" = 64;
"A" = 65;
"B" = 66;
"C" = 67;
"D" = 68;
"E" = 69;
"F" = 70;
"G" = 71;
"H" = 72;
"I" = 73;
"J" = 74;
"K" = 75;
"L" = 76;
"M" = 77;
"N" = 78;
"O" = 79;
"P" = 80;
"Q" = 81;
"R" = 82;
"S" = 83;
"T" = 84;
"U" = 85;
"V" = 86;
"W" = 87;
"X" = 88;
"Y" = 89;
"Z" = 90;
"[" = 91;
"\\" = 92;
"]" = 93;
"^" = 94;
"_" = 95;
"`" = 96;
"a" = 97;
"b" = 98;
"c" = 99;
"d" = 100;
"e" = 101;
"f" = 102;
"g" = 103;
"h" = 104;
"i" = 105;
"j" = 106;
"k" = 107;
"l" = 108;
"m" = 109;
"n" = 110;
"o" = 111;
"p" = 112;
"q" = 113;
"r" = 114;
"s" = 115;
"t" = 116;
"u" = 117;
"v" = 118;
"w" = 119;
"x" = 120;
"y" = 121;
"z" = 122;
"{" = 123;
"|" = 124;
"}" = 125;
"~" = 126;
}

202
lib/asserts.nix Normal file
View File

@@ -0,0 +1,202 @@
{ lib }:
let
inherit (lib.strings)
concatStringsSep
;
inherit (lib.lists)
filter
;
inherit (lib.trivial)
showWarnings
;
in
rec {
/**
Throw if pred is false, else return pred.
Intended to be used to augment asserts with helpful error messages.
# Inputs
`pred`
: Predicate that needs to succeed, otherwise `msg` is thrown
`msg`
: Message to throw in case `pred` fails
# Type
```
assertMsg :: Bool -> String -> Bool
```
# Examples
:::{.example}
## `lib.asserts.assertMsg` usage example
```nix
assertMsg false "nope"
stderr> error: nope
assert assertMsg ("foo" == "bar") "foo is not bar, silly"; ""
stderr> error: foo is not bar, silly
```
:::
*/
# TODO(Profpatsch): add tests that check stderr
assertMsg = pred: msg: pred || throw msg;
/**
Specialized `assertMsg` for checking if `val` is one of the elements
of the list `xs`. Useful for checking enums.
# Inputs
`name`
: The name of the variable the user entered `val` into, for inclusion in the error message
`val`
: The value of what the user provided, to be compared against the values in `xs`
`xs`
: The list of valid values
# Type
```
assertOneOf :: String -> ComparableVal -> List ComparableVal -> Bool
```
# Examples
:::{.example}
## `lib.asserts.assertOneOf` usage example
```nix
let sslLibrary = "libressl";
in assertOneOf "sslLibrary" sslLibrary [ "openssl" "bearssl" ]
stderr> error: sslLibrary must be one of [
stderr> "openssl"
stderr> "bearssl"
stderr> ], but is: "libressl"
```
:::
*/
assertOneOf =
name: val: xs:
assertMsg (lib.elem val xs) "${name} must be one of ${lib.generators.toPretty { } xs}, but is: ${
lib.generators.toPretty { } val
}";
/**
Specialized `assertMsg` for checking if every one of `vals` is one of the elements
of the list `xs`. Useful for checking lists of supported attributes.
# Inputs
`name`
: The name of the variable the user entered `val` into, for inclusion in the error message
`vals`
: The list of values of what the user provided, to be compared against the values in `xs`
`xs`
: The list of valid values
# Type
```
assertEachOneOf :: String -> List ComparableVal -> List ComparableVal -> Bool
```
# Examples
:::{.example}
## `lib.asserts.assertEachOneOf` usage example
```nix
let sslLibraries = [ "libressl" "bearssl" ];
in assertEachOneOf "sslLibraries" sslLibraries [ "openssl" "bearssl" ]
stderr> error: each element in sslLibraries must be one of [
stderr> "openssl"
stderr> "bearssl"
stderr> ], but is: [
stderr> "libressl"
stderr> "bearssl"
stderr> ]
```
:::
*/
assertEachOneOf =
name: vals: xs:
assertMsg (lib.all (val: lib.elem val xs) vals)
"each element in ${name} must be one of ${lib.generators.toPretty { } xs}, but is: ${
lib.generators.toPretty { } vals
}";
/**
Wrap a value with logic that throws an error when assertions
fail and emits any warnings.
# Inputs
`assertions`
: A list of assertions. If any of their `assertion` attrs is `false`, their `message` attrs will be emitted in a `throw`.
`warnings`
: A list of strings to emit as warnings. This function does no filtering on this list.
`val`
: A value to return, wrapped in `warn`, if a `throw` is not necessary.
# Type
```
checkAssertWarn :: [ { assertion :: Bool; message :: String } ] -> [ String ] -> Any -> Any
```
# Examples
:::{.example}
## `lib.asserts.checkAssertWarn` usage example
```nix
checkAssertWarn
[ { assertion = false; message = "Will fail"; } ]
[ ]
null
stderr> error:
stderr> Failed assertions:
stderr> - Will fail
checkAssertWarn
[ { assertion = true; message = "Will not fail"; } ]
[ "Will warn" ]
null
stderr> evaluation warning: Will warn
null
```
:::
*/
checkAssertWarn =
assertions: warnings: val:
let
failedAssertions = map (x: x.message) (filter (x: !x.assertion) assertions);
in
if failedAssertions != [ ] then
throw "\nFailed assertions:\n${concatStringsSep "\n" (map (x: "- ${x}") failedAssertions)}"
else
showWarnings warnings val;
}

2249
lib/attrsets.nix Normal file
View File

File diff suppressed because it is too large Load Diff

148
lib/cli.nix Normal file
View File

@@ -0,0 +1,148 @@
{ lib }:
rec {
/**
Automatically convert an attribute set to command-line options.
This helps protect against malformed command lines and also to reduce
boilerplate related to command-line construction for simple use cases.
`toGNUCommandLineShell` returns an escaped shell string.
# Inputs
`options`
: How to format the arguments, see `toGNUCommandLine`
`attrs`
: The attributes to transform into arguments.
# Examples
:::{.example}
## `lib.cli.toGNUCommandLineShell` usage example
```nix
cli.toGNUCommandLineShell {} {
data = builtins.toJSON { id = 0; };
X = "PUT";
retry = 3;
retry-delay = null;
url = [ "https://example.com/foo" "https://example.com/bar" ];
silent = false;
verbose = true;
}
=> "'-X' 'PUT' '--data' '{\"id\":0}' '--retry' '3' '--url' 'https://example.com/foo' '--url' 'https://example.com/bar' '--verbose'";
```
:::
*/
toGNUCommandLineShell = options: attrs: lib.escapeShellArgs (toGNUCommandLine options attrs);
/**
Automatically convert an attribute set to a list of command-line options.
`toGNUCommandLine` returns a list of string arguments.
# Inputs
`options`
: How to format the arguments, see below.
`attrs`
: The attributes to transform into arguments.
# Options
`mkOptionName`
: How to string-format the option name;
By default one character is a short option (`-`), more than one characters a long option (`--`).
`mkBool`
: How to format a boolean value to a command list;
By default it’s a flag option (only the option name if true, left out completely if false).
`mkList`
: How to format a list value to a command list;
By default the option name is repeated for each value and `mkOption` is applied to the values themselves.
`mkOption`
: How to format any remaining value to a command list;
On the toplevel, booleans and lists are handled by `mkBool` and `mkList`, though they can still appear as values of a list.
By default, everything is printed verbatim and complex types are forbidden (lists, attrsets, functions). `null` values are omitted.
`optionValueSeparator`
: How to separate an option from its flag;
By default, there is no separator, so option `-c` and value `5` would become ["-c" "5"].
This is useful if the command requires equals, for example, `-c=5`.
# Examples
:::{.example}
## `lib.cli.toGNUCommandLine` usage example
```nix
cli.toGNUCommandLine {} {
data = builtins.toJSON { id = 0; };
X = "PUT";
retry = 3;
retry-delay = null;
url = [ "https://example.com/foo" "https://example.com/bar" ];
silent = false;
verbose = true;
}
=> [
"-X" "PUT"
"--data" "{\"id\":0}"
"--retry" "3"
"--url" "https://example.com/foo"
"--url" "https://example.com/bar"
"--verbose"
]
```
:::
*/
toGNUCommandLine =
{
mkOptionName ? k: if builtins.stringLength k == 1 then "-${k}" else "--${k}",
mkBool ? k: v: lib.optional v (mkOptionName k),
mkList ? k: v: lib.concatMap (mkOption k) v,
mkOption ?
k: v:
if v == null then
[ ]
else if optionValueSeparator == null then
[
(mkOptionName k)
(lib.generators.mkValueStringDefault { } v)
]
else
[ "${mkOptionName k}${optionValueSeparator}${lib.generators.mkValueStringDefault { } v}" ],
optionValueSeparator ? null,
}:
options:
let
render =
k: v:
if builtins.isBool v then
mkBool k v
else if builtins.isList v then
mkList k v
else
mkOption k v;
in
builtins.concatLists (lib.mapAttrsToList render options);
}

867
lib/customisation.nix Normal file
View File

@@ -0,0 +1,867 @@
{ lib }:
let
inherit (builtins)
intersectAttrs
unsafeGetAttrPos
;
inherit (lib)
functionArgs
isFunction
mirrorFunctionArgs
isAttrs
setFunctionArgs
optionalAttrs
attrNames
filter
elemAt
concatStringsSep
sortOn
take
length
filterAttrs
optionalString
flip
pathIsDirectory
head
pipe
isDerivation
listToAttrs
mapAttrs
seq
flatten
deepSeq
extends
toFunction
id
;
inherit (lib.strings) levenshtein levenshteinAtMost;
in
rec {
/**
`overrideDerivation drv f` takes a derivation (i.e., the result
of a call to the builtin function `derivation`) and returns a new
derivation in which the attributes of the original are overridden
according to the function `f`. The function `f` is called with
the original derivation attributes.
`overrideDerivation` allows certain "ad-hoc" customisation
scenarios (e.g. in ~/.config/nixpkgs/config.nix). For instance,
if you want to "patch" the derivation returned by a package
function in Nixpkgs to build another version than what the
function itself provides.
For another application, see build-support/vm, where this
function is used to build arbitrary derivations inside a QEMU
virtual machine.
Note that in order to preserve evaluation errors, the new derivation's
outPath depends on the old one's, which means that this function cannot
be used in circular situations when the old derivation also depends on the
new one.
You should in general prefer `drv.overrideAttrs` over this function;
see the nixpkgs manual for more information on overriding.
# Inputs
`drv`
: 1\. Function argument
`f`
: 2\. Function argument
# Type
```
overrideDerivation :: Derivation -> ( Derivation -> AttrSet ) -> Derivation
```
# Examples
:::{.example}
## `lib.customisation.overrideDerivation` usage example
```nix
mySed = overrideDerivation pkgs.gnused (oldAttrs: {
name = "sed-4.2.2-pre";
src = fetchurl {
url = ftp://alpha.gnu.org/gnu/sed/sed-4.2.2-pre.tar.bz2;
hash = "sha256-MxBJRcM2rYzQYwJ5XKxhXTQByvSg5jZc5cSHEZoB2IY=";
};
patches = [];
});
```
:::
*/
overrideDerivation =
drv: f:
let
newDrv = derivation (drv.drvAttrs // (f drv));
in
flip (extendDerivation (seq drv.drvPath true)) newDrv (
{
meta = drv.meta or { };
passthru = if drv ? passthru then drv.passthru else { };
}
// (drv.passthru or { })
// optionalAttrs (drv ? __spliced) {
__spliced = { } // (mapAttrs (_: sDrv: overrideDerivation sDrv f) drv.__spliced);
}
);
/**
`makeOverridable` takes a function from attribute set to attribute set and
injects `override` attribute which can be used to override arguments of
the function.
Please refer to documentation on [`<pkg>.overrideDerivation`](#sec-pkg-overrideDerivation) to learn about `overrideDerivation` and caveats
related to its use.
# Inputs
`f`
: 1\. Function argument
# Type
```
makeOverridable :: (AttrSet -> a) -> AttrSet -> a
```
# Examples
:::{.example}
## `lib.customisation.makeOverridable` usage example
```nix
nix-repl> x = {a, b}: { result = a + b; }
nix-repl> y = lib.makeOverridable x { a = 1; b = 2; }
nix-repl> y
{ override = «lambda»; overrideDerivation = «lambda»; result = 3; }
nix-repl> y.override { a = 10; }
{ override = «lambda»; overrideDerivation = «lambda»; result = 12; }
```
:::
*/
makeOverridable =
f:
let
# Creates a functor with the same arguments as f
mirrorArgs = mirrorFunctionArgs f;
in
mirrorArgs (
origArgs:
let
result = f origArgs;
# Changes the original arguments with (potentially a function that returns) a set of new attributes
overrideWith = newArgs: origArgs // (if isFunction newArgs then newArgs origArgs else newArgs);
# Re-call the function but with different arguments
overrideArgs = mirrorArgs (
/**
Change the arguments with which a certain function is called.
In some cases, you may find a list of possible attributes to pass in this function's `__functionArgs` attribute, but it will not be complete for an original function like `args@{foo, ...}: ...`, which accepts arbitrary attributes.
This function was provided by `lib.makeOverridable`.
*/
newArgs: makeOverridable f (overrideWith newArgs)
);
# Change the result of the function call by applying g to it
overrideResult = g: makeOverridable (mirrorArgs (args: g (f args))) origArgs;
in
if isAttrs result then
result
// {
override = overrideArgs;
overrideDerivation = fdrv: overrideResult (x: overrideDerivation x fdrv);
${if result ? overrideAttrs then "overrideAttrs" else null} =
/**
Override the attributes that were passed to `mkDerivation` in order to generate this derivation.
This function is provided by `lib.makeOverridable`, and indirectly by `callPackage` among others, in order to make the combination of `override` and `overrideAttrs` work.
Specifically, it re-adds the `override` attribute to the result of `overrideAttrs`.
The real implementation of `overrideAttrs` is provided by `stdenv.mkDerivation`.
*/
# NOTE: part of the above documentation had to be duplicated in `mkDerivation`'s `overrideAttrs`.
# design/tech debt issue: https://github.com/NixOS/nixpkgs/issues/273815
fdrv: overrideResult (x: x.overrideAttrs fdrv);
}
else if isFunction result then
# Transform the result into a functor while propagating its arguments
setFunctionArgs result (functionArgs result)
// {
override = overrideArgs;
}
else
result
);
/**
Call the package function in the file `fn` with the required
arguments automatically. The function is called with the
arguments `args`, but any missing arguments are obtained from
`autoArgs`. This function is intended to be partially
parameterised, e.g.,
```nix
callPackage = callPackageWith pkgs;
pkgs = {
libfoo = callPackage ./foo.nix { };
libbar = callPackage ./bar.nix { };
};
```
If the `libbar` function expects an argument named `libfoo`, it is
automatically passed as an argument. Overrides or missing
arguments can be supplied in `args`, e.g.
```nix
libbar = callPackage ./bar.nix {
libfoo = null;
enableX11 = true;
};
```
<!-- TODO: Apply "Example:" tag to the examples above -->
# Inputs
`autoArgs`
: 1\. Function argument
`fn`
: 2\. Function argument
`args`
: 3\. Function argument
# Type
```
callPackageWith :: AttrSet -> ((AttrSet -> a) | Path) -> AttrSet -> a
```
*/
callPackageWith =
autoArgs: fn: args:
let
f = if isFunction fn then fn else import fn;
fargs = functionArgs f;
# All arguments that will be passed to the function
# This includes automatic ones and ones passed explicitly
allArgs = intersectAttrs fargs autoArgs // args;
# a list of argument names that the function requires, but
# wouldn't be passed to it
missingArgs =
# Filter out arguments that have a default value
(
filterAttrs (name: value: !value)
# Filter out arguments that would be passed
(removeAttrs fargs (attrNames allArgs))
);
# Get a list of suggested argument names for a given missing one
getSuggestions =
arg:
pipe (autoArgs // args) [
attrNames
# Only use ones that are at most 2 edits away. While mork would work,
# levenshteinAtMost is only fast for 2 or less.
(filter (levenshteinAtMost 2 arg))
# Put strings with shorter distance first
(sortOn (levenshtein arg))
# Only take the first couple results
(take 3)
# Quote all entries
(map (x: "\"" + x + "\""))
];
prettySuggestions =
suggestions:
if suggestions == [ ] then
""
else if length suggestions == 1 then
", did you mean ${elemAt suggestions 0}?"
else
", did you mean ${concatStringsSep ", " (lib.init suggestions)} or ${lib.last suggestions}?";
errorForArg =
arg:
let
loc = unsafeGetAttrPos arg fargs;
in
"Function called without required argument \"${arg}\" at "
+ "${loc.file}:${toString loc.line}${prettySuggestions (getSuggestions arg)}";
# Only show the error for the first missing argument
error = errorForArg (head (attrNames missingArgs));
in
if missingArgs == { } then
makeOverridable f allArgs
# This needs to be an abort so it can't be caught with `builtins.tryEval`,
# which is used by nix-env and ofborg to filter out packages that don't evaluate.
# This way we're forced to fix such errors in Nixpkgs,
# which is especially relevant with allowAliases = false
else
abort "lib.customisation.callPackageWith: ${error}";
/**
Like callPackage, but for a function that returns an attribute
set of derivations. The override function is added to the
individual attributes.
# Inputs
`autoArgs`
: 1\. Function argument
`fn`
: 2\. Function argument
`args`
: 3\. Function argument
# Type
```
callPackagesWith :: AttrSet -> ((AttrSet -> AttrSet) | Path) -> AttrSet -> AttrSet
```
*/
callPackagesWith =
autoArgs: fn: args:
let
f = if isFunction fn then fn else import fn;
auto = intersectAttrs (functionArgs f) autoArgs;
mirrorArgs = mirrorFunctionArgs f;
origArgs = auto // args;
pkgs = f origArgs;
mkAttrOverridable = name: _: makeOverridable (mirrorArgs (newArgs: (f newArgs).${name})) origArgs;
in
if isDerivation pkgs then
throw (
"function `callPackages` was called on a *single* derivation "
+ ''"${pkgs.name or "<unknown-name>"}";''
+ " did you mean to use `callPackage` instead?"
)
else
mapAttrs mkAttrOverridable pkgs;
/**
Add attributes to each output of a derivation without changing
the derivation itself and check a given condition when evaluating.
# Inputs
`condition`
: 1\. Function argument
`passthru`
: 2\. Function argument
`drv`
: 3\. Function argument
# Type
```
extendDerivation :: Bool -> Any -> Derivation -> Derivation
```
*/
extendDerivation =
condition: passthru: drv:
let
outputs = drv.outputs or [ "out" ];
commonAttrs =
drv // (listToAttrs outputsList) // { all = map (x: x.value) outputsList; } // passthru;
outputToAttrListElement = outputName: {
name = outputName;
value =
commonAttrs
// {
inherit (drv.${outputName}) type outputName;
outputSpecified = true;
drvPath =
assert condition;
drv.${outputName}.drvPath;
outPath =
assert condition;
drv.${outputName}.outPath;
}
//
# TODO: give the derivation control over the outputs.
# `overrideAttrs` may not be the only attribute that needs
# updating when switching outputs.
optionalAttrs (passthru ? overrideAttrs) {
# TODO: also add overrideAttrs when overrideAttrs is not custom, e.g. when not splicing.
overrideAttrs = f: (passthru.overrideAttrs f).${outputName};
};
};
outputsList = map outputToAttrListElement outputs;
in
commonAttrs
// {
drvPath =
assert condition;
drv.drvPath;
outPath =
assert condition;
drv.outPath;
};
/**
Strip a derivation of all non-essential attributes, returning
only those needed by hydra-eval-jobs. Also strictly evaluate the
result to ensure that there are no thunks kept alive to prevent
garbage collection.
# Inputs
`drv`
: 1\. Function argument
# Type
```
hydraJob :: (Derivation | Null) -> (Derivation | Null)
```
*/
hydraJob =
drv:
let
outputs = drv.outputs or [ "out" ];
commonAttrs = {
inherit (drv) name system meta;
inherit outputs;
}
// optionalAttrs (drv._hydraAggregate or false) {
_hydraAggregate = true;
constituents = map hydraJob (flatten drv.constituents);
}
// (listToAttrs outputsList);
makeOutput =
outputName:
let
output = drv.${outputName};
in
{
name = outputName;
value = commonAttrs // {
outPath = output.outPath;
drvPath = output.drvPath;
type = "derivation";
inherit outputName;
};
};
outputsList = map makeOutput outputs;
drv' = (head outputsList).value;
in
if drv == null then null else deepSeq drv' drv';
/**
Make an attribute set (a "scope") from functions that take arguments from that same attribute set.
See [](#ex-makeScope) for how to use it.
# Inputs
1. `newScope` (`AttrSet -> ((AttrSet -> a) | Path) -> AttrSet -> a`)
A function that takes an attribute set `attrs` and returns what ends up as `callPackage` in the output.
Typical values are `callPackageWith` or the output attribute `newScope`.
2. `f` (`AttrSet -> AttrSet`)
A function that takes an attribute set as returned by `makeScope newScope f` (a "scope") and returns any attribute set.
This function is used to compute the fixpoint of the resulting scope using `callPackage`.
Its argument is the lazily evaluated reference to the value of that fixpoint, and is typically called `self` or `final`.
See [](#ex-makeScope) for how to use it.
See [](#sec-functions-library-fixedPoints) for details on fixpoint computation.
# Output
`makeScope` returns an attribute set of a form called `scope`, which also contains the final attributes produced by `f`:
```
scope :: {
callPackage :: ((AttrSet -> a) | Path) -> AttrSet -> a
newScope = AttrSet -> scope
overrideScope = (scope -> scope -> AttrSet) -> scope
packages :: AttrSet -> AttrSet
}
```
- `callPackage` (`((AttrSet -> a) | Path) -> AttrSet -> a`)
A function that
1. Takes a function `p`, or a path to a Nix file that contains a function `p`, which takes an attribute set and returns value of arbitrary type `a`,
2. Takes an attribute set `args` with explicit attributes to pass to `p`,
3. Calls `f` with attributes from the original attribute set `attrs` passed to `newScope` updated with `args`, i.e. `attrs // args`, if they match the attributes in the argument of `p`.
All such functions `p` will be called with the same value for `attrs`.
See [](#ex-makeScope-callPackage) for how to use it.
- `newScope` (`AttrSet -> scope`)
Takes an attribute set `attrs` and returns a scope that extends the original scope.
- `overrideScope` (`(scope -> scope -> AttrSet) -> scope`)
Takes a function `g` of the form `final: prev: { # attributes }` to act as an overlay on `f`, and returns a new scope with values determined by `extends g f`.
See [](https://nixos.org/manual/nixpkgs/unstable/#function-library-lib.fixedPoints.extends) for details.
This allows subsequent modification of the final attribute set in a consistent way, i.e. all functions `p` invoked with `callPackage` will be called with the modified values.
- `packages` (`AttrSet -> AttrSet`)
The value of the argument `f` to `makeScope`.
- final attributes
The final values returned by `f`.
# Examples
:::{#ex-makeScope .example}
# Create an interdependent package set on top of `pkgs`
The functions in `foo.nix` and `bar.nix` can depend on each other, in the sense that `foo.nix` can contain a function that expects `bar` as an attribute in its argument.
```nix
let
pkgs = import <nixpkgs> { };
in
pkgs.lib.makeScope pkgs.newScope (self: {
foo = self.callPackage ./foo.nix { };
bar = self.callPackage ./bar.nix { };
})
```
evaluates to
```nix
{
callPackage = «lambda»;
newScope = «lambda»;
overrideScope = «lambda»;
packages = «lambda»;
foo = «derivation»;
bar = «derivation»;
}
```
:::
:::{#ex-makeScope-callPackage .example}
# Using `callPackage` from a scope
```nix
let
pkgs = import <nixpkgs> { };
inherit (pkgs) lib;
scope = lib.makeScope lib.callPackageWith (self: { a = 1; b = 2; });
three = scope.callPackage ({ a, b }: a + b) { };
four = scope.callPackage ({ a, b }: a + b) { a = 2; };
in
[ three four ]
```
evaluates to
```nix
[ 3 4 ]
```
:::
# Type
```
makeScope :: (AttrSet -> ((AttrSet -> a) | Path) -> AttrSet -> a) -> (AttrSet -> AttrSet) -> scope
```
*/
makeScope =
newScope: f:
let
self = f self // {
newScope = scope: newScope (self // scope);
callPackage = self.newScope { };
overrideScope = g: makeScope newScope (extends g f);
packages = f;
};
in
self;
/**
backward compatibility with old uncurried form; deprecated
# Inputs
`splicePackages`
: 1\. Function argument
`newScope`
: 2\. Function argument
`otherSplices`
: 3\. Function argument
`keep`
: 4\. Function argument
`extra`
: 5\. Function argument
`f`
: 6\. Function argument
*/
makeScopeWithSplicing =
splicePackages: newScope: otherSplices: keep: extra: f:
makeScopeWithSplicing' { inherit splicePackages newScope; } {
inherit
otherSplices
keep
extra
f
;
};
/**
Like makeScope, but aims to support cross compilation. It's still ugly, but
hopefully it helps a little bit.
# Type
```
makeScopeWithSplicing' ::
{ splicePackages :: Splice -> AttrSet
, newScope :: AttrSet -> ((AttrSet -> a) | Path) -> AttrSet -> a
}
-> { otherSplices :: Splice, keep :: AttrSet -> AttrSet, extra :: AttrSet -> AttrSet }
-> AttrSet
Splice ::
{ pkgsBuildBuild :: AttrSet
, pkgsBuildHost :: AttrSet
, pkgsBuildTarget :: AttrSet
, pkgsHostHost :: AttrSet
, pkgsHostTarget :: AttrSet
, pkgsTargetTarget :: AttrSet
}
```
*/
makeScopeWithSplicing' =
{
splicePackages,
newScope,
}:
{
otherSplices,
# Attrs from `self` which won't be spliced.
# Avoid using keep, it's only used for a python hook workaround, added in PR #104201.
# ex: `keep = (self: { inherit (self) aAttr; })`
keep ? (_self: { }),
# Additional attrs to add to the sets `callPackage`.
# When the package is from a subset (but not a subset within a package IS #211340)
# within `spliced0` it will be spliced.
# When using an package outside the set but it's available from `pkgs`, use the package from `pkgs.__splicedPackages`.
# If the package is not available within the set or in `pkgs`, such as a package in a let binding, it will not be spliced
# ex:
# ```
# nix-repl> darwin.apple_sdk.frameworks.CoreFoundation
# «derivation ...CoreFoundation-11.0.0.drv»
# nix-repl> darwin.CoreFoundation
# error: attribute 'CoreFoundation' missing
# nix-repl> darwin.callPackage ({ CoreFoundation }: CoreFoundation) { }
# «derivation ...CoreFoundation-11.0.0.drv»
# ```
extra ? (_spliced0: { }),
f,
}:
let
spliced0 = splicePackages {
pkgsBuildBuild = otherSplices.selfBuildBuild;
pkgsBuildHost = otherSplices.selfBuildHost;
pkgsBuildTarget = otherSplices.selfBuildTarget;
pkgsHostHost = otherSplices.selfHostHost;
pkgsHostTarget = self; # Not `otherSplices.selfHostTarget`;
pkgsTargetTarget = otherSplices.selfTargetTarget;
};
spliced = extra spliced0 // spliced0 // keep self;
self = f self // {
newScope = scope: newScope (spliced // scope);
callPackage = newScope spliced; # == self.newScope {};
# N.B. the other stages of the package set spliced in are *not*
# overridden.
overrideScope =
g:
(makeScopeWithSplicing' { inherit splicePackages newScope; } {
inherit otherSplices keep extra;
f = extends g f;
});
packages = f;
};
in
self;
/**
Define a `mkDerivation`-like function based on another `mkDerivation`-like function.
[`stdenv.mkDerivation`](#part-stdenv) gives access to
its final set of derivation attributes when it is passed a function,
or when it is passed an overlay-style function in `overrideAttrs`.
Instead of composing new `stdenv.mkDerivation`-like build helpers
using normal function composition,
`extendMkDerivation` makes sure that the returned build helper
supports such first class recursion like `mkDerivation` does.
`extendMkDerivation` takes an extra attribute set to configure its behaviour.
One can optionally specify
`transformDrv` to specify a function to apply to the result derivation,
or `inheritFunctionArgs` to decide whether to inherit the `__functionArgs`
from the base build helper.
# Inputs
`extendMkDerivation`-specific configurations
: `constructDrv`: Base build helper, the `mkDerivation`-like build helper to extend.
: `excludeDrvArgNames`: Argument names not to pass from the input fixed-point arguments to `constructDrv`. Note: It doesn't apply to the updating arguments returned by `extendDrvArgs`.
: `extendDrvArgs` : An extension (overlay) of the argument set, like the one taken by [overrideAttrs](#sec-pkg-overrideAttrs) but applied before passing to `constructDrv`.
: `inheritFunctionArgs`: Whether to inherit `__functionArgs` from the base build helper (default to `true`).
: `transformDrv`: Function to apply to the result derivation (default to `lib.id`).
# Type
```
extendMkDerivation ::
{
constructDrv :: ((FixedPointArgs | AttrSet) -> a)
excludeDrvArgNames :: [ String ],
extendDrvArgs :: (AttrSet -> AttrSet -> AttrSet)
inheritFunctionArgs :: Bool,
transformDrv :: a -> a,
}
-> (FixedPointArgs | AttrSet) -> a
FixedPointArgs = AttrSet -> AttrSet
a = Derivation when defining a build helper
```
# Examples
:::{.example}
## `lib.customisation.extendMkDerivation` usage example
```nix-repl
mkLocalDerivation = lib.extendMkDerivation {
constructDrv = pkgs.stdenv.mkDerivation;
excludeDrvArgNames = [ "specialArg" ];
extendDrvArgs =
finalAttrs: args@{ preferLocalBuild ? true, allowSubstitute ? false, specialArg ? (_: false), ... }:
{ inherit preferLocalBuild allowSubstitute; passthru = { inherit specialArg; } // args.passthru or { }; };
}
mkLocalDerivation.__functionArgs
=> { allowSubstitute = true; preferLocalBuild = true; specialArg = true; }
mkLocalDerivation { inherit (pkgs.hello) pname version src; specialArg = _: false; }
=> «derivation /nix/store/xirl67m60ahg6jmzicx43a81g635g8z8-hello-2.12.1.drv»
mkLocalDerivation (finalAttrs: { inherit (pkgs.hello) pname version src; specialArg = _: false; })
=> «derivation /nix/store/xirl67m60ahg6jmzicx43a81g635g8z8-hello-2.12.1.drv»
(mkLocalDerivation (finalAttrs: { inherit (pkgs.hello) pname version src; passthru = { foo = "a"; bar = "${finalAttrs.passthru.foo}b"; }; })).bar
=> "ab"
```
:::
:::{.note}
If `transformDrv` is specified,
it should take care of existing attributes that perform overriding
(e.g., [`overrideAttrs`](#sec-pkg-overrideAttrs))
to ensure that the overriding functionality of the result derivation
work as expected.
Modifications that breaks the overriding include
direct [attribute set update](https://nixos.org/manual/nix/stable/language/operators#update)
and [`lib.extendDerivation`](#function-library-lib.customisation.extendDerivation).
:::
*/
extendMkDerivation =
let
extendsWithExclusion =
excludedNames: g: f: final:
let
previous = f final;
in
removeAttrs previous excludedNames // g final previous;
in
{
constructDrv,
excludeDrvArgNames ? [ ],
extendDrvArgs,
inheritFunctionArgs ? true,
transformDrv ? id,
}:
setFunctionArgs
# Adds the fixed-point style support
(
fpargs:
transformDrv (
constructDrv (extendsWithExclusion excludeDrvArgNames extendDrvArgs (toFunction fpargs))
)
)
# Add __functionArgs
(
# Inherit the __functionArgs from the base build helper
optionalAttrs inheritFunctionArgs (removeAttrs (functionArgs constructDrv) excludeDrvArgNames)
# Recover the __functionArgs from the derived build helper
// functionArgs (extendDrvArgs { })
)
// {
inherit
# Expose to the result build helper.
constructDrv
excludeDrvArgNames
extendDrvArgs
transformDrv
;
};
}

480
lib/debug.nix Normal file
View File

@@ -0,0 +1,480 @@
/**
Collection of functions useful for debugging
broken nix expressions.
* `trace`-like functions take two values, print
the first to stderr and return the second.
* `traceVal`-like functions take one argument
which both printed and returned.
* `traceSeq`-like functions fully evaluate their
traced value before printing (not just to “weak
head normal form” like trace does by default).
* Functions that end in `-Fn` take an additional
function as their first argument, which is applied
to the traced value before it is printed.
*/
{ lib }:
let
inherit (lib)
isList
isAttrs
substring
attrValues
concatLists
const
elem
generators
id
mapAttrs
trace
;
in
rec {
# -- TRACING --
/**
Conditionally trace the supplied message, based on a predicate.
# Inputs
`pred`
: Predicate to check
`msg`
: Message that should be traced
`x`
: Value to return
# Type
```
traceIf :: bool -> string -> a -> a
```
# Examples
:::{.example}
## `lib.debug.traceIf` usage example
```nix
traceIf true "hello" 3
trace: hello
=> 3
```
:::
*/
traceIf =
pred: msg: x:
if pred then trace msg x else x;
/**
Trace the supplied value after applying a function to it, and
return the original value.
# Inputs
`f`
: Function to apply
`x`
: Value to trace and return
# Type
```
traceValFn :: (a -> b) -> a -> a
```
# Examples
:::{.example}
## `lib.debug.traceValFn` usage example
```nix
traceValFn (v: "mystring ${v}") "foo"
trace: mystring foo
=> "foo"
```
:::
*/
traceValFn = f: x: trace (f x) x;
/**
Trace the supplied value and return it.
# Inputs
`x`
: Value to trace and return
# Type
```
traceVal :: a -> a
```
# Examples
:::{.example}
## `lib.debug.traceVal` usage example
```nix
traceVal 42
# trace: 42
=> 42
```
:::
*/
traceVal = traceValFn id;
/**
`builtins.trace`, but the value is `builtins.deepSeq`ed first.
# Inputs
`x`
: The value to trace
`y`
: The value to return
# Type
```
traceSeq :: a -> b -> b
```
# Examples
:::{.example}
## `lib.debug.traceSeq` usage example
```nix
trace { a.b.c = 3; } null
trace: { a = <CODE>; }
=> null
traceSeq { a.b.c = 3; } null
trace: { a = { b = { c = 3; }; }; }
=> null
```
:::
*/
traceSeq = x: y: trace (builtins.deepSeq x x) y;
/**
Like `traceSeq`, but only evaluate down to depth n.
This is very useful because lots of `traceSeq` usages
lead to an infinite recursion.
# Inputs
`depth`
: 1\. Function argument
`x`
: 2\. Function argument
`y`
: 3\. Function argument
# Type
```
traceSeqN :: Int -> a -> b -> b
```
# Examples
:::{.example}
## `lib.debug.traceSeqN` usage example
```nix
traceSeqN 2 { a.b.c = 3; } null
trace: { a = { b = {…}; }; }
=> null
```
:::
*/
traceSeqN =
depth: x: y:
let
snip =
v:
if isList v then
noQuotes "[…]" v
else if isAttrs v then
noQuotes "{…}" v
else
v;
noQuotes = str: v: {
__pretty = const str;
val = v;
};
modify =
n: fn: v:
if (n == 0) then
fn v
else if isList v then
map (modify (n - 1) fn) v
else if isAttrs v then
mapAttrs (const (modify (n - 1) fn)) v
else
v;
in
trace (generators.toPretty { allowPrettyValues = true; } (modify depth snip x)) y;
/**
A combination of `traceVal` and `traceSeq` that applies a
provided function to the value to be traced after `deepSeq`ing
it.
# Inputs
`f`
: Function to apply
`v`
: Value to trace
*/
traceValSeqFn = f: v: traceValFn f (builtins.deepSeq v v);
/**
A combination of `traceVal` and `traceSeq`.
# Inputs
`v`
: Value to trace
*/
traceValSeq = traceValSeqFn id;
/**
A combination of `traceVal` and `traceSeqN` that applies a
provided function to the value to be traced.
# Inputs
`f`
: Function to apply
`depth`
: 2\. Function argument
`v`
: Value to trace
*/
traceValSeqNFn =
f: depth: v:
traceSeqN depth (f v) v;
/**
A combination of `traceVal` and `traceSeqN`.
# Inputs
`depth`
: 1\. Function argument
`v`
: Value to trace
*/
traceValSeqN = traceValSeqNFn id;
/**
Trace the input and output of a function `f` named `name`,
both down to `depth`.
This is useful for adding around a function call,
to see the before/after of values as they are transformed.
# Inputs
`depth`
: 1\. Function argument
`name`
: 2\. Function argument
`f`
: 3\. Function argument
`v`
: 4\. Function argument
# Examples
:::{.example}
## `lib.debug.traceFnSeqN` usage example
```nix
traceFnSeqN 2 "id" (x: x) { a.b.c = 3; }
trace: { fn = "id"; from = { a.b = {…}; }; to = { a.b = {…}; }; }
=> { a.b.c = 3; }
```
:::
*/
traceFnSeqN =
depth: name: f: v:
let
res = f v;
in
lib.traceSeqN (depth + 1) {
fn = name;
from = v;
to = res;
} res;
# -- TESTING --
/**
Evaluates a set of tests.
A test is an attribute set `{expr, expected}`,
denoting an expression and its expected result.
The result is a `list` of __failed tests__, each represented as
`{name, expected, result}`,
- expected
- What was passed as `expected`
- result
- The actual `result` of the test
Used for regression testing of the functions in lib; see
tests.nix for more examples.
Important: Only attributes that start with `test` are executed.
- If you want to run only a subset of the tests add the attribute `tests = ["testName"];`
# Inputs
`tests`
: Tests to run
# Type
```
runTests :: {
tests = [ String ];
${testName} :: {
expr :: a;
expected :: a;
};
}
->
[
{
name :: String;
expected :: a;
result :: a;
}
]
```
# Examples
:::{.example}
## `lib.debug.runTests` usage example
```nix
runTests {
testAndOk = {
expr = lib.and true false;
expected = false;
};
testAndFail = {
expr = lib.and true false;
expected = true;
};
}
->
[
{
name = "testAndFail";
expected = true;
result = false;
}
]
```
:::
*/
runTests =
tests:
concatLists (
attrValues (
mapAttrs (
name: test:
let
testsToRun = if tests ? tests then tests.tests else [ ];
in
if
(substring 0 4 name == "test" || elem name testsToRun)
&& ((testsToRun == [ ]) || elem name tests.tests)
&& (test.expr != test.expected)
then
[
{
inherit name;
expected = test.expected;
result = test.expr;
}
]
else
[ ]
) tests
)
);
/**
Create a test assuming that list elements are `true`.
# Inputs
`expr`
: 1\. Function argument
# Examples
:::{.example}
## `lib.debug.testAllTrue` usage example
```nix
{ testX = allTrue [ true ]; }
```
:::
*/
testAllTrue = expr: {
inherit expr;
expected = map (x: true) expr;
};
}

581
lib/default.nix Normal file
View File

@@ -0,0 +1,581 @@
/*
Library of low-level helper functions for nix expressions.
Please implement (mostly) exhaustive unit tests
for new functions in `./tests.nix`.
*/
let
# A copy of `lib.makeExtensible'` in order to document `extend`.
# It has been leading to some trouble, so we have to document it specially.
makeExtensible' =
rattrs:
let
self = rattrs self // {
/**
Patch the Nixpkgs library
A function that applies patches onto the nixpkgs library.
Usage is discouraged for most scenarios.
:::{.note}
The name `extends` is a bit misleading, as it doesn't actually extend the library, but rather patches it.
It is merely a consequence of being implemented by `makeExtensible`.
:::
# Inputs
- An "extension function" `f` that returns attributes that will be updated in the returned Nixpkgs library.
# Output
A patched Nixpkgs library.
:::{.warning}
This functionality is intended as an escape hatch for when the provided version of the Nixpkgs library has a flaw.
If you were to use it to add new functionality, you will run into compatibility and interoperability issues.
:::
*/
extend = f: lib.makeExtensible (lib.extends f rattrs);
};
in
self;
lib = makeExtensible' (
self:
let
callLibs = file: import file { lib = self; };
in
{
# often used, or depending on very little
trivial = callLibs ./trivial.nix;
fixedPoints = callLibs ./fixed-points.nix;
# datatypes
attrsets = callLibs ./attrsets.nix;
lists = callLibs ./lists.nix;
strings = callLibs ./strings.nix;
stringsWithDeps = callLibs ./strings-with-deps.nix;
# packaging
customisation = callLibs ./customisation.nix;
derivations = callLibs ./derivations.nix;
maintainers = import ../maintainers/maintainer-list.nix;
teams = callLibs ../maintainers/team-list.nix;
meta = callLibs ./meta.nix;
versions = callLibs ./versions.nix;
# module system
modules = callLibs ./modules.nix;
options = callLibs ./options.nix;
types = callLibs ./types.nix;
# constants
licenses = callLibs ./licenses.nix;
sourceTypes = callLibs ./source-types.nix;
systems = callLibs ./systems;
# serialization
cli = callLibs ./cli.nix;
gvariant = callLibs ./gvariant.nix;
generators = callLibs ./generators.nix;
# misc
asserts = callLibs ./asserts.nix;
debug = callLibs ./debug.nix;
misc = callLibs ./deprecated/misc.nix;
# domain-specific
fetchers = callLibs ./fetchers.nix;
# Eval-time filesystem handling
path = callLibs ./path;
filesystem = callLibs ./filesystem.nix;
fileset = callLibs ./fileset;
sources = callLibs ./sources.nix;
# back-compat aliases
platforms = self.systems.doubles;
# linux kernel configuration
kernel = callLibs ./kernel.nix;
# network
network = callLibs ./network;
# TODO: For consistency, all builtins should also be available from a sub-library;
# these are the only ones that are currently not
inherit (builtins)
addErrorContext
isPath
trace
typeOf
unsafeGetAttrPos
;
inherit (self.trivial)
id
const
pipe
concat
or
and
xor
bitAnd
bitOr
bitXor
bitNot
boolToString
boolToYesNo
mergeAttrs
flip
defaultTo
mapNullable
inNixShell
isFloat
min
max
importJSON
importTOML
warn
warnIf
warnIfNot
throwIf
throwIfNot
checkListOfEnum
info
showWarnings
nixpkgsVersion
version
isInOldestRelease
oldestSupportedReleaseIsAtLeast
mod
compare
splitByAndCompare
seq
deepSeq
lessThan
add
sub
functionArgs
setFunctionArgs
isFunction
toFunction
mirrorFunctionArgs
fromHexString
toHexString
toBaseDigits
inPureEvalMode
isBool
isInt
pathExists
genericClosure
readFile
;
inherit (self.fixedPoints)
fix
fix'
converge
extends
composeExtensions
composeManyExtensions
makeExtensible
makeExtensibleWithCustomName
toExtension
;
inherit (self.attrsets)
attrByPath
hasAttrByPath
setAttrByPath
getAttrFromPath
attrVals
attrNames
attrValues
getAttrs
catAttrs
filterAttrs
filterAttrsRecursive
foldlAttrs
foldAttrs
collect
nameValuePair
mapAttrs
mapAttrs'
mapAttrsToList
attrsToList
concatMapAttrs
mapAttrsRecursive
mapAttrsRecursiveCond
mapAttrsToListRecursive
mapAttrsToListRecursiveCond
genAttrs
genAttrs'
isDerivation
toDerivation
optionalAttrs
zipAttrsWithNames
zipAttrsWith
zipAttrs
recursiveUpdateUntil
recursiveUpdate
matchAttrs
mergeAttrsList
overrideExisting
showAttrPath
getOutput
getFirstOutput
getBin
getLib
getStatic
getDev
getInclude
getMan
chooseDevOutputs
zipWithNames
zip
recurseIntoAttrs
dontRecurseIntoAttrs
cartesianProduct
cartesianProductOfSets
mapCartesianProduct
updateManyAttrsByPath
listToAttrs
hasAttr
getAttr
isAttrs
intersectAttrs
removeAttrs
;
inherit (self.lists)
singleton
forEach
map
foldr
fold
foldl
foldl'
imap0
imap1
filter
ifilter0
concatMap
flatten
remove
findSingle
findFirst
any
all
count
optional
optionals
toList
range
replicate
partition
zipListsWith
zipLists
reverseList
listDfs
toposort
sort
sortOn
naturalSort
compareLists
take
takeEnd
drop
dropEnd
sublist
last
init
crossLists
unique
uniqueStrings
allUnique
intersectLists
subtractLists
mutuallyExclusive
groupBy
groupBy'
concatLists
genList
length
head
tail
elem
elemAt
isList
;
inherit (self.strings)
concatStrings
concatMapStrings
concatImapStrings
stringLength
substring
isString
replaceString
replaceStrings
intersperse
concatStringsSep
concatMapStringsSep
concatMapAttrsStringSep
concatImapStringsSep
concatLines
makeSearchPath
makeSearchPathOutput
makeLibraryPath
makeIncludePath
makeBinPath
optionalString
hasInfix
hasPrefix
hasSuffix
join
stringToCharacters
stringAsChars
escape
escapeShellArg
escapeShellArgs
isStorePath
isStringLike
isValidPosixName
toShellVar
toShellVars
trim
trimWith
escapeRegex
escapeURL
escapeXML
replaceChars
lowerChars
upperChars
toLower
toUpper
toCamelCase
toSentenceCase
addContextFrom
splitString
splitStringBy
removePrefix
removeSuffix
versionOlder
versionAtLeast
getName
getVersion
match
split
cmakeOptionType
cmakeBool
cmakeFeature
mesonOption
mesonBool
mesonEnable
nameFromURL
enableFeature
enableFeatureAs
withFeature
withFeatureAs
fixedWidthString
fixedWidthNumber
toInt
toIntBase10
readPathsFromFile
fileContents
;
inherit (self.stringsWithDeps)
textClosureList
textClosureMap
noDepEntry
fullDepEntry
packEntry
stringAfter
;
inherit (self.customisation)
overrideDerivation
makeOverridable
callPackageWith
callPackagesWith
extendDerivation
hydraJob
makeScope
makeScopeWithSplicing
makeScopeWithSplicing'
extendMkDerivation
;
inherit (self.derivations) lazyDerivation optionalDrvAttr warnOnInstantiate;
inherit (self.generators) mkLuaInline;
inherit (self.meta)
addMetaAttrs
dontDistribute
setName
updateName
appendToName
mapDerivationAttrset
setPrio
lowPrio
lowPrioSet
hiPrio
hiPrioSet
licensesSpdx
getLicenseFromSpdxId
getLicenseFromSpdxIdOr
getExe
getExe'
;
inherit (self.filesystem)
pathType
pathIsDirectory
pathIsRegularFile
packagesFromDirectoryRecursive
;
inherit (self.sources)
cleanSourceFilter
cleanSource
sourceByRegex
sourceFilesBySuffices
commitIdFromGitRepo
cleanSourceWith
pathHasContext
canCleanSource
pathIsGitRepo
revOrTag
repoRevToName
;
inherit (self.modules)
evalModules
setDefaultModuleLocation
unifyModuleSyntax
applyModuleArgsIfFunction
mergeModules
mergeModules'
mergeOptionDecls
mergeDefinitions
pushDownProperties
dischargeProperties
filterOverrides
sortProperties
fixupOptionType
mkIf
mkAssert
mkDefinition
mkMerge
mkOverride
mkOptionDefault
mkDefault
mkImageMediaOverride
mkForce
mkVMOverride
mkFixStrictness
mkOrder
mkBefore
mkAfter
mkAliasDefinitions
mkAliasAndWrapDefinitions
fixMergeModules
mkRemovedOptionModule
mkRenamedOptionModule
mkRenamedOptionModuleWith
mkMergedOptionModule
mkChangedOptionModule
mkAliasOptionModule
mkDerivedConfig
doRename
mkAliasOptionModuleMD
;
evalOptionValue = lib.warn "External use of `lib.evalOptionValue` is deprecated. If your use case isn't covered by non-deprecated functions, we'd like to know more and perhaps support your use case well, instead of providing access to these low level functions. In this case please open an issue in https://github.com/nixos/nixpkgs/issues/." self.modules.evalOptionValue;
inherit (self.options)
isOption
mkEnableOption
mkSinkUndeclaredOptions
mergeDefaultOption
mergeOneOption
mergeEqualOption
mergeUniqueOption
getValues
getFiles
optionAttrSetToDocList
optionAttrSetToDocList'
scrubOptionValue
literalExpression
literalExample
showOption
showOptionWithDefLocs
showFiles
unknownModule
mkOption
mkPackageOption
literalMD
;
inherit (self.types)
isType
setType
defaultTypeMerge
defaultFunctor
isOptionType
mkOptionType
;
inherit (self.asserts)
assertMsg
assertOneOf
;
inherit (self.debug)
traceIf
traceVal
traceValFn
traceSeq
traceSeqN
traceValSeq
traceValSeqFn
traceValSeqN
traceValSeqNFn
traceFnSeqN
runTests
testAllTrue
;
inherit (self.misc)
maybeEnv
defaultMergeArg
defaultMerge
foldArgs
maybeAttrNullable
maybeAttr
ifEnable
checkFlag
getValue
checkReqs
uniqList
uniqListExt
condConcat
lazyGenericClosure
innerModifySumArgs
modifySumArgs
innerClosePropagation
closePropagation
mapAttrsFlatten
nvs
setAttr
setAttrMerge
mergeAttrsWithFunc
mergeAttrsConcatenateValues
mergeAttrsNoOverride
mergeAttrByFunc
mergeAttrsByFuncDefaults
mergeAttrsByFuncDefaultsClean
mergeAttrBy
fakeHash
fakeSha256
fakeSha512
nixType
imap
;
inherit (self.versions)
splitVersion
;
inherit (self.network.ipv6)
mkEUI64Suffix
;
}
);
in
lib

10
lib/deprecated/README.md Normal file
View File

@@ -0,0 +1,10 @@
# lib/deprecated
Do not add any new functions to this directory.
This directory contains the `lib.misc` sublibrary, which - as a location - is deprecated.
Furthermore, some of the functions inside are of *dubious* utility, and should perhaps be avoided, while some functions *may still be needed*.
This directory does not play a role in the deprecation process for library functions.
They should be deprecated in place, by putting a `lib.warn` or `lib.warnIf` call around the function.

492
lib/deprecated/misc.nix Normal file
View File

@@ -0,0 +1,492 @@
{ lib }:
let
inherit (lib)
and
any
attrByPath
attrNames
compare
concat
concatMap
elem
filter
foldl
foldr
genericClosure
head
imap1
init
isAttrs
isFunction
isInt
isList
lists
listToAttrs
mapAttrs
mergeAttrs
meta
nameValuePair
tail
toList
warn
;
inherit (lib.attrsets) removeAttrs mapAttrsToList;
# returns default if env var is not set
maybeEnv =
name: default:
let
value = builtins.getEnv name;
in
if value == "" then default else value;
defaultMergeArg = x: y: if builtins.isAttrs y then y else (y x);
defaultMerge = x: y: x // (defaultMergeArg x y);
foldArgs =
merger: f: init: x:
let
arg = (merger init (defaultMergeArg init x));
# now add the function with composed args already applied to the final attrs
base = (
setAttrMerge "passthru" { } (f arg) (
z:
z
// {
function = foldArgs merger f arg;
args = (attrByPath [ "passthru" "args" ] { } z) // x;
}
)
);
withStdOverrides = base // {
override = base.passthru.function;
};
in
withStdOverrides;
# shortcut for attrByPath ["name"] default attrs
maybeAttrNullable = maybeAttr;
# shortcut for attrByPath ["name"] default attrs
maybeAttr =
name: default: attrs:
attrs.${name} or default;
# Returns the second argument if the first one is true or the empty version
# of the second argument.
ifEnable =
cond: val:
if cond then
val
else if builtins.isList val then
[ ]
else if builtins.isAttrs val then
{ }
# else if builtins.isString val then ""
else if val == true || val == false then
false
else
null;
# Returns true only if there is an attribute and it is true.
checkFlag =
attrSet: name:
if name == "true" then
true
else if name == "false" then
false
else if (elem name (attrByPath [ "flags" ] [ ] attrSet)) then
true
else
attrByPath [ name ] false attrSet;
# Input : attrSet, [ [name default] ... ], name
# Output : its value or default.
getValue =
attrSet: argList: name:
(attrByPath [ name ] (
if checkFlag attrSet name then
true
else if argList == [ ] then
null
else
let
x = builtins.head argList;
in
if (head x) == name then (head (tail x)) else (getValue attrSet (tail argList) name)
) attrSet);
# Input : attrSet, [[name default] ...], [ [flagname reqs..] ... ]
# Output : are reqs satisfied? It's asserted.
checkReqs =
attrSet: argList: condList:
(foldr and true (
map (
x:
let
name = (head x);
in
(
(checkFlag attrSet name)
-> (foldr and true (
map (
y:
let
val = (getValue attrSet argList y);
in
(val != null) && (val != false)
) (tail x)
))
)
) condList
));
# This function has O(n^2) performance.
uniqList =
{
inputList,
acc ? [ ],
}:
let
go =
xs: acc:
if xs == [ ] then
[ ]
else
let
x = head xs;
y = if elem x acc then [ ] else [ x ];
in
y ++ go (tail xs) (y ++ acc);
in
go inputList acc;
uniqListExt =
{
inputList,
outputList ? [ ],
getter ? (x: x),
compare ? (x: y: x == y),
}:
if inputList == [ ] then
outputList
else
let
x = head inputList;
isX = y: (compare (getter y) (getter x));
newOutputList = outputList ++ (if any isX outputList then [ ] else [ x ]);
in
uniqListExt {
outputList = newOutputList;
inputList = (tail inputList);
inherit getter compare;
};
condConcat =
name: list: checker:
if list == [ ] then
name
else if checker (head list) then
condConcat (name + (head (tail list))) (tail (tail list)) checker
else
condConcat name (tail (tail list)) checker;
lazyGenericClosure =
{ startSet, operator }:
let
work =
list: doneKeys: result:
if list == [ ] then
result
else
let
x = head list;
key = x.key;
in
if elem key doneKeys then
work (tail list) doneKeys result
else
work (tail list ++ operator x) ([ key ] ++ doneKeys) ([ x ] ++ result);
in
work startSet [ ] [ ];
innerModifySumArgs =
f: x: a: b:
if b == null then (f a b) // x else innerModifySumArgs f x (a // b);
modifySumArgs = f: x: innerModifySumArgs f x { };
innerClosePropagation =
acc: xs:
if xs == [ ] then
acc
else
let
y = head xs;
ys = tail xs;
in
if !isAttrs y then
innerClosePropagation acc ys
else
let
acc' = [ y ] ++ acc;
in
innerClosePropagation acc' (uniqList {
inputList =
(maybeAttrNullable "propagatedBuildInputs" [ ] y)
++ (maybeAttrNullable "propagatedNativeBuildInputs" [ ] y)
++ ys;
acc = acc';
});
closePropagationSlow = list: (uniqList { inputList = (innerClosePropagation [ ] list); });
# This is an optimisation of closePropagation which avoids the O(n^2) behavior
# Using a list of derivations, it generates the full closure of the propagatedXXXBuildInputs
# The ordering / sorting / comparison is done based on the `outPath`
# attribute of each derivation.
# On some benchmarks, it performs up to 15 times faster than closePropagation.
# See https://github.com/NixOS/nixpkgs/pull/194391 for details.
closePropagationFast =
list:
map (x: x.val) (
builtins.genericClosure {
startSet = map (x: {
key = x.outPath;
val = x;
}) (builtins.filter (x: x != null) list);
operator =
item:
if !builtins.isAttrs item.val then
[ ]
else
builtins.concatMap (
x:
if x != null then
[
{
key = x.outPath;
val = x;
}
]
else
[ ]
) ((item.val.propagatedBuildInputs or [ ]) ++ (item.val.propagatedNativeBuildInputs or [ ]));
}
);
closePropagation = if builtins ? genericClosure then closePropagationFast else closePropagationSlow;
# calls a function (f attr value ) for each record item. returns a list
mapAttrsFlatten = warn "lib.misc.mapAttrsFlatten is deprecated, please use lib.attrsets.mapAttrsToList instead." mapAttrsToList;
# attribute set containing one attribute
nvs = name: value: listToAttrs [ (nameValuePair name value) ];
# adds / replaces an attribute of an attribute set
setAttr =
set: name: v:
set // (nvs name v);
# setAttrMerge (similar to mergeAttrsWithFunc but only merges the values of a particular name)
# setAttrMerge "a" [] { a = [2];} (x: x ++ [3]) -> { a = [2 3]; }
# setAttrMerge "a" [] { } (x: x ++ [3]) -> { a = [ 3]; }
setAttrMerge =
name: default: attrs: f:
setAttr attrs name (f (maybeAttr name default attrs));
# Using f = a: b = b the result is similar to //
# merge attributes with custom function handling the case that the attribute
# exists in both sets
mergeAttrsWithFunc =
f: set1: set2:
foldr (n: set: if set ? ${n} then setAttr set n (f set.${n} set2.${n}) else set) (set2 // set1) (
attrNames set2
);
# merging two attribute set concatenating the values of same attribute names
# eg { a = 7; } { a = [ 2 3 ]; } becomes { a = [ 7 2 3 ]; }
mergeAttrsConcatenateValues = mergeAttrsWithFunc (a: b: (toList a) ++ (toList b));
# merges attributes using //, if a name exists in both attributes
# an error will be triggered unless its listed in mergeLists
# so you can mergeAttrsNoOverride { buildInputs = [a]; } { buildInputs = [a]; } {} to get
# { buildInputs = [a b]; }
# merging buildPhase doesn't really make sense. The cases will be rare where appending /prefixing will fit your needs?
# in these cases the first buildPhase will override the second one
# ! deprecated, use mergeAttrByFunc instead
mergeAttrsNoOverride =
{
mergeLists ? [
"buildInputs"
"propagatedBuildInputs"
],
overrideSnd ? [ "buildPhase" ],
}:
attrs1: attrs2:
foldr (
n: set:
setAttr set n (
if set ? ${n} then # merge
if
elem n mergeLists # attribute contains list, merge them by concatenating
then
attrs2.${n} ++ attrs1.${n}
else if elem n overrideSnd then
attrs1.${n}
else
throw "error mergeAttrsNoOverride, attribute ${n} given in both attributes - no merge func defined"
else
attrs2.${n} # add attribute not existing in attr1
)
) attrs1 (attrNames attrs2);
# example usage:
# mergeAttrByFunc {
# inherit mergeAttrBy; # defined below
# buildInputs = [ a b ];
# } {
# buildInputs = [ c d ];
# };
# will result in
# { mergeAttrsBy = [...]; buildInputs = [ a b c d ]; }
# is used by defaultOverridableDelayableArgs and can be used when composing using
# foldArgs, composedArgsAndFun or applyAndFun. Example: composableDerivation in all-packages.nix
mergeAttrByFunc =
x: y:
let
mergeAttrBy2 = {
mergeAttrBy = mergeAttrs;
}
// (maybeAttr "mergeAttrBy" { } x)
// (maybeAttr "mergeAttrBy" { } y);
in
foldr mergeAttrs { } [
x
y
(mapAttrs
(
a: v: # merge special names using given functions
if x ? ${a} then
if y ? ${a} then
v x.${a} y.${a} # both have attr, use merge func
else
x.${a} # only x has attr
else
y.${a} # only y has attr)
)
(
removeAttrs mergeAttrBy2
# don't merge attrs which are neither in x nor y
(filter (a: !x ? ${a} && !y ? ${a}) (attrNames mergeAttrBy2))
)
)
];
mergeAttrsByFuncDefaults = foldl mergeAttrByFunc { inherit mergeAttrBy; };
mergeAttrsByFuncDefaultsClean = list: removeAttrs (mergeAttrsByFuncDefaults list) [ "mergeAttrBy" ];
# sane defaults (same name as attr name so that inherit can be used)
mergeAttrBy = # { buildInputs = concatList; [...]; passthru = mergeAttr; [..]; }
listToAttrs (
map (n: nameValuePair n concat) [
"nativeBuildInputs"
"buildInputs"
"propagatedBuildInputs"
"configureFlags"
"prePhases"
"postAll"
"patches"
]
)
// listToAttrs (
map (n: nameValuePair n mergeAttrs) [
"passthru"
"meta"
"cfg"
"flags"
]
)
// listToAttrs (
map (n: nameValuePair n (a: b: "${a}\n${b}")) [
"preConfigure"
"postInstall"
]
);
nixType =
x:
if isAttrs x then
if x ? outPath then "derivation" else "attrs"
else if isFunction x then
"function"
else if isList x then
"list"
else if x == true then
"bool"
else if x == false then
"bool"
else if x == null then
"null"
else if isInt x then
"int"
else
"string";
/**
# Deprecated
For historical reasons, imap has an index starting at 1.
But for consistency with the rest of the library we want an index
starting at zero.
*/
imap = imap1;
# Fake hashes. Can be used as hash placeholders, when computing hash ahead isn't trivial
fakeHash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
fakeSha256 = "0000000000000000000000000000000000000000000000000000000000000000";
fakeSha512 = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000";
in
# Everything in this attrset is the public interface of the file.
{
inherit
checkFlag
checkReqs
closePropagation
closePropagationFast
closePropagationSlow
condConcat
defaultMerge
defaultMergeArg
fakeHash
fakeSha256
fakeSha512
foldArgs
getValue
ifEnable
imap
innerClosePropagation
innerModifySumArgs
lazyGenericClosure
mapAttrsFlatten
maybeAttr
maybeAttrNullable
maybeEnv
mergeAttrBy
mergeAttrByFunc
mergeAttrsByFuncDefaults
mergeAttrsByFuncDefaultsClean
mergeAttrsConcatenateValues
mergeAttrsNoOverride
mergeAttrsWithFunc
modifySumArgs
nixType
nvs
setAttr
setAttrMerge
uniqList
uniqListExt
;
}

254
lib/derivations.nix Normal file
View File

@@ -0,0 +1,254 @@
{ lib }:
let
inherit (lib)
genAttrs
isString
mapAttrs
removeAttrs
throwIfNot
;
showMaybeAttrPosPre =
prefix: attrName: v:
let
pos = builtins.unsafeGetAttrPos attrName v;
in
if pos == null then "" else "${prefix}${pos.file}:${toString pos.line}:${toString pos.column}";
showMaybePackagePosPre =
prefix: pkg:
if pkg ? meta.position && isString pkg.meta.position then "${prefix}${pkg.meta.position}" else "";
in
{
/**
Restrict a derivation to a predictable set of attribute names, so
that the returned attrset is not strict in the actual derivation,
saving a lot of computation when the derivation is non-trivial.
This is useful in situations where a derivation might only be used for its
passthru attributes, improving evaluation performance.
The returned attribute set is lazy in `derivation`. Specifically, this
means that the derivation will not be evaluated in at least the
situations below.
For illustration and/or testing, we define derivation such that its
evaluation is very noticeable.
let derivation = throw "This won't be evaluated.";
In the following expressions, `derivation` will _not_ be evaluated:
(lazyDerivation { inherit derivation; }).type
attrNames (lazyDerivation { inherit derivation; })
(lazyDerivation { inherit derivation; } // { foo = true; }).foo
(lazyDerivation { inherit derivation; meta.foo = true; }).meta
In these expressions, `derivation` _will_ be evaluated:
"${lazyDerivation { inherit derivation }}"
(lazyDerivation { inherit derivation }).outPath
(lazyDerivation { inherit derivation }).meta
And the following expressions are not valid, because the refer to
implementation details and/or attributes that may not be present on
some derivations:
(lazyDerivation { inherit derivation }).buildInputs
(lazyDerivation { inherit derivation }).passthru
(lazyDerivation { inherit derivation }).pythonPath
# Inputs
Takes an attribute set with the following attributes
`derivation`
: The derivation to be wrapped.
`meta`
: Optional meta attribute.
While this function is primarily about derivations, it can improve
the `meta` package attribute, which is usually specified through
`mkDerivation`.
`passthru`
: Optional extra values to add to the returned attrset.
This can be used for adding package attributes, such as `tests`.
`outputs`
: Optional list of assumed outputs. Default: ["out"]
This must match the set of outputs that the returned derivation has.
You must use this when the derivation has multiple outputs.
*/
lazyDerivation =
args@{
derivation,
meta ? null,
passthru ? { },
outputs ? [ "out" ],
}:
let
# These checks are strict in `drv` and some `drv` attributes, but the
# attrset spine returned by lazyDerivation does not depend on it.
# Instead, the individual derivation attributes do depend on it.
checked =
throwIfNot (derivation.type or null == "derivation") "lazyDerivation: input must be a derivation."
throwIfNot
# NOTE: Technically we could require our outputs to be a subset of the
# actual ones, or even leave them unchecked and fail on a lazy basis.
# However, consider the case where an output is added in the underlying
# derivation, such as dev. lazyDerivation would remove it and cause it
# to fail as a buildInputs item, without any indication as to what
# happened. Hence the more stringent condition. We could consider
# adding a flag to control this behavior if there's a valid case for it,
# but the documentation must have a note like this.
(derivation.outputs == outputs)
''
lib.lazyDerivation: The derivation ${derivation.name or "<unknown>"} has outputs that don't match the assumed outputs.
Assumed outputs passed to lazyDerivation${showMaybeAttrPosPre ",\n at " "outputs" args}:
${lib.generators.toPretty { multiline = false; } outputs};
Actual outputs of the derivation${showMaybePackagePosPre ",\n defined at " derivation}:
${lib.generators.toPretty { multiline = false; } derivation.outputs}
If the outputs are known ahead of evaluating the derivation,
then update the lazyDerivation call to match the actual outputs, in the same order.
If lazyDerivation is passed a literal value, just change it to the actual outputs.
As a result it will work as before / as intended.
Otherwise, when the outputs are dynamic and can't be known ahead of time, it won't
be possible to add laziness, but lib.lazyDerivation may still be useful for trimming
the attributes.
If you want to keep trimming the attributes, make sure that the package is in a
variable (don't evaluate it twice!) and pass the variable and its outputs attribute
to lib.lazyDerivation. This largely defeats laziness, but keeps the trimming.
If none of the above works for you, replace the lib.lazyDerivation call by the
expression in the derivation argument.
''
derivation;
in
{
# Hardcoded `type`
#
# `lazyDerivation` requires its `derivation` argument to be a derivation,
# so if it is not, that is a programming error by the caller and not
# something that `lazyDerivation` consumers should be able to correct
# for after the fact.
# So, to improve laziness, we assume correctness here and check it only
# when actual derivation values are accessed later.
type = "derivation";
# A fixed set of derivation values, so that `lazyDerivation` can return
# its attrset before evaluating `derivation`.
# This must only list attributes that are available on _all_ derivations.
inherit (checked)
outPath
outputName
drvPath
name
system
;
inherit outputs;
# The meta attribute can either be taken from the derivation, or if the
# `lazyDerivation` caller knew a shortcut, be taken from there.
meta = args.meta or checked.meta;
}
// genAttrs outputs (outputName: checked.${outputName})
// passthru;
/**
Conditionally set a derivation attribute.
Because `mkDerivation` sets `__ignoreNulls = true`, a derivation
attribute set to `null` will not impact the derivation output hash.
Thus, this function passes through its `value` argument if the `cond`
is `true`, but returns `null` if not.
# Inputs
`cond`
: Condition
`value`
: Attribute value
# Type
```
optionalDrvAttr :: Bool -> a -> a | Null
```
# Examples
:::{.example}
## `lib.derivations.optionalDrvAttr` usage example
```nix
(stdenv.mkDerivation {
name = "foo";
x = optionalDrvAttr true 1;
y = optionalDrvAttr false 1;
}).drvPath == (stdenv.mkDerivation {
name = "foo";
x = 1;
}).drvPath
=> true
```
:::
*/
optionalDrvAttr = cond: value: if cond then value else null;
/**
Wrap a derivation such that instantiating it produces a warning.
All attributes will be wrapped with `lib.warn` except from `.meta`, `.name`,
and `.type` which are used by `nix search`, and `.outputName` which avoids
double warnings with `nix-instantiate` and `nix-build`.
# Inputs
`msg`
: The warning message to emit (via `lib.warn`).
`drv`
: The derivation to wrap.
# Examples
:::{.example}
## `lib.derivations.warnOnInstantiate` usage example
```nix
{
myPackage = warnOnInstantiate "myPackage has been renamed to my-package" my-package;
}
```
:::
*/
warnOnInstantiate =
msg: drv:
let
drvToWrap = removeAttrs drv [
"meta"
"name"
"type"
"outputName"
];
in
drv // mapAttrs (_: lib.warn msg) drvToWrap;
}

220
lib/fetchers.nix Normal file
View File

@@ -0,0 +1,220 @@
# snippets that can be shared by multiple fetchers (pkgs/build-support)
{ lib }:
let
commonH = hashTypes: rec {
hashNames = [ "hash" ] ++ hashTypes;
hashSet = lib.genAttrs hashNames (lib.const { });
};
fakeH = {
hash = lib.fakeHash;
sha256 = lib.fakeSha256;
sha512 = lib.fakeSha512;
};
in
rec {
proxyImpureEnvVars = [
# We borrow these environment variables from the caller to allow
# easy proxy configuration. This is impure, but a fixed-output
# derivation like fetchurl is allowed to do so since its result is
# by definition pure.
"http_proxy"
"https_proxy"
"ftp_proxy"
"all_proxy"
"no_proxy"
"HTTP_PROXY"
"HTTPS_PROXY"
"FTP_PROXY"
"ALL_PROXY"
"NO_PROXY"
# https proxies typically need to inject custom root CAs too
"NIX_SSL_CERT_FILE"
];
/**
Converts an attrset containing one of `hash`, `sha256` or `sha512`,
into one containing `outputHash{,Algo}` as accepted by `mkDerivation`.
An appropriate “fake hash” is substituted when the hash value is `""`,
as is the [convention for fetchers](#sec-pkgs-fetchers-updating-source-hashes-fakehash-method).
All other attributes in the set remain as-is.
# Example
```nix
normalizeHash { } { hash = ""; foo = "bar"; }
=>
{
outputHash = lib.fakeHash;
outputHashAlgo = null;
foo = "bar";
}
```
```nix
normalizeHash { } { sha256 = lib.fakeSha256; }
=>
{
outputHash = lib.fakeSha256;
outputHashAlgo = "sha256";
}
```
```nix
normalizeHash { } { sha512 = lib.fakeSha512; }
=>
{
outputHash = lib.fakeSha512;
outputHashAlgo = "sha512";
}
```
# Type
```
normalizeHash :: { hashTypes :: List String, required :: Bool } -> AttrSet -> AttrSet
```
# Arguments
hashTypes
: the set of attribute names accepted as hash inputs, in addition to `hash`
required
: whether to throw if no hash was present in the input; otherwise returns the original input, unmodified
*/
normalizeHash =
{
hashTypes ? [ "sha256" ],
required ? true,
}:
let
inherit (lib)
concatMapStringsSep
head
tail
throwIf
;
inherit (lib.attrsets)
attrsToList
intersectAttrs
removeAttrs
optionalAttrs
;
inherit (commonH hashTypes) hashNames hashSet;
in
args:
if args ? "outputHash" then
args
else
let
# The argument hash, as a {name, value} pair
h =
# All hashes passed in arguments (possibly 0 or >1) as a list of {name, value} pairs
let
hashesAsNVPairs = attrsToList (intersectAttrs hashSet args);
in
if hashesAsNVPairs == [ ] then
throwIf required "fetcher called without `hash`" null
else if tail hashesAsNVPairs != [ ] then
throw "fetcher called with mutually-incompatible arguments: ${
concatMapStringsSep ", " (a: a.name) hashesAsNVPairs
}"
else
head hashesAsNVPairs;
in
removeAttrs args hashNames
// (optionalAttrs (h != null) {
outputHashAlgo = if h.name == "hash" then null else h.name;
outputHash =
if h.value == "" then
fakeH.${h.name} or (throw "no “fake hash” defined for ${h.name}")
else
h.value;
});
/**
Wraps a function which accepts `outputHash{,Algo}` into one which accepts `hash` or `sha{256,512}`
# Example
```nix
withNormalizedHash { hashTypes = [ "sha256" "sha512" ]; } (
{ outputHash, outputHashAlgo, ... }:
...
)
```
is a function which accepts one of `hash`, `sha256`, or `sha512` (or the original's `outputHash` and `outputHashAlgo`).
Its `functionArgs` metadata only lists `hash` as a parameter, optional iff. `outputHash` was an optional parameter of
the original function. `sha256`, `sha512`, `outputHash`, or `outputHashAlgo` are not mentioned in the `functionArgs`
metadata.
# Type
```
withNormalizedHash :: { hashTypes :: List String } -> (AttrSet -> T) -> (AttrSet -> T)
```
# Arguments
hashTypes
: the set of attribute names accepted as hash inputs, in addition to `hash`
: they must correspond to a valid value for `outputHashAlgo`, currently one of: `md5`, `sha1`, `sha256`, or `sha512`.
f
: the function to be wrapped
::: {.note}
In nixpkgs, `mkDerivation` rejects MD5 `outputHash`es, and SHA-1 is being deprecated.
As such, there is no reason to add `md5` to `hashTypes`, and
`sha1` should only ever be included for backwards compatibility.
:::
# Output
`withNormalizedHash { inherit hashTypes; } f` is functionally equivalent to
```nix
args: f (normalizeHash {
inherit hashTypes;
required = !(lib.functionArgs f).outputHash;
} args)
```
However, `withNormalizedHash` preserves `functionArgs` metadata insofar as possible,
and is implemented somewhat more efficiently.
*/
withNormalizedHash =
{
hashTypes ? [ "sha256" ],
}:
fetcher:
let
inherit (lib.attrsets) intersectAttrs removeAttrs;
inherit (lib.trivial) functionArgs setFunctionArgs;
inherit (commonH hashTypes) hashSet;
fArgs = functionArgs fetcher;
normalize = normalizeHash {
inherit hashTypes;
required = !fArgs.outputHash;
};
in
# The o.g. fetcher must *only* accept outputHash and outputHashAlgo
assert fArgs ? outputHash && fArgs ? outputHashAlgo;
assert intersectAttrs fArgs hashSet == { };
setFunctionArgs (args: fetcher (normalize args)) (
removeAttrs fArgs [
"outputHash"
"outputHashAlgo"
]
// {
hash = fArgs.outputHash;
}
);
}

269
lib/fileset/README.md Normal file
View File

@@ -0,0 +1,269 @@
# File set library
This is the internal contributor documentation.
The user documentation is [in the Nixpkgs manual](https://nixos.org/manual/nixpkgs/unstable/#sec-fileset).
## Goals
The main goal of the file set library is to be able to select local files that should be added to the Nix store.
It should have the following properties:
- Easy:
The functions should have obvious semantics, be low in number and be composable.
- Safe:
Throw early and helpful errors when mistakes are detected.
- Lazy:
Only compute values when necessary.
Non-goals are:
- Efficient:
If the abstraction proves itself worthwhile but too slow, it can be still be optimized further.
## Tests
Tests are declared in [`tests.sh`](./tests.sh) and can be run using
```
./tests.sh
```
## Benchmark
A simple benchmark against the HEAD commit can be run using
```
./benchmark.sh HEAD
```
This is intended to be run manually and is not checked by CI.
## Internal representation
The internal representation is versioned in order to allow file sets from different Nixpkgs versions to be composed with each other, see [`internal.nix`](./internal.nix) for the versions and conversions between them.
This section describes only the current representation, but past versions will have to be supported by the code.
### `fileset`
An attribute set with these values:
- `_type` (constant string `"fileset"`):
Tag to indicate this value is a file set.
- `_internalVersion` (constant `3`, the current version):
Version of the representation.
- `_internalIsEmptyWithoutBase` (bool):
Whether this file set is the empty file set without a base path.
If `true`, `_internalBase*` and `_internalTree` are not set.
This is the only way to represent an empty file set without needing a base path.
Such a value can be used as the identity element for `union` and the return value of `unions []` and co.
- `_internalBase` (path):
Any files outside of this path cannot influence the set of files.
This is always a directory and should be as long as possible.
This is used by `lib.fileset.toSource` to check that all files are under the `root` argument
- `_internalBaseRoot` (path):
The filesystem root of `_internalBase`, same as `(lib.path.splitRoot _internalBase).root`.
This is here because this needs to be computed anyway, and this computation shouldn't be duplicated.
- `_internalBaseComponents` (list of strings):
The path components of `_internalBase`, same as `lib.path.subpath.components (lib.path.splitRoot _internalBase).subpath`.
This is here because this needs to be computed anyway, and this computation shouldn't be duplicated.
- `_internalTree` ([filesetTree](#filesettree)):
A tree representation of all included files under `_internalBase`.
- `__noEval` (error):
An error indicating that directly evaluating file sets is not supported.
## `filesetTree`
One of the following:
- `{ <name> = filesetTree; }`:
A directory with a nested `filesetTree` value for directory entries.
Entries not included may either be omitted or set to `null`, as necessary to improve efficiency or laziness.
- `"directory"`:
A directory with all its files included recursively, allowing early cutoff for some operations.
This specific string is chosen to be compatible with `builtins.readDir` for a simpler implementation.
- `"regular"`, `"symlink"`, `"unknown"` or any other non-`"directory"` string:
A nested file with its file type.
These specific strings are chosen to be compatible with `builtins.readDir` for a simpler implementation.
Distinguishing between different file types is not strictly necessary for the functionality this library,
but it does allow nicer printing of file sets.
- `null`:
A file or directory that is excluded from the tree.
It may still exist on the file system.
## API design decisions
This section justifies API design decisions.
### Internal structure
The representation of the file set data type is internal and can be changed over time.
Arguments:
- (+) The point of this library is to provide high-level functions, users don't need to be concerned with how it's implemented
- (+) It allows adjustments to the representation, which is especially useful in the early days of the library.
- (+) It still allows the representation to be stabilized later if necessary and if it has proven itself
### Influence tracking
File set operations internally track the top-most directory that could influence the exact contents of a file set.
Specifically, `toSource` requires that the given `fileset` is completely determined by files within the directory specified by the `root` argument.
For example, even with `dir/file.txt` being the only file in `./.`, `toSource { root = ./dir; fileset = ./.; }` gives an error.
This is because `fileset` may as well be the result of filtering `./.` in a way that excludes `dir`.
Arguments:
- (+) This gives us the guarantee that adding new files to a project never breaks a file set expression.
This is also true in a lesser form for removed files:
only removing files explicitly referenced by paths can break a file set expression.
- (+) This can be removed later, if we discover it's too restrictive
- (-) It leads to errors when a sensible result could sometimes be returned, such as in the above example.
### Empty file set without a base
There is a special representation for an empty file set without a base path.
This is used for return values that should be empty but when there's no base path that would makes sense.
Arguments:
- Alternative: This could also be represented using `_internalBase = /.` and `_internalTree = null`.
- (+) Removes the need for a special representation.
- (-) Due to [influence tracking](#influence-tracking),
`union empty ./.` would have `/.` as the base path,
which would then prevent `toSource { root = ./.; fileset = union empty ./.; }` from working,
which is not as one would expect.
- (-) With the assumption that there can be multiple filesystem roots (as established with the [path library](../path/README.md)),
this would have to cause an error with `union empty pathWithAnotherFilesystemRoot`,
which is not as one would expect.
- Alternative: Do not have such a value and error when it would be needed as a return value
- (+) Removes the need for a special representation.
- (-) Leaves us with no identity element for `union` and no reasonable return value for `unions []`.
From a set theory perspective, which has a well-known notion of empty sets, this is unintuitive.
### No intersection for lists
While there is `intersection a b`, there is no function `intersections [ a b c ]`.
Arguments:
- (+) There is no known use case for such a function, it can be added later if a use case arises
- (+) There is no suitable return value for `intersections [ ]`, see also "Nullary intersections" [here](https://en.wikipedia.org/w/index.php?title=List_of_set_identities_and_relations&oldid=1177174035#Definitions)
- (-) Could throw an error for that case
- (-) Create a special value to represent "all the files" and return that
- (+) Such a value could then not be used with `fileFilter` unless the internal representation is changed considerably
- (-) Could return the empty file set
- (+) This would be wrong in set theory
- (-) Inconsistent with `union` and `unions`
### Intersection base path
The base path of the result of an `intersection` is the longest base path of the arguments.
E.g. the base path of `intersection ./foo ./foo/bar` is `./foo/bar`.
Meanwhile `intersection ./foo ./bar` returns the empty file set without a base path.
Arguments:
- Alternative: Use the common prefix of all base paths as the resulting base path
- (-) This is unnecessarily strict, because the purpose of the base path is to track the directory under which files _could_ be in the file set.
It should be as long as possible.
All files contained in `intersection ./foo ./foo/bar` will be under `./foo/bar` (never just under `./foo`), and `intersection ./foo ./bar` will never contain any files (never under `./.`).
This would lead to `toSource` having to unexpectedly throw errors for cases such as `toSource { root = ./foo; fileset = intersect ./foo base; }`, where `base` may be `./bar` or `./.`.
- (-) There is no benefit to the user, since base path is not directly exposed in the interface
### Empty directories
File sets can only represent a _set_ of local files.
Directories on their own are not representable.
Arguments:
- (+) There does not seem to be a sensible set of combinators when directories can be represented on their own.
Here's some possibilities:
- `./.` represents the files in `./.` _and_ the directory itself including its subdirectories, meaning that even if there's no files, the entire structure of `./.` is preserved
In that case, what should `fileFilter (file: false) ./.` return?
It could return the entire directory structure unchanged, but with all files removed, which would not be what one would expect.
Trying to have a filter function that also supports directories will lead to the question of:
What should the behavior be if `./foo` itself is excluded but all of its contents are included?
It leads to having to define when directories are recursed into, but then we're effectively back at how the `builtins.path`-based filters work.
- `./.` represents all files in `./.` _and_ the directory itself, but not its subdirectories, meaning that at least `./.` will be preserved even if it's empty.
In that case, `intersection ./. ./foo` should only include files and no directories themselves, since `./.` includes only `./.` as a directory, and same for `./foo`, so there's no overlap in directories.
But intuitively this operation should result in the same as `./foo` – everything else is just confusing.
- (+) This matches how Git only supports files, so developers should already be used to it.
- (-) Empty directories (even if they contain nested directories) are neither representable nor preserved when coercing from paths.
- (+) It is very rare that empty directories are necessary.
- (+) We can implement a workaround, allowing `toSource` to take an extra argument for ensuring certain extra directories exist in the result.
- (-) It slows down store imports, since the evaluator needs to traverse the entire tree to remove any empty directories
- (+) This can still be optimized by introducing more Nix builtins if necessary
### String paths
File sets do not support Nix store paths in strings such as `"/nix/store/...-source"`.
Arguments:
- (+) Such paths are usually produced by derivations, which means `toSource` would either:
- Require [Import From Derivation](https://nixos.org/manual/nix/unstable/language/import-from-derivation) (IFD) if `builtins.path` is used as the underlying primitive
- Require importing the entire `root` into the store such that derivations can be used to do the filtering
- (+) The convenient path coercion like `union ./foo ./bar` wouldn't work for absolute paths, requiring more verbose alternate interfaces:
- `let root = "/nix/store/...-source"; in union "${root}/foo" "${root}/bar"`
Verbose and dangerous because if `root` was a path, the entire path would get imported into the store.
- `toSource { root = "/nix/store/...-source"; fileset = union "./foo" "./bar"; }`
Does not allow debug printing intermediate file set contents, since we don't know the paths contents before having a `root`.
- `let fs = lib.fileset.withRoot "/nix/store/...-source"; in fs.union "./foo" "./bar"`
Makes library functions impure since they depend on the contextual root path, questionable composability.
- (+) The point of the file set abstraction is to specify which files should get imported into the store.
This use case makes little sense for files that are already in the store.
This should be a separate abstraction as e.g. `pkgs.drvLayout` instead, which could have a similar interface but be specific to derivations.
Additional capabilities could be supported that can't be done at evaluation time, such as renaming files, creating new directories, setting executable bits, etc.
- (+) An API for filtering/transforming Nix store paths could be much more powerful,
because it's not limited to just what is possible at evaluation time with `builtins.path`.
Operations such as moving and adding files would be supported.
### Single files
File sets cannot add single files to the store, they can only import files under directories.
Arguments:
- (+) There's no point in using this library for a single file, since you can't do anything other than add it to the store or not.
And it would be unclear how the library should behave if the one file wouldn't be added to the store:
`toSource { root = ./file.nix; fileset = <empty>; }` has no reasonable result because returning an empty store path wouldn't match the file type, and there's no way to have an empty file store path, whatever that would mean.
### `fileFilter` takes a path
The `fileFilter` function takes a path, and not a file set, as its second argument.
- (-) Makes it harder to compose functions, since the file set type, the return value, can't be passed to the function itself like `fileFilter predicate fileset`
- (+) It's still possible to use `intersection` to filter on file sets: `intersection fileset (fileFilter predicate ./.)`
- (-) This does need an extra `./.` argument that's not obvious
- (+) This could always be `/.` or the project directory, `intersection` will make it lazy
- (+) In the future this will allow `fileFilter` to support a predicate property like `subpath` and/or `components` in a reproducible way.
This wouldn't be possible if it took a file set, because file sets don't have a predictable absolute path.
- (-) What about the base path?
- (+) That can change depending on which files are included, so if it's used for `fileFilter`
it would change the `subpath`/`components` value depending on which files are included.
- (+) If necessary, this restriction can be relaxed later, the opposite wouldn't be possible
### Strict path existence checking
Coercing paths that don't exist to file sets always gives an error.
- (-) Sometimes you want to remove a file that may not always exist using `difference ./. ./does-not-exist`,
but this does not work because coercion of `./does-not-exist` fails,
even though its existence would have no influence on the result.
- (+) This is dangerous, because you wouldn't be protected against typos anymore.
E.g. when trying to prevent `./secret` from being imported, a typo like `difference ./. ./sercet` would import it regardless.
- (+) `difference ./. (maybeMissing ./does-not-exist)` can be used to do this more explicitly.
- (+) `difference ./. (difference ./foo ./foo/bar)` should report an error when `./foo/bar` does not exist ("double negation").
Unfortunately, the current internal representation does not lend itself to a behavior where both `difference x ./does-not-exists` and double negation are handled and checked correctly.
This could be fixed, but would require significant changes to the internal representation that are not worth the effort and the risk of introducing implicit behavior.

140
lib/fileset/benchmark.sh Executable file
View File

@@ -0,0 +1,140 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p sta jq bc nix -I nixpkgs=../..
# shellcheck disable=SC2016
# Benchmarks lib.fileset
# Run:
# [nixpkgs]$ lib/fileset/benchmark.sh HEAD
set -euo pipefail
shopt -s inherit_errexit dotglob
if (( $# == 0 )); then
echo "Usage: $0 HEAD"
echo "Benchmarks the current tree against the HEAD commit. Any git ref will work."
exit 1
fi
compareTo=$1
SCRIPT_FILE=$(readlink -f "${BASH_SOURCE[0]}")
SCRIPT_DIR=$(dirname "$SCRIPT_FILE")
nixpkgs=$(cd "$SCRIPT_DIR/../.."; pwd)
tmp="$(mktemp -d)"
clean_up() {
rm -rf "$tmp"
}
trap clean_up EXIT SIGINT SIGTERM
work="$tmp/work"
mkdir "$work"
cd "$work"
declare -a stats=(
".envs.elements"
".envs.number"
".gc.totalBytes"
".list.concats"
".list.elements"
".nrFunctionCalls"
".nrLookups"
".nrOpUpdates"
".nrPrimOpCalls"
".nrThunks"
".sets.elements"
".sets.number"
".symbols.number"
".values.number"
)
runs=10
run() {
# Empty the file
: > cpuTimes
for i in $(seq 0 "$runs"); do
NIX_PATH=nixpkgs=$1 NIX_SHOW_STATS=1 NIX_SHOW_STATS_PATH=$tmp/stats.json \
nix-instantiate --eval --strict --show-trace >/dev/null \
--expr 'with import <nixpkgs/lib>; with fileset; '"$2"
# Only measure the time after the first run, one is warmup
if (( i > 0 )); then
jq '.cpuTime' "$tmp/stats.json" >> cpuTimes
fi
done
# Compute mean and standard deviation
read -r mean sd < <(sta --mean --sd --brief <cpuTimes)
jq --argjson mean "$mean" --argjson sd "$sd" \
'.cpuTimeMean = $mean | .cpuTimeSd = $sd' \
"$tmp/stats.json"
}
bench() {
echo "Benchmarking expression $1" >&2
#echo "Running benchmark on index" >&2
run "$nixpkgs" "$1" > "$tmp/new.json"
(
#echo "Checking out $compareTo" >&2
git -C "$nixpkgs" worktree add --quiet "$tmp/worktree" "$compareTo"
trap 'git -C "$nixpkgs" worktree remove "$tmp/worktree"' EXIT
#echo "Running benchmark on $compareTo" >&2
run "$tmp/worktree" "$1" > "$tmp/old.json"
)
read -r oldMean oldSd newMean newSd percentageMean percentageSd < \
<(jq -rn --slurpfile old "$tmp/old.json" --slurpfile new "$tmp/new.json" \
' $old[0].cpuTimeMean as $om
| $old[0].cpuTimeSd as $os
| $new[0].cpuTimeMean as $nm
| $new[0].cpuTimeSd as $ns
| (100 / $om * $nm) as $pm
# Copied from https://github.com/sharkdp/hyperfine/blob/b38d550b89b1dab85139eada01c91a60798db9cc/src/benchmark/relative_speed.rs#L46-L53
| ($pm * pow(pow($ns / $nm; 2) + pow($os / $om; 2); 0.5)) as $ps
| [ $om, $os, $nm, $ns, $pm, $ps ]
| @sh')
echo -e "Mean CPU time $newMean (σ = $newSd) for $runs runs is \e[0;33m$percentageMean% (σ = $percentageSd%)\e[0m of the old value $oldMean (σ = $oldSd)" >&2
different=0
for stat in "${stats[@]}"; do
oldValue=$(jq "$stat" "$tmp/old.json")
newValue=$(jq "$stat" "$tmp/new.json")
if (( oldValue != newValue )); then
percent=$(bc <<< "scale=100; result = 100/$oldValue*$newValue; scale=4; result / 1")
if (( oldValue < newValue )); then
echo -e "Statistic $stat ($newValue) is \e[0;31m$percent% (+$(( newValue - oldValue )))\e[0m of the old value $oldValue" >&2
else
echo -e "Statistic $stat ($newValue) is \e[0;32m$percent% (-$(( oldValue - newValue )))\e[0m of the old value $oldValue" >&2
fi
(( different++ )) || true
fi
done
echo "$different stats differ between the current tree and $compareTo"
echo ""
}
# Create a fairly populated tree
touch f{0..5}
mkdir d{0..5}
mkdir e{0..5}
touch d{0..5}/f{0..5}
mkdir -p d{0..5}/d{0..5}
mkdir -p e{0..5}/e{0..5}
touch d{0..5}/d{0..5}/f{0..5}
mkdir -p d{0..5}/d{0..5}/d{0..5}
mkdir -p e{0..5}/e{0..5}/e{0..5}
touch d{0..5}/d{0..5}/d{0..5}/f{0..5}
mkdir -p d{0..5}/d{0..5}/d{0..5}/d{0..5}
mkdir -p e{0..5}/e{0..5}/e{0..5}/e{0..5}
touch d{0..5}/d{0..5}/d{0..5}/d{0..5}/f{0..5}
bench 'toSource { root = ./.; fileset = ./.; }'
rm -rf -- *
touch {0..1000}
bench 'toSource { root = ./.; fileset = unions (mapAttrsToList (name: value: ./. + "/${name}") (builtins.readDir ./.)); }'
rm -rf -- *

1008
lib/fileset/default.nix Normal file
View File

File diff suppressed because it is too large Load Diff

980
lib/fileset/internal.nix Normal file
View File

@@ -0,0 +1,980 @@
{
lib ? import ../.,
}:
let
inherit (builtins)
isAttrs
isPath
isString
nixVersion
pathExists
readDir
split
trace
typeOf
fetchGit
;
inherit (lib.attrsets)
attrNames
attrValues
mapAttrs
mapAttrsToList
optionalAttrs
zipAttrsWith
;
inherit (lib.filesystem)
pathType
;
inherit (lib.lists)
all
commonPrefix
concatLists
elemAt
filter
findFirst
findFirstIndex
foldl'
head
length
sublist
tail
;
inherit (lib.path)
append
splitRoot
hasStorePathPrefix
splitStorePath
;
inherit (lib.path.subpath)
components
join
;
inherit (lib.strings)
isStringLike
concatStringsSep
substring
stringLength
hasSuffix
versionAtLeast
;
inherit (lib.trivial)
inPureEvalMode
;
in
# Rare case of justified usage of rec:
# - This file is internal, so the return value doesn't matter, no need to make things overridable
# - The functions depend on each other
# - We want to expose all of these functions for easy testing
rec {
# If you change the internal representation, make sure to:
# - Increment this version
# - Add an additional migration function below
# - Update the description of the internal representation in ./README.md
_currentVersion = 3;
# Migrations between versions. The 0th element converts from v0 to v1, and so on
migrations = [
# Convert v0 into v1: Add the _internalBase{Root,Components} attributes
(
filesetV0:
let
parts = splitRoot filesetV0._internalBase;
in
filesetV0
// {
_internalVersion = 1;
_internalBaseRoot = parts.root;
_internalBaseComponents = components parts.subpath;
}
)
# Convert v1 into v2: filesetTree's can now also omit attributes to signal paths not being included
(
filesetV1:
# This change is backwards compatible (but not forwards compatible, so we still need a new version)
filesetV1
// {
_internalVersion = 2;
}
)
# Convert v2 into v3: filesetTree's now have a representation for an empty file set without a base path
(
filesetV2:
filesetV2
// {
# All v1 file sets are not the new empty file set
_internalIsEmptyWithoutBase = false;
_internalVersion = 3;
}
)
];
_noEvalMessage = ''
lib.fileset: Directly evaluating a file set is not supported.
To turn it into a usable source, use `lib.fileset.toSource`.
To pretty-print the contents, use `lib.fileset.trace` or `lib.fileset.traceVal`.'';
# The empty file set without a base path
_emptyWithoutBase = {
_type = "fileset";
_internalVersion = _currentVersion;
# The one and only!
_internalIsEmptyWithoutBase = true;
# Due to alphabetical ordering, this is evaluated last,
# which makes the nix repl output nicer than if it would be ordered first.
# It also allows evaluating it strictly up to this error, which could be useful
_noEval = throw _noEvalMessage;
};
# Create a fileset, see ./README.md#fileset
# Type: path -> filesetTree -> fileset
_create =
base: tree:
let
# Decompose the base into its components
# See ../path/README.md for why we're not just using `toString`
parts = splitRoot base;
in
{
_type = "fileset";
_internalVersion = _currentVersion;
_internalIsEmptyWithoutBase = false;
_internalBase = base;
_internalBaseRoot = parts.root;
_internalBaseComponents = components parts.subpath;
_internalTree = tree;
# Due to alphabetical ordering, this is evaluated last,
# which makes the nix repl output nicer than if it would be ordered first.
# It also allows evaluating it strictly up to this error, which could be useful
_noEval = throw _noEvalMessage;
};
# Coerce a value to a fileset, erroring when the value cannot be coerced.
# The string gives the context for error messages.
# Type: String -> (fileset | Path) -> fileset
_coerce =
context: value:
if value._type or "" == "fileset" then
if value._internalVersion > _currentVersion then
throw ''
${context} is a file set created from a future version of the file set library with a different internal representation:
- Internal version of the file set: ${toString value._internalVersion}
- Internal version of the library: ${toString _currentVersion}
Make sure to update your Nixpkgs to have a newer version of `lib.fileset`.''
else if value._internalVersion < _currentVersion then
let
# Get all the migration functions necessary to convert from the old to the current version
migrationsToApply = sublist value._internalVersion (
_currentVersion - value._internalVersion
) migrations;
in
foldl' (value: migration: migration value) value migrationsToApply
else
value
else if !isPath value then
if value ? _isLibCleanSourceWith then
throw ''
${context} is a `lib.sources`-based value, but it should be a file set or a path instead.
To convert a `lib.sources`-based value to a file set you can use `lib.fileset.fromSource`.
Note that this only works for sources created from paths.''
else if isStringLike value then
throw ''
${context} ("${toString value}") is a string-like value, but it should be a file set or a path instead.
Paths represented as strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.''
else
throw ''${context} is of type ${typeOf value}, but it should be a file set or a path instead.''
else if !pathExists value then
throw ''
${context} (${toString value}) is a path that does not exist.
To create a file set from a path that may not exist, use `lib.fileset.maybeMissing`.''
else
_singleton value;
# Coerce many values to filesets, erroring when any value cannot be coerced,
# or if the filesystem root of the values doesn't match.
# Type: String -> [ { context :: String, value :: fileset | Path } ] -> [ fileset ]
_coerceMany =
functionContext: list:
let
filesets = map ({ context, value }: _coerce "${functionContext}: ${context}" value) list;
# Find the first value with a base, there may be none!
firstWithBase = findFirst (fileset: !fileset._internalIsEmptyWithoutBase) null filesets;
# This value is only accessed if first != null
firstBaseRoot = firstWithBase._internalBaseRoot;
# Finds the first element with a filesystem root different than the first element, if any
differentIndex = findFirstIndex (
fileset:
# The empty value without a base doesn't have a base path
!fileset._internalIsEmptyWithoutBase && firstBaseRoot != fileset._internalBaseRoot
) null filesets;
in
# Only evaluates `differentIndex` if there are any elements with a base
if firstWithBase != null && differentIndex != null then
throw ''
${functionContext}: Filesystem roots are not the same:
${(head list).context}: Filesystem root is "${toString firstBaseRoot}"
${(elemAt list differentIndex).context}: Filesystem root is "${toString (elemAt filesets differentIndex)._internalBaseRoot}"
Different filesystem roots are not supported.''
else
filesets;
# Create a file set from a path.
# Type: Path -> fileset
_singleton =
path:
let
type = pathType path;
in
if type == "directory" then
_create path type
else
# This turns a file path ./default.nix into a fileset with
# - _internalBase: ./.
# - _internalTree: {
# "default.nix" = <type>;
# }
# See ./README.md#single-files
_create (dirOf path) {
${baseNameOf path} = type;
};
# Expand a directory representation to an equivalent one in attribute set form.
# All directory entries are included in the result.
# Type: Path -> filesetTree -> { <name> = filesetTree; }
_directoryEntries =
path: value:
if value == "directory" then
readDir path
else
# Set all entries not present to null
mapAttrs (name: value: null) (readDir path) // value;
/**
A normalisation of a filesetTree suitable filtering with `builtins.path`:
- Replace all directories that have no files with `null`.
This removes directories that would be empty
- Replace all directories with all files with `"directory"`.
This speeds up the source filter function
Note that this function is strict, it evaluates the entire tree
# Inputs
`path`
: 1\. Function argument
`tree`
: 2\. Function argument
# Type
```
Path -> filesetTree -> filesetTree
```
*/
_normaliseTreeFilter =
path: tree:
if tree == "directory" || isAttrs tree then
let
entries = _directoryEntries path tree;
normalisedSubtrees = mapAttrs (name: _normaliseTreeFilter (path + "/${name}")) entries;
subtreeValues = attrValues normalisedSubtrees;
in
# This triggers either when all files in a directory are filtered out
# Or when the directory doesn't contain any files at all
if all isNull subtreeValues then
null
# Triggers when we have the same as a `readDir path`, so we can turn it back into an equivalent "directory".
else if all isString subtreeValues then
"directory"
else
normalisedSubtrees
else
tree;
/**
A minimal normalisation of a filesetTree, intended for pretty-printing:
- If all children of a path are recursively included or empty directories, the path itself is also recursively included
- If all children of a path are fully excluded or empty directories, the path itself is an empty directory
- Other empty directories are represented with the special "emptyDir" string
While these could be replaced with `null`, that would take another mapAttrs
Note that this function is partially lazy.
# Inputs
`path`
: 1\. Function argument
`tree`
: 2\. Function argument
# Type
```
Path -> filesetTree -> filesetTree (with "emptyDir"'s)
```
*/
_normaliseTreeMinimal =
path: tree:
if tree == "directory" || isAttrs tree then
let
entries = _directoryEntries path tree;
normalisedSubtrees = mapAttrs (name: _normaliseTreeMinimal (path + "/${name}")) entries;
subtreeValues = attrValues normalisedSubtrees;
in
# If there are no entries, or all entries are empty directories, return "emptyDir".
# After this branch we know that there's at least one file
if all (value: value == "emptyDir") subtreeValues then
"emptyDir"
# If all subtrees are fully included or empty directories
# (both of which are coincidentally represented as strings), return "directory".
# This takes advantage of the fact that empty directories can be represented as included directories.
# Note that the tree == "directory" check allows avoiding recursion
else if tree == "directory" || all (value: isString value) subtreeValues then
"directory"
# If all subtrees are fully excluded or empty directories, return null.
# This takes advantage of the fact that empty directories can be represented as excluded directories
else if all (value: isNull value || value == "emptyDir") subtreeValues then
null
# Mix of included and excluded entries
else
normalisedSubtrees
else
tree;
# Trace a filesetTree in a pretty way when the resulting value is evaluated.
# This can handle both normal filesetTree's, and ones returned from _normaliseTreeMinimal
# Type: Path -> filesetTree (with "emptyDir"'s) -> Null
_printMinimalTree =
base: tree:
let
treeSuffix =
tree:
if isAttrs tree then
""
else if tree == "directory" then
" (all files in directory)"
else
# This does "leak" the file type strings of the internal representation,
# but this is the main reason these file type strings even are in the representation!
# TODO: Consider removing that information from the internal representation for performance.
# The file types can still be printed by querying them only during tracing
" (${tree})";
# Only for attribute set trees
traceTreeAttrs =
prevLine: indent: tree:
foldl' (
prevLine: name:
let
subtree = tree.${name};
# Evaluating this prints the line for this subtree
thisLine = trace "${indent}- ${name}${treeSuffix subtree}" prevLine;
in
if subtree == null || subtree == "emptyDir" then
# Don't print anything at all if this subtree is empty
prevLine
else if isAttrs subtree then
# A directory with explicit entries
# Do print this node, but also recurse
traceTreeAttrs thisLine "${indent} " subtree
else
# Either a file, or a recursively included directory
# Do print this node but no further recursion needed
thisLine
) prevLine (attrNames tree);
# Evaluating this will print the first line
firstLine =
if tree == null || tree == "emptyDir" then
trace "(empty)" null
else
trace "${toString base}${treeSuffix tree}" null;
in
if isAttrs tree then traceTreeAttrs firstLine "" tree else firstLine;
# Pretty-print a file set in a pretty way when the resulting value is evaluated
# Type: fileset -> Null
_printFileset =
fileset:
if fileset._internalIsEmptyWithoutBase then
trace "(empty)" null
else
_printMinimalTree fileset._internalBase (
_normaliseTreeMinimal fileset._internalBase fileset._internalTree
);
# Turn a fileset into a source filter function suitable for `builtins.path`
# Only directories recursively containing at least one files are recursed into
# Type: fileset -> (String -> String -> Bool)
_toSourceFilter =
fileset:
let
# Simplify the tree, necessary to make sure all empty directories are null
# which has the effect that they aren't included in the result
tree = _normaliseTreeFilter fileset._internalBase fileset._internalTree;
# The base path as a string with a single trailing slash
baseString =
if fileset._internalBaseComponents == [ ] then
# Need to handle the filesystem root specially
"/"
else
"/" + concatStringsSep "/" fileset._internalBaseComponents + "/";
baseLength = stringLength baseString;
# Check whether a list of path components under the base path exists in the tree.
# This function is called often, so it should be fast.
# Type: [ String ] -> Bool
inTree =
components:
let
recurse =
index: localTree:
if isAttrs localTree then
# We have an attribute set, meaning this is a directory with at least one file
if index >= length components then
# The path may have no more components though, meaning the filter is running on the directory itself,
# so we always include it, again because there's at least one file in it.
true
else
# If we do have more components, the filter runs on some entry inside this directory, so we need to recurse
# We do +2 because builtins.split is an interleaved list of the inbetweens and the matches
recurse (index + 2) localTree.${elemAt components index}
else
# If it's not an attribute set it can only be either null (in which case it's not included)
# or a string ("directory" or "regular", etc.) in which case it's included
localTree != null;
in
recurse 0 tree;
# Filter suited when there's no files
empty = _: _: false;
# Filter suited when there's some files
# This can't be used for when there's no files, because the base directory is always included
nonEmpty =
path: type:
let
# Add a slash to the path string, turning "/foo" to "/foo/",
# making sure to not have any false prefix matches below.
# Note that this would produce "//" for "/",
# but builtins.path doesn't call the filter function on the `path` argument itself,
# meaning this function can never receive "/" as an argument
pathSlash = path + "/";
in
(
# Same as `hasPrefix pathSlash baseString`, but more efficient.
# With base /foo/bar we need to include /foo:
# hasPrefix "/foo/" "/foo/bar/"
if substring 0 (stringLength pathSlash) baseString == pathSlash then
true
# Same as `! hasPrefix baseString pathSlash`, but more efficient.
# With base /foo/bar we need to exclude /baz
# ! hasPrefix "/baz/" "/foo/bar/"
else if substring 0 baseLength pathSlash != baseString then
false
else
# Same as `removePrefix baseString path`, but more efficient.
# From the above code we know that hasPrefix baseString pathSlash holds, so this is safe.
# We don't use pathSlash here because we only needed the trailing slash for the prefix matching.
# With base /foo and path /foo/bar/baz this gives
# inTree (split "/" (removePrefix "/foo/" "/foo/bar/baz"))
# == inTree (split "/" "bar/baz")
# == inTree [ "bar" "baz" ]
inTree (split "/" (substring baseLength (-1) path))
)
# This is a way have an additional check in case the above is true without any significant performance cost
&& (
# This relies on the fact that Nix only distinguishes path types "directory", "regular", "symlink" and "unknown",
# so everything except "unknown" is allowed, seems reasonable to rely on that
type != "unknown"
|| throw ''
lib.fileset.toSource: `fileset` contains a file that cannot be added to the store: ${path}
This file is neither a regular file nor a symlink, the only file types supported by the Nix store.
Therefore the file set cannot be added to the Nix store as is. Make sure to not include that file to avoid this error.''
);
in
# Special case because the code below assumes that the _internalBase is always included in the result
# which shouldn't be done when we have no files at all in the base
# This also forces the tree before returning the filter, leads to earlier error messages
if fileset._internalIsEmptyWithoutBase || tree == null then empty else nonEmpty;
# Turn a builtins.filterSource-based source filter on a root path into a file set
# containing only files included by the filter.
# The filter is lazily called as necessary to determine whether paths are included
# Type: Path -> (String -> String -> Bool) -> fileset
_fromSourceFilter =
root: sourceFilter:
let
# During the recursion we need to track both:
# - The path value such that we can safely call `readDir` on it
# - The path string value such that we can correctly call the `filter` with it
#
# While we could just recurse with the path value,
# this would then require converting it to a path string for every path,
# which is a fairly expensive operation
# Create a file set from a directory entry
fromDirEntry =
path: pathString: type:
# The filter needs to run on the path as a string
if !sourceFilter pathString type then
null
else if type == "directory" then
fromDir path pathString
else
type;
# Create a file set from a directory
fromDir =
path: pathString:
mapAttrs
# This looks a bit funny, but we need both the path-based and the path string-based values
(name: fromDirEntry (path + "/${name}") (pathString + "/${name}"))
# We need to readDir on the path value, because reading on a path string
# would be unspecified if there are multiple filesystem roots
(readDir path);
rootPathType = pathType root;
# We need to convert the path to a string to imitate what builtins.path calls the filter function with.
# We don't want to rely on `toString` for this though because it's not very well defined, see ../path/README.md
# So instead we use `lib.path.splitRoot` to safely deconstruct the path into its filesystem root and subpath
# We don't need the filesystem root though, builtins.path doesn't expose that in any way to the filter.
# So we only need the components, which we then turn into a string as one would expect.
rootString = "/" + concatStringsSep "/" (components (splitRoot root).subpath);
in
if rootPathType == "directory" then
# We imitate builtins.path not calling the filter on the root path
_create root (fromDir root rootString)
else
# Direct files are always included by builtins.path without calling the filter
# But we need to lift up the base path to its parent to satisfy the base path invariant
_create (dirOf root) {
${baseNameOf root} = rootPathType;
};
# Turns a file set into the list of file paths it includes.
# Type: fileset -> [ Path ]
_toList =
fileset:
let
recurse =
path: tree:
if isAttrs tree then
concatLists (mapAttrsToList (name: value: recurse (path + "/${name}") value) tree)
else if tree == "directory" then
recurse path (readDir path)
else if tree == null then
[ ]
else
[ path ];
in
if fileset._internalIsEmptyWithoutBase then
[ ]
else
recurse fileset._internalBase fileset._internalTree;
# Transforms the filesetTree of a file set to a shorter base path, e.g.
# _shortenTreeBase [ "foo" ] (_create /foo/bar null)
# => { bar = null; }
_shortenTreeBase =
targetBaseComponents: fileset:
let
recurse =
index:
# If we haven't reached the required depth yet
if index < length fileset._internalBaseComponents then
# Create an attribute set and recurse as the value, this can be lazily evaluated this way
{ ${elemAt fileset._internalBaseComponents index} = recurse (index + 1); }
else
# Otherwise we reached the appropriate depth, here's the original tree
fileset._internalTree;
in
recurse (length targetBaseComponents);
# Transforms the filesetTree of a file set to a longer base path, e.g.
# _lengthenTreeBase [ "foo" "bar" ] (_create /foo { bar.baz = "regular"; })
# => { baz = "regular"; }
_lengthenTreeBase =
targetBaseComponents: fileset:
let
recurse =
index: tree:
# If the filesetTree is an attribute set and we haven't reached the required depth yet
if isAttrs tree && index < length targetBaseComponents then
# Recurse with the tree under the right component (which might not exist)
recurse (index + 1) (tree.${elemAt targetBaseComponents index} or null)
else
# For all values here we can just return the tree itself:
# tree == null -> the result is also null, everything is excluded
# tree == "directory" -> the result is also "directory",
# because the base path is always a directory and everything is included
# isAttrs tree -> the result is `tree`
# because we don't need to recurse any more since `index == length longestBaseComponents`
tree;
in
recurse (length fileset._internalBaseComponents) fileset._internalTree;
# Computes the union of a list of filesets.
# The filesets must already be coerced and validated to be in the same filesystem root
# Type: [ Fileset ] -> Fileset
_unionMany =
filesets:
let
# All filesets that have a base, aka not the ones that are the empty value without a base
filesetsWithBase = filter (fileset: !fileset._internalIsEmptyWithoutBase) filesets;
# The first fileset that has a base.
# This value is only accessed if there are at all.
firstWithBase = head filesetsWithBase;
# To be able to union filesetTree's together, they need to have the same base path.
# Base paths can be unioned by taking their common prefix,
# e.g. such that `union /foo/bar /foo/baz` has the base path `/foo`
# A list of path components common to all base paths.
# Note that commonPrefix can only be fully evaluated,
# so this cannot cause a stack overflow due to a build-up of unevaluated thunks.
commonBaseComponents =
foldl' (components: el: commonPrefix components el._internalBaseComponents)
firstWithBase._internalBaseComponents
# We could also not do the `tail` here to avoid a list allocation,
# but then we'd have to pay for a potentially expensive
# but unnecessary `commonPrefix` call
(tail filesetsWithBase);
# The common base path assembled from a filesystem root and the common components
commonBase = append firstWithBase._internalBaseRoot (join commonBaseComponents);
# A list of filesetTree's that all have the same base path
# This is achieved by nesting the trees into the components they have over the common base path
# E.g. `union /foo/bar /foo/baz` has the base path /foo
# So the tree under `/foo/bar` gets nested under `{ bar = ...; ... }`,
# while the tree under `/foo/baz` gets nested under `{ baz = ...; ... }`
# Therefore allowing combined operations over them.
trees = map (_shortenTreeBase commonBaseComponents) filesetsWithBase;
# Folds all trees together into a single one using _unionTree
# We do not use a fold here because it would cause a thunk build-up
# which could cause a stack overflow for a large number of trees
resultTree = _unionTrees trees;
in
# If there's no values with a base, we have no files
if filesetsWithBase == [ ] then _emptyWithoutBase else _create commonBase resultTree;
# The union of multiple filesetTree's with the same base path.
# Later elements are only evaluated if necessary.
# Type: [ filesetTree ] -> filesetTree
_unionTrees =
trees:
let
stringIndex = findFirstIndex isString null trees;
withoutNull = filter (tree: tree != null) trees;
in
if stringIndex != null then
# If there's a string, it's always a fully included tree (dir or file),
# no need to look at other elements
elemAt trees stringIndex
else if withoutNull == [ ] then
# If all trees are null, then the resulting tree is also null
null
else
# The non-null elements have to be attribute sets representing partial trees
# We need to recurse into those
zipAttrsWith (name: _unionTrees) withoutNull;
# Computes the intersection of a list of filesets.
# The filesets must already be coerced and validated to be in the same filesystem root
# Type: Fileset -> Fileset -> Fileset
_intersection =
fileset1: fileset2:
let
# The common base components prefix, e.g.
# (/foo/bar, /foo/bar/baz) -> /foo/bar
# (/foo/bar, /foo/baz) -> /foo
commonBaseComponentsLength =
# TODO: Have a `lib.lists.commonPrefixLength` function such that we don't need the list allocation from commonPrefix here
length (commonPrefix fileset1._internalBaseComponents fileset2._internalBaseComponents);
# To be able to intersect filesetTree's together, they need to have the same base path.
# Base paths can be intersected by taking the longest one (if any)
# The fileset with the longest base, if any, e.g.
# (/foo/bar, /foo/bar/baz) -> /foo/bar/baz
# (/foo/bar, /foo/baz) -> null
longestBaseFileset =
if commonBaseComponentsLength == length fileset1._internalBaseComponents then
# The common prefix is the same as the first path, so the second path is equal or longer
fileset2
else if commonBaseComponentsLength == length fileset2._internalBaseComponents then
# The common prefix is the same as the second path, so the first path is longer
fileset1
else
# The common prefix is neither the first nor the second path
# This means there's no overlap between the two sets
null;
# Whether the result should be the empty value without a base
resultIsEmptyWithoutBase =
# If either fileset is the empty fileset without a base, the intersection is too
fileset1._internalIsEmptyWithoutBase
|| fileset2._internalIsEmptyWithoutBase
# If there is no overlap between the base paths
|| longestBaseFileset == null;
# Lengthen each fileset's tree to the longest base prefix
tree1 = _lengthenTreeBase longestBaseFileset._internalBaseComponents fileset1;
tree2 = _lengthenTreeBase longestBaseFileset._internalBaseComponents fileset2;
# With two filesetTree's with the same base, we can compute their intersection
resultTree = _intersectTree tree1 tree2;
in
if resultIsEmptyWithoutBase then
_emptyWithoutBase
else
_create longestBaseFileset._internalBase resultTree;
# The intersection of two filesetTree's with the same base path
# The second element is only evaluated as much as necessary.
# Type: filesetTree -> filesetTree -> filesetTree
_intersectTree =
lhs: rhs:
if isAttrs lhs && isAttrs rhs then
# Both sides are attribute sets, we can recurse for the attributes existing on both sides
mapAttrs (name: _intersectTree lhs.${name}) (builtins.intersectAttrs lhs rhs)
else if lhs == null || isString rhs then
# If the lhs is null, the result should also be null
# And if the rhs is the identity element
# (a string, aka it includes everything), then it's also the lhs
lhs
else
# In all other cases it's the rhs
rhs;
# Compute the set difference between two file sets.
# The filesets must already be coerced and validated to be in the same filesystem root.
# Type: Fileset -> Fileset -> Fileset
_difference =
positive: negative:
let
# The common base components prefix, e.g.
# (/foo/bar, /foo/bar/baz) -> /foo/bar
# (/foo/bar, /foo/baz) -> /foo
commonBaseComponentsLength =
# TODO: Have a `lib.lists.commonPrefixLength` function such that we don't need the list allocation from commonPrefix here
length (commonPrefix positive._internalBaseComponents negative._internalBaseComponents);
# We need filesetTree's with the same base to be able to compute the difference between them
# This here is the filesetTree from the negative file set, but for a base path that matches the positive file set.
# Examples:
# For `difference /foo /foo/bar`, `negativeTreeWithPositiveBase = { bar = "directory"; }`
# because under the base path of `/foo`, only `bar` from the negative file set is included
# For `difference /foo/bar /foo`, `negativeTreeWithPositiveBase = "directory"`
# because under the base path of `/foo/bar`, everything from the negative file set is included
# For `difference /foo /bar`, `negativeTreeWithPositiveBase = null`
# because under the base path of `/foo`, nothing from the negative file set is included
negativeTreeWithPositiveBase =
if commonBaseComponentsLength == length positive._internalBaseComponents then
# The common prefix is the same as the positive base path, so the second path is equal or longer.
# We need to _shorten_ the negative filesetTree to the same base path as the positive one
# E.g. for `difference /foo /foo/bar` the common prefix is /foo, equal to the positive file set's base
# So we need to shorten the base of the tree for the negative argument from /foo/bar to just /foo
_shortenTreeBase positive._internalBaseComponents negative
else if commonBaseComponentsLength == length negative._internalBaseComponents then
# The common prefix is the same as the negative base path, so the first path is longer.
# We need to lengthen the negative filesetTree to the same base path as the positive one.
# E.g. for `difference /foo/bar /foo` the common prefix is /foo, equal to the negative file set's base
# So we need to lengthen the base of the tree for the negative argument from /foo to /foo/bar
_lengthenTreeBase positive._internalBaseComponents negative
else
# The common prefix is neither the first nor the second path.
# This means there's no overlap between the two file sets,
# and nothing from the negative argument should get removed from the positive one
# E.g for `difference /foo /bar`, we remove nothing to get the same as `/foo`
null;
resultingTree =
_differenceTree positive._internalBase positive._internalTree
negativeTreeWithPositiveBase;
in
# If the first file set is empty, we can never have any files in the result
if positive._internalIsEmptyWithoutBase then
_emptyWithoutBase
# If the second file set is empty, nothing gets removed, so the result is just the first file set
else if negative._internalIsEmptyWithoutBase then
positive
else
# We use the positive file set base for the result,
# because only files from the positive side may be included,
# which is what base path is for
_create positive._internalBase resultingTree;
# Computes the set difference of two filesetTree's
# Type: Path -> filesetTree -> filesetTree
_differenceTree =
path: lhs: rhs:
# If the lhs doesn't have any files, or the right hand side includes all files
if lhs == null || isString rhs then
# The result will always be empty
null
# If the right hand side has no files
else if rhs == null then
# The result is always the left hand side, because nothing gets removed
lhs
else
# Otherwise we always have two attribute sets to recurse into
mapAttrs (name: lhsValue: _differenceTree (path + "/${name}") lhsValue (rhs.${name} or null)) (
_directoryEntries path lhs
);
# Filters all files in a path based on a predicate
# Type: ({ name, type, ... } -> Bool) -> Path -> FileSet
_fileFilter =
predicate: root:
let
# Check the predicate for a single file
# Type: String -> String -> filesetTree
fromFile =
name: type:
if
predicate {
inherit name type;
hasExt = ext: hasSuffix ".${ext}" name;
# To ensure forwards compatibility with more arguments being added in the future,
# adding an attribute which can't be deconstructed :)
"lib.fileset.fileFilter: The predicate function passed as the first argument must be able to handle extra attributes for future compatibility. If you're using `{ name, file, hasExt }:`, use `{ name, file, hasExt, ... }:` instead." =
null;
}
then
type
else
null;
# Check the predicate for all files in a directory
# Type: Path -> filesetTree
fromDir =
path:
mapAttrs (
name: type: if type == "directory" then fromDir (path + "/${name}") else fromFile name type
) (readDir path);
rootType = pathType root;
in
if rootType == "directory" then
_create root (fromDir root)
else
# Single files are turned into a directory containing that file or nothing.
_create (dirOf root) {
${baseNameOf root} = fromFile (baseNameOf root) rootType;
};
# Mirrors the contents of a Nix store path relative to a local path as a file set.
# Some notes:
# - The store path is read at evaluation time.
# - The store path must not include files that don't exist in the respective local path.
#
# Type: Path -> String -> FileSet
_mirrorStorePath =
localPath: storePath:
let
recurse =
focusedStorePath:
mapAttrs (
name: type: if type == "directory" then recurse (focusedStorePath + "/${name}") else type
) (builtins.readDir focusedStorePath);
in
_create localPath (recurse storePath);
# Create a file set from the files included in the result of a fetchGit call
# Type: String -> String -> Path -> Attrs -> FileSet
_fromFetchGit =
function: argument: path: extraFetchGitAttrs:
let
# The code path for when isStorePath is true
tryStorePath =
if pathExists (path + "/.git") then
# If there is a `.git` directory in the path,
# it means that the path was imported unfiltered into the Nix store.
# This function should throw in such a case, because
# - `fetchGit` doesn't generally work with `.git` directories in store paths
# - Importing the entire path could include Git-tracked files
throw ''
lib.fileset.${function}: The ${argument} (${toString path}) is a store path within a working tree of a Git repository.
This indicates that a source directory was imported into the store using a method such as `import "''${./.}"` or `path:.`.
This function currently does not support such a use case, since it currently relies on `builtins.fetchGit`.
You could make this work by using a fetcher such as `fetchGit` instead of copying the whole repository.
If you can't avoid copying the repo to the store, see https://github.com/NixOS/nix/issues/9292.''
else
# Otherwise we're going to assume that the path was a Git directory originally,
# but it was fetched using a method that already removed files not tracked by Git,
# such as `builtins.fetchGit`, `pkgs.fetchgit` or others.
# So we can just import the path in its entirety.
_singleton path;
# The code path for when isStorePath is false
tryFetchGit =
let
# This imports the files unnecessarily, which currently can't be avoided
# because `builtins.fetchGit` is the only function exposing which files are tracked by Git.
# With the [lazy trees PR](https://github.com/NixOS/nix/pull/6530),
# the unnecessarily import could be avoided.
# However a simpler alternative still would be [a builtins.gitLsFiles](https://github.com/NixOS/nix/issues/2944).
fetchResult = fetchGit (
{
url = path;
shallow = true;
}
// extraFetchGitAttrs
);
in
# We can identify local working directories by checking for .git,
# see https://git-scm.com/docs/gitrepository-layout#_description.
# Note that `builtins.fetchGit` _does_ work for bare repositories (where there's no `.git`),
# even though `git ls-files` wouldn't return any files in that case.
if !pathExists (path + "/.git") then
throw "lib.fileset.${function}: Expected the ${argument} (${toString path}) to point to a local working tree of a Git repository, but it's not."
else
_mirrorStorePath path fetchResult.outPath;
in
if !isPath path then
throw "lib.fileset.${function}: Expected the ${argument} to be a path, but it's a ${typeOf path} instead."
else if pathType path != "directory" then
throw "lib.fileset.${function}: Expected the ${argument} (${toString path}) to be a directory, but it's a file instead."
else if hasStorePathPrefix path then
tryStorePath
else
tryFetchGit;
}

29
lib/fileset/mock-splitRoot.nix Normal file
View File

@@ -0,0 +1,29 @@
# This overlay implements mocking of the lib.path.splitRoot function
# It pretends that the last component named "mock-root" is the root:
#
# splitRoot /foo/mock-root/bar/mock-root/baz
# => {
# root = /foo/mock-root/bar/mock-root;
# subpath = "./baz";
# }
self: super: {
path = super.path // {
splitRoot =
path:
let
parts = super.path.splitRoot path;
components = self.path.subpath.components parts.subpath;
count = self.length components;
rootIndex =
count
- self.lists.findFirstIndex (component: component == "mock-root") (self.length components) (
self.reverseList components
);
root = self.path.append parts.root (self.path.subpath.join (self.take rootIndex components));
subpath = self.path.subpath.join (self.drop rootIndex components);
in
{
inherit root subpath;
};
};
}

1582
lib/fileset/tests.sh Executable file
View File

File diff suppressed because it is too large Load Diff

498
lib/filesystem.nix Normal file
View File

@@ -0,0 +1,498 @@
/**
Functions for querying information about the filesystem
without copying any files to the Nix store.
*/
{ lib }:
# Tested in lib/tests/filesystem.sh
let
inherit (builtins)
readDir
pathExists
toString
;
inherit (lib.filesystem)
pathIsDirectory
pathIsRegularFile
pathType
packagesFromDirectoryRecursive
;
inherit (lib.strings)
hasSuffix
;
in
{
/**
The type of a path. The path needs to exist and be accessible.
The result is either "directory" for a directory, "regular" for a regular file, "symlink" for a symlink, or "unknown" for anything else.
# Inputs
path
: The path to query
# Type
```
pathType :: Path -> String
```
# Examples
:::{.example}
## `lib.filesystem.pathType` usage example
```nix
pathType /.
=> "directory"
pathType /some/file.nix
=> "regular"
```
:::
*/
pathType =
builtins.readFileType or
# Nix <2.14 compatibility shim
(
path:
if
!pathExists path
# Fail irrecoverably to mimic the historic behavior of this function and
# the new builtins.readFileType
then
abort "lib.filesystem.pathType: Path ${toString path} does not exist."
# The filesystem root is the only path where `dirOf / == /` and
# `baseNameOf /` is not valid. We can detect this and directly return
# "directory", since we know the filesystem root can't be anything else.
else if dirOf path == path then
"directory"
else
(readDir (dirOf path)).${baseNameOf path}
);
/**
Whether a path exists and is a directory.
# Inputs
`path`
: 1\. Function argument
# Type
```
pathIsDirectory :: Path -> Bool
```
# Examples
:::{.example}
## `lib.filesystem.pathIsDirectory` usage example
```nix
pathIsDirectory /.
=> true
pathIsDirectory /this/does/not/exist
=> false
pathIsDirectory /some/file.nix
=> false
```
:::
*/
pathIsDirectory = path: pathExists path && pathType path == "directory";
/**
Whether a path exists and is a regular file, meaning not a symlink or any other special file type.
# Inputs
`path`
: 1\. Function argument
# Type
```
pathIsRegularFile :: Path -> Bool
```
# Examples
:::{.example}
## `lib.filesystem.pathIsRegularFile` usage example
```nix
pathIsRegularFile /.
=> false
pathIsRegularFile /this/does/not/exist
=> false
pathIsRegularFile /some/file.nix
=> true
```
:::
*/
pathIsRegularFile = path: pathExists path && pathType path == "regular";
/**
A map of all haskell packages defined in the given path,
identified by having a cabal file with the same name as the
directory itself.
# Inputs
`root`
: The directory within to search
# Type
```
Path -> Map String Path
```
*/
haskellPathsInDir =
root:
let
# Files in the root
root-files = builtins.attrNames (builtins.readDir root);
# Files with their full paths
root-files-with-paths = map (file: {
name = file;
value = root + "/${file}";
}) root-files;
# Subdirectories of the root with a cabal file.
cabal-subdirs = builtins.filter (
{ name, value }: builtins.pathExists (value + "/${name}.cabal")
) root-files-with-paths;
in
builtins.listToAttrs cabal-subdirs;
/**
Find the first directory containing a file matching 'pattern'
upward from a given 'file'.
Returns 'null' if no directories contain a file matching 'pattern'.
# Inputs
`pattern`
: The pattern to search for
`file`
: The file to start searching upward from
# Type
```
RegExp -> Path -> Nullable { path : Path; matches : [ MatchResults ]; }
```
*/
locateDominatingFile =
pattern: file:
let
go =
path:
let
files = builtins.attrNames (builtins.readDir path);
matches = builtins.filter (match: match != null) (map (builtins.match pattern) files);
in
if builtins.length matches != 0 then
{ inherit path matches; }
else if path == /. then
null
else
go (dirOf path);
parent = dirOf file;
isDir =
let
base = baseNameOf file;
type = (builtins.readDir parent).${base} or null;
in
file == /. || type == "directory";
in
go (if isDir then file else parent);
/**
Given a directory, return a flattened list of all files within it recursively.
# Inputs
`dir`
: The path to recursively list
# Type
```
Path -> [ Path ]
```
*/
listFilesRecursive =
dir:
lib.flatten (
lib.mapAttrsToList (
name: type:
if type == "directory" then
lib.filesystem.listFilesRecursive (dir + "/${name}")
else
dir + "/${name}"
) (builtins.readDir dir)
);
/**
Transform a directory tree containing package files suitable for
`callPackage` into a matching nested attribute set of derivations.
For a directory tree like this:
```
my-packages
├── a.nix
├── b.nix
├── c
│ ├── my-extra-feature.patch
│ ├── package.nix
│ └── support-definitions.nix
└── my-namespace
├── d.nix
├── e.nix
└── f
└── package.nix
```
`packagesFromDirectoryRecursive` will produce an attribute set like this:
```nix
# packagesFromDirectoryRecursive {
# callPackage = pkgs.callPackage;
# directory = ./my-packages;
# }
{
a = pkgs.callPackage ./my-packages/a.nix { };
b = pkgs.callPackage ./my-packages/b.nix { };
c = pkgs.callPackage ./my-packages/c/package.nix { };
my-namespace = {
d = pkgs.callPackage ./my-packages/my-namespace/d.nix { };
e = pkgs.callPackage ./my-packages/my-namespace/e.nix { };
f = pkgs.callPackage ./my-packages/my-namespace/f/package.nix { };
};
}
```
In particular:
- If the input directory contains a `package.nix` file, then
`callPackage <directory>/package.nix { }` is returned.
- Otherwise, the input directory's contents are listed and transformed into
an attribute set.
- If a regular file's name has the `.nix` extension, it is turned into attribute
where:
- The attribute name is the file name without the `.nix` extension
- The attribute value is `callPackage <file path> { }`
- Directories are turned into an attribute where:
- The attribute name is the name of the directory
- The attribute value is the result of calling
`packagesFromDirectoryRecursive { ... }` on the directory.
As a result, directories with no `.nix` files (including empty
directories) will be transformed into empty attribute sets.
- Other files are ignored, including symbolic links to directories and to regular `.nix`
files; this is because nixlang code cannot distinguish the type of a link's target.
# Type
```
packagesFromDirectoryRecursive :: {
callPackage :: Path -> {} -> a,
newScope? :: AttrSet -> scope,
directory :: Path,
} -> AttrSet
```
# Inputs
`callPackage`
: The function used to convert a Nix file's path into a leaf of the attribute set.
It is typically the `callPackage` function, taken from either `pkgs` or a new scope corresponding to the `directory`.
`newScope`
: If present, this function is used when recursing into a directory, to generate a new scope.
The arguments are updated with the scope's `callPackage` and `newScope` functions, so packages can require
anything in their scope, or in an ancestor of their scope.
`directory`
: The directory to read package files from.
# Examples
:::{.example}
## Basic use of `lib.packagesFromDirectoryRecursive`
```nix
packagesFromDirectoryRecursive {
inherit (pkgs) callPackage;
directory = ./my-packages;
}
=> { ... }
```
In this case, `callPackage` will only search `pkgs` for a file's input parameters.
In other words, a file cannot refer to another file in the directory in its input parameters.
:::
::::{.example}
## Create a scope for the nix files found in a directory
```nix
packagesFromDirectoryRecursive {
inherit (pkgs) callPackage newScope;
directory = ./my-packages;
}
=> { ... }
```
For example, take the following directory structure:
```
my-packages
├── a.nix → { b }: assert b ? b1; ...
└── b
├── b1.nix → { a }: ...
└── b2.nix
```
Here, `b1.nix` can specify `{ a }` as a parameter, which `callPackage` will resolve as expected.
Likewise, `a.nix` receive an attrset corresponding to the contents of the `b` directory.
:::{.note}
`a.nix` cannot directly take as inputs packages defined in a child directory, such as `b1`.
:::
::::
*/
packagesFromDirectoryRecursive =
let
inherit (lib)
concatMapAttrs
id
makeScope
recurseIntoAttrs
removeSuffix
;
# Generate an attrset corresponding to a given directory.
# This function is outside `packagesFromDirectoryRecursive`'s lambda expression,
# to prevent accidentally using its parameters.
processDir =
{ callPackage, directory, ... }@args:
concatMapAttrs (
name: type:
# for each directory entry
let
path = directory + "/${name}";
in
if type == "directory" then
{
# recurse into directories
"${name}" = packagesFromDirectoryRecursive (
args
// {
directory = path;
}
);
}
else if type == "regular" && hasSuffix ".nix" name then
{
# call .nix files
"${removeSuffix ".nix" name}" = callPackage path { };
}
else if type == "regular" then
{
# ignore non-nix files
}
else
throw ''
lib.filesystem.packagesFromDirectoryRecursive: Unsupported file type ${type} at path ${toString path}
''
) (builtins.readDir directory);
in
{
callPackage,
newScope ? throw "lib.packagesFromDirectoryRecursive: newScope wasn't passed in args",
directory,
}@args:
let
defaultPath = directory + "/package.nix";
in
if pathExists defaultPath then
# if `${directory}/package.nix` exists, call it directly
callPackage defaultPath { }
else if args ? newScope then
# Create a new scope and mark it `recurseForDerivations`.
# This lets the packages refer to each other.
# See:
# [lib.makeScope](https://nixos.org/manual/nixpkgs/unstable/#function-library-lib.customisation.makeScope) and
# [lib.recurseIntoAttrs](https://nixos.org/manual/nixpkgs/unstable/#function-library-lib.customisation.makeScope)
recurseIntoAttrs (
makeScope newScope (
self:
# generate the attrset representing the directory, using the new scope's `callPackage` and `newScope`
processDir (
args
// {
inherit (self) callPackage newScope;
}
)
)
)
else
processDir args;
/**
Append `/default.nix` if the passed path is a directory.
# Type
```
resolveDefaultNix :: (Path | String) -> (Path | String)
```
# Inputs
A single argument which can be a [path](https://nix.dev/manual/nix/stable/language/types#type-path) value or a string containing an absolute path.
# Output
If the input refers to a directory that exists, the output is that same path with `/default.nix` appended.
Furthermore, if the input is a string that ends with `/`, `default.nix` is appended to it.
Otherwise, the input is returned unchanged.
# Examples
:::{.example}
## `lib.filesystem.resolveDefaultNix` usage example
This expression checks whether `a` and `b` refer to the same locally available Nix file path.
```nix
resolveDefaultNix a == resolveDefaultNix b
```
For instance, if `a` is `/some/dir` and `b` is `/some/dir/default.nix`, and `/some/dir/` exists, the expression evaluates to `true`, despite `a` and `b` being different references to the same Nix file.
*/
resolveDefaultNix =
v:
if pathIsDirectory v then
v + "/default.nix"
else if lib.isString v && hasSuffix "/" v then
# A path ending in `/` can only refer to a directory, so we take the hint, even if we can't verify the validity of the path's `/` assertion.
# A `/` is already present, so we don't add another one.
v + "default.nix"
else
v;
}

525
lib/fixed-points.nix Normal file
View File

@@ -0,0 +1,525 @@
{ lib, ... }:
rec {
/**
`fix f` computes the fixed point of the given function `f`. In other words, the return value is `x` in `x = f x`.
`f` must be a lazy function.
This means that `x` must be a value that can be partially evaluated,
such as an attribute set, a list, or a function.
This way, `f` can use one part of `x` to compute another part.
**Relation to syntactic recursion**
This section explains `fix` by refactoring from syntactic recursion to a call of `fix` instead.
For context, Nix lets you define attributes in terms of other attributes syntactically using the [`rec { }` syntax](https://nixos.org/manual/nix/stable/language/constructs.html#recursive-sets).
```nix
nix-repl> rec {
foo = "foo";
bar = "bar";
foobar = foo + bar;
}
{ bar = "bar"; foo = "foo"; foobar = "foobar"; }
```
This is convenient when constructing a value to pass to a function for example,
but an equivalent effect can be achieved with the `let` binding syntax:
```nix
nix-repl> let self = {
foo = "foo";
bar = "bar";
foobar = self.foo + self.bar;
}; in self
{ bar = "bar"; foo = "foo"; foobar = "foobar"; }
```
But in general you can get more reuse out of `let` bindings by refactoring them to a function.
```nix
nix-repl> f = self: {
foo = "foo";
bar = "bar";
foobar = self.foo + self.bar;
}
```
This is where `fix` comes in, it contains the syntactic recursion that's not in `f` anymore.
```nix
nix-repl> fix = f:
let self = f self; in self;
```
By applying `fix` we get the final result.
```nix
nix-repl> fix f
{ bar = "bar"; foo = "foo"; foobar = "foobar"; }
```
Such a refactored `f` using `fix` is not useful by itself.
See [`extends`](#function-library-lib.fixedPoints.extends) for an example use case.
There `self` is also often called `final`.
# Inputs
`f`
: 1\. Function argument
# Type
```
fix :: (a -> a) -> a
```
# Examples
:::{.example}
## `lib.fixedPoints.fix` usage example
```nix
fix (self: { foo = "foo"; bar = "bar"; foobar = self.foo + self.bar; })
=> { bar = "bar"; foo = "foo"; foobar = "foobar"; }
fix (self: [ 1 2 (elemAt self 0 + elemAt self 1) ])
=> [ 1 2 3 ]
```
:::
*/
fix =
f:
let
x = f x;
in
x;
/**
A variant of `fix` that records the original recursive attribute set in the
result, in an attribute named `__unfix__`.
This is useful in combination with the `extends` function to
implement deep overriding.
# Inputs
`f`
: 1\. Function argument
*/
fix' =
f:
let
x = f x // {
__unfix__ = f;
};
in
x;
/**
Returns the fixpoint that `f` converges to when called iteratively, starting
with the input `x`.
```
nix-repl> converge (x: x / 2) 16
0
```
# Inputs
`f`
: 1\. Function argument
`x`
: 2\. Function argument
# Type
```
(a -> a) -> a -> a
```
*/
converge =
f: x:
let
x' = f x;
in
if x' == x then x else converge f x';
/**
Extend a function using an overlay.
Overlays allow modifying and extending fixed-point functions, specifically ones returning attribute sets.
A fixed-point function is a function which is intended to be evaluated by passing the result of itself as the argument.
This is possible due to Nix's lazy evaluation.
A fixed-point function returning an attribute set has the form
```nix
final: {
# attributes
}
```
where `final` refers to the lazily evaluated attribute set returned by the fixed-point function.
An overlay to such a fixed-point function has the form
```nix
final: prev: {
# attributes
}
```
where `prev` refers to the result of the original function to `final`, and `final` is the result of the composition of the overlay and the original function.
Applying an overlay is done with `extends`:
```nix
let
f = final: {
# attributes
};
overlay = final: prev: {
# attributes
};
in extends overlay f;
```
To get the value of `final`, use `lib.fix`:
```nix
let
f = final: {
# attributes
};
overlay = final: prev: {
# attributes
};
g = extends overlay f;
in fix g
```
:::{.note}
The argument to the given fixed-point function after applying an overlay will *not* refer to its own return value, but rather to the value after evaluating the overlay function.
The given fixed-point function is called with a separate argument than if it was evaluated with `lib.fix`.
:::
:::{.example}
# Extend a fixed-point function with an overlay
Define a fixed-point function `f` that expects its own output as the argument `final`:
```nix-repl
f = final: {
# Constant value a
a = 1;
# b depends on the final value of a, available as final.a
b = final.a + 2;
}
```
Evaluate this using [`lib.fix`](#function-library-lib.fixedPoints.fix) to get the final result:
```nix-repl
fix f
=> { a = 1; b = 3; }
```
An overlay represents a modification or extension of such a fixed-point function.
Here's an example of an overlay:
```nix-repl
overlay = final: prev: {
# Modify the previous value of a, available as prev.a
a = prev.a + 10;
# Extend the attribute set with c, letting it depend on the final values of a and b
c = final.a + final.b;
}
```
Use `extends overlay f` to apply the overlay to the fixed-point function `f`.
This produces a new fixed-point function `g` with the combined behavior of `f` and `overlay`:
```nix-repl
g = extends overlay f
```
The result is a function, so we can't print it directly, but it's the same as:
```nix-repl
g' = final: {
# The constant from f, but changed with the overlay
a = 1 + 10;
# Unchanged from f
b = final.a + 2;
# Extended in the overlay
c = final.a + final.b;
}
```
Evaluate this using [`lib.fix`](#function-library-lib.fixedPoints.fix) again to get the final result:
```nix-repl
fix g
=> { a = 11; b = 13; c = 24; }
```
:::
# Inputs
`overlay`
: The overlay to apply to the fixed-point function
`f`
: The fixed-point function
# Type
```
extends :: (Attrs -> Attrs -> Attrs) # The overlay to apply to the fixed-point function
-> (Attrs -> Attrs) # A fixed-point function
-> (Attrs -> Attrs) # The resulting fixed-point function
```
# Examples
:::{.example}
## `lib.fixedPoints.extends` usage example
```nix
f = final: { a = 1; b = final.a + 2; }
fix f
=> { a = 1; b = 3; }
fix (extends (final: prev: { a = prev.a + 10; }) f)
=> { a = 11; b = 13; }
fix (extends (final: prev: { b = final.a + 5; }) f)
=> { a = 1; b = 6; }
fix (extends (final: prev: { c = final.a + final.b; }) f)
=> { a = 1; b = 3; c = 4; }
```
:::
*/
extends =
overlay: f:
# The result should be thought of as a function, the argument of that function is not an argument to `extends` itself
(
final:
let
prev = f final;
in
prev // overlay final prev
);
/**
Compose two overlay functions and return a single overlay function that combines them.
For more details see: [composeManyExtensions](#function-library-lib.fixedPoints.composeManyExtensions).
*/
composeExtensions =
f: g: final: prev:
let
fApplied = f final prev;
prev' = prev // fApplied;
in
fApplied // g final prev';
/**
Composes a list of [`overlays`](#chap-overlays) and returns a single overlay function that combines them.
:::{.note}
The result is produced by using the update operator `//`.
This means nested values of previous overlays are not merged recursively.
In other words, previously defined attributes are replaced, ignoring the previous value, unless referenced by the overlay; for example `final: prev: { foo = final.foo + 1; }`.
:::
# Inputs
`extensions`
: A list of overlay functions
:::{.note}
The order of the overlays in the list is important.
:::
: Each overlay function takes two arguments, by convention `final` and `prev`, and returns an attribute set.
- `final` is the result of the fixed-point function, with all overlays applied.
- `prev` is the result of the previous overlay function(s).
# Type
```
# Pseudo code
let
# final prev
# ↓ ↓
OverlayFn = { ... } -> { ... } -> { ... };
in
composeManyExtensions :: ListOf OverlayFn -> OverlayFn
```
# Examples
:::{.example}
## `lib.fixedPoints.composeManyExtensions` usage example
```nix
let
# The "original function" that is extended by the overlays.
# Note that it doesn't have prev: as argument since no overlay function precedes it.
original = final: { a = 1; };
# Each overlay function has 'final' and 'prev' as arguments.
overlayA = final: prev: { b = final.c; c = 3; };
overlayB = final: prev: { c = 10; x = prev.c or 5; };
extensions = composeManyExtensions [ overlayA overlayB ];
# Calculate the fixed point of all composed overlays.
fixedpoint = lib.fix (lib.extends extensions original );
in fixedpoint
=>
{
a = 1;
b = 10;
c = 10;
x = 3;
}
```
:::
*/
composeManyExtensions = lib.foldr (x: y: composeExtensions x y) (final: prev: { });
/**
Create an overridable, recursive attribute set. For example:
```
nix-repl> obj = makeExtensible (final: { })
nix-repl> obj
{ __unfix__ = «lambda»; extend = «lambda»; }
nix-repl> obj = obj.extend (final: prev: { foo = "foo"; })
nix-repl> obj
{ __unfix__ = «lambda»; extend = «lambda»; foo = "foo"; }
nix-repl> obj = obj.extend (final: prev: { foo = prev.foo + " + "; bar = "bar"; foobar = final.foo + final.bar; })
nix-repl> obj
{ __unfix__ = «lambda»; bar = "bar"; extend = «lambda»; foo = "foo + "; foobar = "foo + bar"; }
```
*/
makeExtensible = makeExtensibleWithCustomName "extend";
/**
Same as `makeExtensible` but the name of the extending attribute is
customized.
# Inputs
`extenderName`
: 1\. Function argument
`rattrs`
: 2\. Function argument
*/
makeExtensibleWithCustomName =
extenderName: rattrs:
fix' (
self:
(rattrs self)
// {
${extenderName} = f: makeExtensibleWithCustomName extenderName (extends f rattrs);
}
);
/**
Convert to an extending function (overlay).
`toExtension` is the `toFunction` for extending functions (a.k.a. extensions or overlays).
It converts a non-function or a single-argument function to an extending function,
while returning a two-argument function as-is.
That is, it takes a value of the shape `x`, `prev: x`, or `final: prev: x`,
and returns `final: prev: x`, assuming `x` is not a function.
This function takes care of the input to `stdenv.mkDerivation`'s
`overrideAttrs` function.
It bridges the gap between `<pkg>.overrideAttrs`
before and after the overlay-style support.
# Inputs
`f`
: The function or value to convert to an extending function.
# Type
```
toExtension ::
b' -> Any -> Any -> b'
or
toExtension ::
(a -> b') -> Any -> a -> b'
or
toExtension ::
(a -> a -> b) -> a -> a -> b
where b' = ! Callable
Set a = b = b' = AttrSet & ! Callable to make toExtension return an extending function.
```
# Examples
:::{.example}
## `lib.fixedPoints.toExtension` usage example
```nix
fix (final: { a = 0; c = final.a; })
=> { a = 0; c = 0; };
fix (extends (toExtension { a = 1; b = 2; }) (final: { a = 0; c = final.a; }))
=> { a = 1; b = 2; c = 1; };
fix (extends (toExtension (prev: { a = 1; b = prev.a; })) (final: { a = 0; c = final.a; }))
=> { a = 1; b = 0; c = 1; };
fix (extends (toExtension (final: prev: { a = 1; b = prev.a; c = final.a + 1 })) (final: { a = 0; c = final.a; }))
=> { a = 1; b = 0; c = 2; };
```
:::
*/
toExtension =
f:
if lib.isFunction f then
final: prev:
let
fPrev = f prev;
in
if lib.isFunction fPrev then
# f is (final: prev: { ... })
f final prev
else
# f is (prev: { ... })
fPrev
else
# f is not a function; probably { ... }
final: prev: f;
}

21
lib/flake-version-info.nix Normal file
View File

@@ -0,0 +1,21 @@
# This function produces a lib overlay to be used by the nixpkgs
# & nixpkgs/lib flakes to provide meaningful values for
# `lib.trivial.version` et al..
#
# Internal and subject to change, don't use this anywhere else!
# Instead, consider using a public interface, such as this flake here
# in this directory, `lib/`, or use the nixpkgs flake, which applies
# this logic for you in its `lib` output attribute.
self: # from the flake
finalLib: prevLib: # lib overlay
{
trivial = prevLib.trivial // {
versionSuffix = ".${
finalLib.substring 0 8 (self.lastModifiedDate or "19700101")
}.${self.shortRev or "dirty"}";
revisionWithDefault = default: self.rev or default;
};
}

12
lib/flake.nix Normal file
View File

@@ -0,0 +1,12 @@
{
description = "Library of low-level helper functions for nix expressions.";
outputs =
{ self }:
let
lib0 = import ./.;
in
{
lib = lib0.extend (import ./flake-version-info.nix self);
};
}

924
lib/generators.nix Normal file
View File

@@ -0,0 +1,924 @@
/**
Functions that generate widespread file
formats from nix data structures.
They all follow a similar interface:
```nix
generator { config-attrs } data
```
`config-attrs` are “holes” in the generators
with sensible default implementations that
can be overwritten. The default implementations
are mostly generators themselves, called with
their respective default values; they can be reused.
Tests can be found in ./tests/misc.nix
Further Documentation can be found [here](#sec-generators).
*/
{ lib }:
let
inherit (lib)
addErrorContext
assertMsg
attrNames
concatLists
concatMapStringsSep
concatStrings
concatStringsSep
const
elem
escape
filter
flatten
foldl
functionArgs # Note: not the builtin; considers `__functor` in attrsets.
gvariant
hasInfix
head
id
init
isAttrs
isBool
isDerivation
isFloat
isFunction # Note: not the builtin; considers `__functor` in attrsets.
isInt
isList
isPath
isString
last
length
mapAttrs
mapAttrsToList
optionals
recursiveUpdate
replaceStrings
reverseList
splitString
tail
toList
;
inherit (lib.strings)
escapeNixIdentifier
floatToString
match
split
toJSON
typeOf
escapeXML
;
## -- HELPER FUNCTIONS & DEFAULTS --
in
rec {
/**
Convert a value to a sensible default string representation.
The builtin `toString` function has some strange defaults,
suitable for bash scripts but not much else.
# Inputs
Options
: Empty set, there may be configuration options in the future
`v`
: 2\. Function argument
*/
mkValueStringDefault =
{ }:
v:
let
err = t: v: abort ("generators.mkValueStringDefault: " + "${t} not supported: ${toPretty { } v}");
in
if isInt v then
toString v
# convert derivations to store paths
else if isDerivation v then
toString v
# we default to not quoting strings
else if isString v then
v
# isString returns "1", which is not a good default
else if true == v then
"true"
# here it returns to "", which is even less of a good default
else if false == v then
"false"
else if null == v then
"null"
# if you have lists you probably want to replace this
else if isList v then
err "lists" v
# same as for lists, might want to replace
else if isAttrs v then
err "attrsets" v
# functions can’t be printed of course
else if isFunction v then
err "functions" v
# Floats currently can't be converted to precise strings,
# condition warning on nix version once this isn't a problem anymore
# See https://github.com/NixOS/nix/pull/3480
else if isFloat v then
floatToString v
else
err "this value is" (toString v);
/**
Generate a line of key k and value v, separated by
character sep. If sep appears in k, it is escaped.
Helper for synaxes with different separators.
mkValueString specifies how values should be formatted.
```nix
mkKeyValueDefault {} ":" "f:oo" "bar"
> "f\:oo:bar"
```
# Inputs
Structured function argument
: mkValueString (optional, default: `mkValueStringDefault {}`)
: Function to convert values to strings
`sep`
: 2\. Function argument
`k`
: 3\. Function argument
`v`
: 4\. Function argument
*/
mkKeyValueDefault =
{
mkValueString ? mkValueStringDefault { },
}:
sep: k: v:
"${escape [ sep ] k}${sep}${mkValueString v}";
## -- FILE FORMAT GENERATORS --
/**
Generate a key-value-style config file from an attrset.
# Inputs
Structured function argument
: mkKeyValue (optional, default: `mkKeyValueDefault {} "="`)
: format a setting line from key and value
: listsAsDuplicateKeys (optional, default: `false`)
: allow lists as values for duplicate keys
: indent (optional, default: `""`)
: Initial indentation level
*/
toKeyValue =
{
mkKeyValue ? mkKeyValueDefault { } "=",
listsAsDuplicateKeys ? false,
indent ? "",
}:
let
mkLine = k: v: indent + mkKeyValue k v + "\n";
mkLines =
if listsAsDuplicateKeys then
k: v: map (mkLine k) (if isList v then v else [ v ])
else
k: v: [ (mkLine k v) ];
in
attrs: concatStrings (concatLists (mapAttrsToList mkLines attrs));
/**
Generate an INI-style config file from an
attrset of sections to an attrset of key-value pairs.
# Inputs
Structured function argument
: mkSectionName (optional, default: `(name: escape [ "[" "]" ] name)`)
: apply transformations (e.g. escapes) to section names
: mkKeyValue (optional, default: `{} "="`)
: format a setting line from key and value
: listsAsDuplicateKeys (optional, default: `false`)
: allow lists as values for duplicate keys
# Examples
:::{.example}
## `lib.generators.toINI` usage example
```nix
generators.toINI {} {
foo = { hi = "${pkgs.hello}"; ciao = "bar"; };
baz = { "also, integers" = 42; };
}
> [baz]
> also, integers=42
>
> [foo]
> ciao=bar
> hi=/nix/store/y93qql1p5ggfnaqjjqhxcw0vqw95rlz0-hello-2.10
```
The mk* configuration attributes can generically change
the way sections and key-value strings are generated.
For more examples see the test cases in ./tests/misc.nix.
:::
*/
toINI =
{
mkSectionName ? (name: escape [ "[" "]" ] name),
mkKeyValue ? mkKeyValueDefault { } "=",
listsAsDuplicateKeys ? false,
}:
attrsOfAttrs:
let
# map function to string for each key val
mapAttrsToStringsSep =
sep: mapFn: attrs:
concatStringsSep sep (mapAttrsToList mapFn attrs);
mkSection =
sectName: sectValues:
''
[${mkSectionName sectName}]
''
+ toKeyValue { inherit mkKeyValue listsAsDuplicateKeys; } sectValues;
in
# map input to ini sections
mapAttrsToStringsSep "\n" mkSection attrsOfAttrs;
/**
Generate an INI-style config file from an attrset
specifying the global section (no header), and an
attrset of sections to an attrset of key-value pairs.
# Inputs
1\. Structured function argument
: mkSectionName (optional, default: `(name: escape [ "[" "]" ] name)`)
: apply transformations (e.g. escapes) to section names
: mkKeyValue (optional, default: `{} "="`)
: format a setting line from key and value
: listsAsDuplicateKeys (optional, default: `false`)
: allow lists as values for duplicate keys
2\. Structured function argument
: globalSection (required)
: global section key-value pairs
: sections (optional, default: `{}`)
: attrset of sections to key-value pairs
# Examples
:::{.example}
## `lib.generators.toINIWithGlobalSection` usage example
```nix
generators.toINIWithGlobalSection {} {
globalSection = {
someGlobalKey = "hi";
};
sections = {
foo = { hi = "${pkgs.hello}"; ciao = "bar"; };
baz = { "also, integers" = 42; };
}
> someGlobalKey=hi
>
> [baz]
> also, integers=42
>
> [foo]
> ciao=bar
> hi=/nix/store/y93qql1p5ggfnaqjjqhxcw0vqw95rlz0-hello-2.10
```
The mk* configuration attributes can generically change
the way sections and key-value strings are generated.
For more examples see the test cases in ./tests/misc.nix.
:::
If you don’t need a global section, you can also use
`generators.toINI` directly, which only takes
the part in `sections`.
*/
toINIWithGlobalSection =
{
mkSectionName ? (name: escape [ "[" "]" ] name),
mkKeyValue ? mkKeyValueDefault { } "=",
listsAsDuplicateKeys ? false,
}:
{
globalSection,
sections ? { },
}:
(
if globalSection == { } then
""
else
(toKeyValue { inherit mkKeyValue listsAsDuplicateKeys; } globalSection) + "\n"
)
+ (toINI { inherit mkSectionName mkKeyValue listsAsDuplicateKeys; } sections);
/**
Generate a git-config file from an attrset.
It has two major differences from the regular INI format:
1. values are indented with tabs
2. sections can have sub-sections
Further: https://git-scm.com/docs/git-config#EXAMPLES
# Examples
:::{.example}
## `lib.generators.toGitINI` usage example
```nix
generators.toGitINI {
url."ssh://git@github.com/".insteadOf = "https://github.com";
user.name = "edolstra";
}
> [url "ssh://git@github.com/"]
> insteadOf = "https://github.com"
>
> [user]
> name = "edolstra"
```
:::
# Inputs
`attrs`
: Key-value pairs to be converted to a git-config file.
See: https://git-scm.com/docs/git-config#_variables for possible values.
*/
toGitINI =
attrs:
let
mkSectionName =
name:
let
containsQuote = hasInfix ''"'' name;
sections = splitString "." name;
section = head sections;
subsections = tail sections;
subsection = concatStringsSep "." subsections;
in
if containsQuote || subsections == [ ] then name else ''${section} "${subsection}"'';
mkValueString =
v:
let
escapedV = ''"${replaceStrings [ "\n" " " ''"'' "\\" ] [ "\\n" "\\t" ''\"'' "\\\\" ] v}"'';
in
mkValueStringDefault { } (if isString v then escapedV else v);
# generation for multiple ini values
mkKeyValue =
k: v:
let
mkKeyValue = mkKeyValueDefault { inherit mkValueString; } " = " k;
in
concatStringsSep "\n" (map (kv: "\t" + mkKeyValue kv) (toList v));
# converts { a.b.c = 5; } to { "a.b".c = 5; } for toINI
gitFlattenAttrs =
let
recurse =
path: value:
if isAttrs value && !isDerivation value then
mapAttrsToList (name: value: recurse ([ name ] ++ path) value) value
else if length path > 1 then
{
${concatStringsSep "." (reverseList (tail path))}.${head path} = value;
}
else
{
${head path} = value;
};
in
attrs: foldl recursiveUpdate { } (flatten (recurse [ ] attrs));
toINI_ = toINI { inherit mkKeyValue mkSectionName; };
in
toINI_ (gitFlattenAttrs attrs);
/**
mkKeyValueDefault wrapper that handles dconf INI quirks.
The main differences of the format is that it requires strings to be quoted.
*/
mkDconfKeyValue = mkKeyValueDefault { mkValueString = v: toString (gvariant.mkValue v); } "=";
/**
Generates INI in dconf keyfile style. See https://help.gnome.org/admin/system-admin-guide/stable/dconf-keyfiles.html.en
for details.
*/
toDconfINI = toINI { mkKeyValue = mkDconfKeyValue; };
/**
Recurses through a `Value` limited to a certain depth. (`depthLimit`)
If the depth is exceeded, an error is thrown, unless `throwOnDepthLimit` is set to `false`.
# Inputs
Structured function argument
: depthLimit (required)
: If this option is not null, the given value will stop evaluating at a certain depth
: throwOnDepthLimit (optional, default: `true`)
: If this option is true, an error will be thrown, if a certain given depth is exceeded
Value
: The value to be evaluated recursively
*/
withRecursion =
{
depthLimit,
throwOnDepthLimit ? true,
}:
assert isInt depthLimit;
let
specialAttrs = [
"__functor"
"__functionArgs"
"__toString"
"__pretty"
];
stepIntoAttr = evalNext: name: if elem name specialAttrs then id else evalNext;
transform =
depth:
if depthLimit != null && depth > depthLimit then
if throwOnDepthLimit then
throw "Exceeded maximum eval-depth limit of ${toString depthLimit} while trying to evaluate with `generators.withRecursion'!"
else
const "<unevaluated>"
else
id;
mapAny =
depth: v:
let
evalNext = x: mapAny (depth + 1) (transform (depth + 1) x);
in
if isAttrs v then
mapAttrs (stepIntoAttr evalNext) v
else if isList v then
map evalNext v
else
transform (depth + 1) v;
in
mapAny 0;
/**
Pretty print a value, akin to `builtins.trace`.
Should probably be a builtin as well.
The pretty-printed string should be suitable for rendering default values
in the NixOS manual. In particular, it should be as close to a valid Nix expression
as possible.
# Inputs
Structured function argument
: allowPrettyValues
: If this option is true, attrsets like { __pretty = fn; val = …; }
will use fn to convert val to a pretty printed representation.
(This means fn is type Val -> String.)
: multiline
: If this option is true, the output is indented with newlines for attribute sets and lists
: indent
: Initial indentation level
Value
: The value to be pretty printed
*/
toPretty =
{
allowPrettyValues ? false,
multiline ? true,
indent ? "",
}:
let
go =
indent: v:
let
introSpace = if multiline then "\n${indent} " else " ";
outroSpace = if multiline then "\n${indent}" else " ";
in
if isInt v then
toString v
# toString loses precision on floats, so we use toJSON instead. This isn't perfect
# as the resulting string may not parse back as a float (e.g. 42, 1e-06), but for
# pretty-printing purposes this is acceptable.
else if isFloat v then
builtins.toJSON v
else if isString v then
let
lines = filter (v: !isList v) (split "\n" v);
escapeSingleline = escape [
"\\"
"\""
"\${"
];
escapeMultiline = replaceStrings [ "\${" "''" ] [ "''\${" "'''" ];
singlelineResult = "\"" + concatStringsSep "\\n" (map escapeSingleline lines) + "\"";
multilineResult =
let
escapedLines = map escapeMultiline lines;
# The last line gets a special treatment: if it's empty, '' is on its own line at the "outer"
# indentation level. Otherwise, '' is appended to the last line.
lastLine = last escapedLines;
in
"''"
+ introSpace
+ concatStringsSep introSpace (init escapedLines)
+ (if lastLine == "" then outroSpace else introSpace + lastLine)
+ "''";
in
if multiline && length lines > 1 then multilineResult else singlelineResult
else if true == v then
"true"
else if false == v then
"false"
else if null == v then
"null"
else if isPath v then
toString v
else if isList v then
if v == [ ] then
"[ ]"
else
"[" + introSpace + concatMapStringsSep introSpace (go (indent + " ")) v + outroSpace + "]"
else if isFunction v then
let
fna = functionArgs v;
showFnas = concatStringsSep ", " (
mapAttrsToList (name: hasDefVal: if hasDefVal then name + "?" else name) fna
);
in
if fna == { } then "<function>" else "<function, args: {${showFnas}}>"
else if isAttrs v then
# apply pretty values if allowed
if allowPrettyValues && v ? __pretty && v ? val then
v.__pretty v.val
else if v == { } then
"{ }"
else if v ? type && v.type == "derivation" then
"<derivation ${v.name or "???"}>"
else
"{"
+ introSpace
+ concatStringsSep introSpace (
mapAttrsToList (
name: value:
"${escapeNixIdentifier name} = ${
addErrorContext "while evaluating an attribute `${name}`" (go (indent + " ") value)
};"
) v
)
+ outroSpace
+ "}"
else
abort "generators.toPretty: should never happen (v = ${v})";
in
go indent;
/**
Translate a simple Nix expression to [Plist notation](https://en.wikipedia.org/wiki/Property_list).
# Inputs
Structured function argument
: escape (optional, default: `false`)
: If this option is true, XML special characters are escaped in string values and keys
Value
: The value to be converted to Plist
*/
toPlist =
{
escape ? false,
}:
v:
let
expr =
ind: x:
if x == null then
""
else if isBool x then
bool ind x
else if isInt x then
int ind x
else if isString x then
str ind x
else if isList x then
list ind x
else if isAttrs x then
attrs ind x
else if isPath x then
str ind (toString x)
else if isFloat x then
float ind x
else
abort "generators.toPlist: should never happen (v = ${v})";
literal = ind: x: ind + x;
maybeEscapeXML = if escape then escapeXML else x: x;
bool = ind: x: literal ind (if x then "<true/>" else "<false/>");
int = ind: x: literal ind "<integer>${toString x}</integer>";
str = ind: x: literal ind "<string>${maybeEscapeXML x}</string>";
key = ind: x: literal ind "<key>${maybeEscapeXML x}</key>";
float = ind: x: literal ind "<real>${toString x}</real>";
indent = ind: expr "\t${ind}";
item = ind: concatMapStringsSep "\n" (indent ind);
list =
ind: x:
concatStringsSep "\n" [
(literal ind "<array>")
(item ind x)
(literal ind "</array>")
];
attrs =
ind: x:
concatStringsSep "\n" [
(literal ind "<dict>")
(attr ind x)
(literal ind "</dict>")
];
attr =
let
attrFilter = name: value: name != "_module" && value != null;
in
ind: x:
concatStringsSep "\n" (
flatten (
mapAttrsToList (
name: value:
optionals (attrFilter name value) [
(key "\t${ind}" name)
(expr "\t${ind}" value)
]
) x
)
);
in
# TODO: As discussed in #356502, deprecated functionality should be removed sometime after 25.11.
lib.warnIf (!escape && lib.oldestSupportedReleaseIsAtLeast 2505)
"Using `lib.generators.toPlist` without `escape = true` is deprecated"
''
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
${expr "" v}
</plist>'';
/**
Translate a simple Nix expression to Dhall notation.
Note that integers are translated to Integer and never
the Natural type.
# Inputs
Options
: Empty set, there may be configuration options in the future
Value
: The value to be converted to Dhall
*/
toDhall =
{ }@args:
v:
let
concatItems = concatStringsSep ", ";
in
if isAttrs v then
"{ ${concatItems (mapAttrsToList (key: value: "${key} = ${toDhall args value}") v)} }"
else if isList v then
"[ ${concatItems (map (toDhall args) v)} ]"
else if isInt v then
"${if v < 0 then "" else "+"}${toString v}"
else if isBool v then
(if v then "True" else "False")
else if isFunction v then
abort "generators.toDhall: cannot convert a function to Dhall"
else if v == null then
abort "generators.toDhall: cannot convert a null to Dhall"
else
toJSON v;
/**
Translate a simple Nix expression to Lua representation with occasional
Lua-inlines that can be constructed by mkLuaInline function.
Configuration:
* multiline - by default is true which results in indented block-like view.
* indent - initial indent.
* asBindings - by default generate single value, but with this use attrset to set global vars.
Attention:
Regardless of multiline parameter there is no trailing newline.
# Inputs
Structured function argument
: multiline (optional, default: `true`)
: If this option is true, the output is indented with newlines for attribute sets and lists
: indent (optional, default: `""`)
: Initial indentation level
: asBindings (optional, default: `false`)
: Interpret as variable bindings
Value
: The value to be converted to Lua
# Type
```
toLua :: AttrSet -> Any -> String
```
# Examples
:::{.example}
## `lib.generators.toLua` usage example
```nix
generators.toLua {}
{
cmd = [ "typescript-language-server" "--stdio" ];
settings.workspace.library = mkLuaInline ''vim.api.nvim_get_runtime_file("", true)'';
}
->
{
["cmd"] = {
"typescript-language-server",
"--stdio"
},
["settings"] = {
["workspace"] = {
["library"] = (vim.api.nvim_get_runtime_file("", true))
}
}
}
```
:::
*/
toLua =
{
multiline ? true,
indent ? "",
asBindings ? false,
}@args:
v:
let
innerIndent = "${indent} ";
introSpace = if multiline then "\n${innerIndent}" else " ";
outroSpace = if multiline then "\n${indent}" else " ";
innerArgs = args // {
indent = if asBindings then indent else innerIndent;
asBindings = false;
};
concatItems = concatStringsSep ",${introSpace}";
isLuaInline =
{
_type ? null,
...
}:
_type == "lua-inline";
generatedBindings =
assert assertMsg (badVarNames == [ ]) "Bad Lua var names: ${toPretty { } badVarNames}";
concatStrings (mapAttrsToList (key: value: "${indent}${key} = ${toLua innerArgs value}\n") v);
# https://en.wikibooks.org/wiki/Lua_Programming/variable#Variable_names
matchVarName = match "[[:alpha:]_][[:alnum:]_]*(\\.[[:alpha:]_][[:alnum:]_]*)*";
badVarNames = filter (name: matchVarName name == null) (attrNames v);
in
if asBindings then
generatedBindings
else if v == null then
"nil"
else if isInt v || isFloat v || isString v || isBool v then
toJSON v
else if isPath v || isDerivation v then
toJSON "${v}"
else if isList v then
(
if v == [ ] then
"{}"
else
"{${introSpace}${concatItems (map (value: "${toLua innerArgs value}") v)}${outroSpace}}"
)
else if isAttrs v then
(
if isLuaInline v then
"(${v.expr})"
else if v == { } then
"{}"
else
"{${introSpace}${
concatItems (mapAttrsToList (key: value: "[${toJSON key}] = ${toLua innerArgs value}") v)
}${outroSpace}}"
)
else
abort "generators.toLua: type ${typeOf v} is unsupported";
/**
Mark string as Lua expression to be inlined when processed by toLua.
# Inputs
`expr`
: 1\. Function argument
# Type
```
mkLuaInline :: String -> AttrSet
```
*/
mkLuaInline = expr: {
_type = "lua-inline";
inherit expr;
};
}
// {
/**
Generates JSON from an arbitrary (non-function) value.
For more information see the documentation of the builtin.
# Inputs
Options
: Empty set, there may be configuration options in the future
Value
: The value to be converted to JSON
*/
toJSON = { }: lib.strings.toJSON;
/**
YAML has been a strict superset of JSON since 1.2, so we
use toJSON. Before it only had a few differences referring
to implicit typing rules, so it should work with older
parsers as well.
# Inputs
Options
: Empty set, there may be configuration options in the future
Value
: The value to be converted to YAML
*/
toYAML = { }: lib.strings.toJSON;
}

592
lib/gvariant.nix Normal file
View File

@@ -0,0 +1,592 @@
/**
A partial and basic implementation of GVariant formatted strings.
See [GVariant Format Strings](https://docs.gtk.org/glib/gvariant-format-strings.html) for details.
:::{.warning}
This API is not considered fully stable and it might therefore
change in backwards incompatible ways without prior notice.
:::
*/
# This file is based on https://github.com/nix-community/home-manager
# Copyright (c) 2017-2022 Home Manager contributors
{ lib }:
let
inherit (lib)
concatMapStringsSep
concatStrings
escape
head
replaceString
;
mkPrimitive = t: v: {
_type = "gvariant";
type = t;
value = v;
__toString = self: "@${self.type} ${toString self.value}"; # https://docs.gtk.org/glib/gvariant-text.html
};
type = {
arrayOf = t: "a${t}";
maybeOf = t: "m${t}";
tupleOf = ts: "(${concatStrings ts})";
dictionaryEntryOf = nameType: valueType: "{${nameType}${valueType}}";
string = "s";
boolean = "b";
uchar = "y";
int16 = "n";
uint16 = "q";
int32 = "i";
uint32 = "u";
int64 = "x";
uint64 = "t";
double = "d";
variant = "v";
};
in
rec {
inherit type;
/**
Check if a value is a GVariant value
# Inputs
`v`
: value to check
# Type
```
isGVariant :: Any -> Bool
```
*/
isGVariant = v: v._type or "" == "gvariant";
intConstructors = [
{
name = "mkInt32";
type = type.int32;
min = -2147483648;
max = 2147483647;
}
{
name = "mkUint32";
type = type.uint32;
min = 0;
max = 4294967295;
}
{
name = "mkInt64";
type = type.int64;
# Nix does not support such large numbers.
min = null;
max = null;
}
{
name = "mkUint64";
type = type.uint64;
min = 0;
# Nix does not support such large numbers.
max = null;
}
{
name = "mkInt16";
type = type.int16;
min = -32768;
max = 32767;
}
{
name = "mkUint16";
type = type.uint16;
min = 0;
max = 65535;
}
{
name = "mkUchar";
type = type.uchar;
min = 0;
max = 255;
}
];
/**
Returns the GVariant value that most closely matches the given Nix value.
If no GVariant value can be found unambiguously then error is thrown.
# Inputs
`v`
: 1\. Function argument
# Type
```
mkValue :: Any -> gvariant
```
*/
mkValue =
v:
if builtins.isBool v then
mkBoolean v
else if builtins.isFloat v then
mkDouble v
else if builtins.isString v then
mkString v
else if builtins.isList v then
mkArray v
else if isGVariant v then
v
else if builtins.isInt v then
let
validConstructors = builtins.filter (
{ min, max, ... }: (min == null || min <= v) && (max == null || v <= max)
) intConstructors;
in
throw ''
The GVariant type for number “${toString v}” is unclear.
Please wrap the value with one of the following, depending on the value type in GSettings schema:
${lib.concatMapStringsSep "\n" (
{ name, type, ... }: "- `lib.gvariant.${name}` for `${type}`"
) validConstructors}
''
else if builtins.isAttrs v then
throw "Cannot construct GVariant value from an attribute set. If you want to construct a dictionary, you will need to create an array containing items constructed with `lib.gvariant.mkDictionaryEntry`."
else
throw "The GVariant type of “${builtins.typeOf v}” can't be inferred.";
/**
Returns the GVariant array from the given type of the elements and a Nix list.
# Inputs
`elems`
: 1\. Function argument
# Type
```
mkArray :: [Any] -> gvariant
```
# Examples
:::{.example}
## `lib.gvariant.mkArray` usage example
```nix
# Creating a string array
lib.gvariant.mkArray [ "a" "b" "c" ]
```
:::
*/
mkArray =
elems:
let
vs = map mkValue (lib.throwIf (elems == [ ]) "Please create empty array with mkEmptyArray." elems);
elemType = lib.throwIfNot (lib.all (t: (head vs).type == t) (
map (v: v.type) vs
)) "Elements in a list should have same type." (head vs).type;
in
mkPrimitive (type.arrayOf elemType) vs
// {
__toString = self: "@${self.type} [${concatMapStringsSep "," toString self.value}]";
};
/**
Returns the GVariant array from the given empty Nix list.
# Inputs
`elemType`
: 1\. Function argument
# Type
```
mkEmptyArray :: gvariant.type -> gvariant
```
# Examples
:::{.example}
## `lib.gvariant.mkEmptyArray` usage example
```nix
# Creating an empty string array
lib.gvariant.mkEmptyArray (lib.gvariant.type.string)
```
:::
*/
mkEmptyArray =
elemType:
mkPrimitive (type.arrayOf elemType) [ ]
// {
__toString = self: "@${self.type} []";
};
/**
Returns the GVariant variant from the given Nix value. Variants are containers
of different GVariant type.
# Inputs
`elem`
: 1\. Function argument
# Type
```
mkVariant :: Any -> gvariant
```
# Examples
:::{.example}
## `lib.gvariant.mkVariant` usage example
```nix
lib.gvariant.mkArray [
(lib.gvariant.mkVariant "a string")
(lib.gvariant.mkVariant (lib.gvariant.mkInt32 1))
]
```
:::
*/
mkVariant =
elem:
let
gvarElem = mkValue elem;
in
mkPrimitive type.variant gvarElem
// {
__toString = self: "<${toString self.value}>";
};
/**
Returns the GVariant dictionary entry from the given key and value.
# Inputs
`name`
: The key of the entry
`value`
: The value of the entry
# Type
```
mkDictionaryEntry :: String -> Any -> gvariant
```
# Examples
:::{.example}
## `lib.gvariant.mkDictionaryEntry` usage example
```nix
# A dictionary describing an Epiphany’s search provider
[
(lib.gvariant.mkDictionaryEntry "url" (lib.gvariant.mkVariant "https://duckduckgo.com/?q=%s&t=epiphany"))
(lib.gvariant.mkDictionaryEntry "bang" (lib.gvariant.mkVariant "!d"))
(lib.gvariant.mkDictionaryEntry "name" (lib.gvariant.mkVariant "DuckDuckGo"))
]
```
:::
*/
mkDictionaryEntry =
name: value:
let
name' = mkValue name;
value' = mkValue value;
dictionaryType = type.dictionaryEntryOf name'.type value'.type;
in
mkPrimitive dictionaryType { inherit name value; }
// {
__toString = self: "@${self.type} {${name'},${value'}}";
};
/**
Returns the GVariant maybe from the given element type.
# Inputs
`elemType`
: 1\. Function argument
`elem`
: 2\. Function argument
# Type
```
mkMaybe :: gvariant.type -> Any -> gvariant
```
*/
mkMaybe =
elemType: elem:
mkPrimitive (type.maybeOf elemType) elem
// {
__toString =
self: if self.value == null then "@${self.type} nothing" else "just ${toString self.value}";
};
/**
Returns the GVariant nothing from the given element type.
# Inputs
`elemType`
: 1\. Function argument
# Type
```
mkNothing :: gvariant.type -> gvariant
```
*/
mkNothing = elemType: mkMaybe elemType null;
/**
Returns the GVariant just from the given Nix value.
# Inputs
`elem`
: 1\. Function argument
# Type
```
mkJust :: Any -> gvariant
```
*/
mkJust =
elem:
let
gvarElem = mkValue elem;
in
mkMaybe gvarElem.type gvarElem;
/**
Returns the GVariant tuple from the given Nix list.
# Inputs
`elems`
: 1\. Function argument
# Type
```
mkTuple :: [Any] -> gvariant
```
*/
mkTuple =
elems:
let
gvarElems = map mkValue elems;
tupleType = type.tupleOf (map (e: e.type) gvarElems);
in
mkPrimitive tupleType gvarElems
// {
__toString = self: "@${self.type} (${concatMapStringsSep "," toString self.value})";
};
/**
Returns the GVariant boolean from the given Nix bool value.
# Inputs
`v`
: 1\. Function argument
# Type
```
mkBoolean :: Bool -> gvariant
```
*/
mkBoolean =
v:
mkPrimitive type.boolean v
// {
__toString = self: if self.value then "true" else "false";
};
/**
Returns the GVariant string from the given Nix string value.
# Inputs
`v`
: 1\. Function argument
# Type
```
mkString :: String -> gvariant
```
*/
mkString =
v:
let
sanitize = s: replaceString "\n" "\\n" (escape [ "'" "\\" ] s);
in
mkPrimitive type.string v
// {
__toString = self: "'${sanitize self.value}'";
};
/**
Returns the GVariant object path from the given Nix string value.
# Inputs
`v`
: 1\. Function argument
# Type
```
mkObjectpath :: String -> gvariant
```
*/
mkObjectpath =
v:
mkPrimitive type.string v
// {
__toString = self: "objectpath '${escape [ "'" ] self.value}'";
};
/**
Returns the GVariant uchar from the given Nix int value.
# Type
```
mkUchar :: Int -> gvariant
```
*/
mkUchar = mkPrimitive type.uchar;
/**
Returns the GVariant int16 from the given Nix int value.
# Type
```
mkInt16 :: Int -> gvariant
```
*/
mkInt16 = mkPrimitive type.int16;
/**
Returns the GVariant uint16 from the given Nix int value.
# Type
```
mkUint16 :: Int -> gvariant
```
*/
mkUint16 = mkPrimitive type.uint16;
/**
Returns the GVariant int32 from the given Nix int value.
# Inputs
`v`
: 1\. Function argument
# Type
```
mkInt32 :: Int -> gvariant
```
*/
mkInt32 =
v:
mkPrimitive type.int32 v
// {
__toString = self: toString self.value;
};
/**
Returns the GVariant uint32 from the given Nix int value.
# Type
```
mkUint32 :: Int -> gvariant
```
*/
mkUint32 = mkPrimitive type.uint32;
/**
Returns the GVariant int64 from the given Nix int value.
# Type
```
mkInt64 :: Int -> gvariant
```
*/
mkInt64 = mkPrimitive type.int64;
/**
Returns the GVariant uint64 from the given Nix int value.
# Type
```
mkUint64 :: Int -> gvariant
```
*/
mkUint64 = mkPrimitive type.uint64;
/**
Returns the GVariant double from the given Nix float value.
# Inputs
`v`
: 1\. Function argument
# Type
```
mkDouble :: Float -> gvariant
```
*/
mkDouble =
v:
mkPrimitive type.double v
// {
__toString = self: toString self.value;
};
}

40
lib/kernel.nix Normal file
View File

@@ -0,0 +1,40 @@
{ lib }:
let
inherit (lib) mkIf versionAtLeast versionOlder;
in
{
# Keeping these around in case we decide to change this horrible implementation :)
option = x: x // { optional = true; };
yes = {
tristate = "y";
optional = false;
};
no = {
tristate = "n";
optional = false;
};
module = {
tristate = "m";
optional = false;
};
unset = {
tristate = null;
optional = false;
};
freeform = x: {
freeform = x;
optional = false;
};
# Common patterns/legacy used in common-config/hardened/config.nix
whenHelpers = version: {
whenAtLeast = ver: mkIf (versionAtLeast version ver);
whenOlder = ver: mkIf (versionOlder version ver);
# range is (inclusive, exclusive)
whenBetween = verLow: verHigh: mkIf (versionAtLeast version verLow && versionOlder version verHigh);
};
}

1578
lib/licenses.nix Normal file
View File

File diff suppressed because it is too large Load Diff

2010
lib/lists.nix Normal file
View File

File diff suppressed because it is too large Load Diff

681
lib/meta.nix Normal file
View File

@@ -0,0 +1,681 @@
/**
Some functions for manipulating meta attributes, as well as the
name attribute.
*/
{ lib }:
let
inherit (lib)
matchAttrs
any
all
isDerivation
getBin
assertMsg
;
inherit (lib.attrsets) mapAttrs' filterAttrs;
inherit (builtins)
isString
match
typeOf
elemAt
;
in
rec {
/**
Add to or override the meta attributes of the given
derivation.
# Inputs
`newAttrs`
: 1\. Function argument
`drv`
: 2\. Function argument
# Examples
:::{.example}
## `lib.meta.addMetaAttrs` usage example
```nix
addMetaAttrs {description = "Bla blah";} somePkg
```
:::
*/
addMetaAttrs =
newAttrs: drv:
if drv ? overrideAttrs then
drv.overrideAttrs (old: {
meta = (old.meta or { }) // newAttrs;
})
else
drv // { meta = (drv.meta or { }) // newAttrs; };
/**
Disable Hydra builds of given derivation.
# Inputs
`drv`
: 1\. Function argument
*/
dontDistribute = drv: addMetaAttrs { hydraPlatforms = [ ]; } drv;
/**
Change the [symbolic name of a derivation](https://nixos.org/manual/nix/stable/language/derivations.html#attr-name).
:::{.warning}
Dependent derivations will be rebuilt when the symbolic name is changed.
:::
# Inputs
`name`
: 1\. Function argument
`drv`
: 2\. Function argument
*/
setName = name: drv: drv // { inherit name; };
/**
Like `setName`, but takes the previous name as an argument.
# Inputs
`updater`
: 1\. Function argument
`drv`
: 2\. Function argument
# Examples
:::{.example}
## `lib.meta.updateName` usage example
```nix
updateName (oldName: oldName + "-experimental") somePkg
```
:::
*/
updateName = updater: drv: drv // { name = updater (drv.name); };
/**
Append a suffix to the name of a package (before the version
part).
# Inputs
`suffix`
: 1\. Function argument
*/
appendToName =
suffix:
updateName (
name:
let
x = builtins.parseDrvName name;
in
"${x.name}-${suffix}-${x.version}"
);
/**
Apply a function to each derivation and only to derivations in an attrset.
# Inputs
`f`
: 1\. Function argument
`set`
: 2\. Function argument
*/
mapDerivationAttrset =
f: set: lib.mapAttrs (name: pkg: if lib.isDerivation pkg then (f pkg) else pkg) set;
/**
The default priority of packages in Nix. See `defaultPriority` in [`src/nix/profile.cc`](https://github.com/NixOS/nix/blob/master/src/nix/profile.cc#L47).
*/
defaultPriority = 5;
/**
Set the nix-env priority of the package. Note that higher values are lower priority, and vice versa.
# Inputs
`priority`
: 1\. The priority to set.
`drv`
: 2\. Function argument
*/
setPrio = priority: addMetaAttrs { inherit priority; };
/**
Decrease the nix-env priority of the package, i.e., other
versions/variants of the package will be preferred.
# Inputs
`drv`
: 1\. Function argument
*/
lowPrio = setPrio 10;
/**
Apply lowPrio to an attrset with derivations.
# Inputs
`set`
: 1\. Function argument
*/
lowPrioSet = set: mapDerivationAttrset lowPrio set;
/**
Increase the nix-env priority of the package, i.e., this
version/variant of the package will be preferred.
# Inputs
`drv`
: 1\. Function argument
*/
hiPrio = setPrio (-10);
/**
Apply hiPrio to an attrset with derivations.
# Inputs
`set`
: 1\. Function argument
*/
hiPrioSet = set: mapDerivationAttrset hiPrio set;
/**
Check to see if a platform is matched by the given `meta.platforms`
element.
A `meta.platform` pattern is either
1. (legacy) a system string.
2. (modern) a pattern for the entire platform structure (see `lib.systems.inspect.platformPatterns`).
3. (modern) a pattern for the platform `parsed` field (see `lib.systems.inspect.patterns`).
We can inject these into a pattern for the whole of a structured platform,
and then match that.
# Inputs
`platform`
: 1\. Function argument
`elem`
: 2\. Function argument
# Examples
:::{.example}
## `lib.meta.platformMatch` usage example
```nix
lib.meta.platformMatch { system = "aarch64-darwin"; } "aarch64-darwin"
=> true
```
:::
*/
platformMatch =
platform: elem:
(
# Check with simple string comparison if elem was a string.
#
# The majority of comparisons done with this function will be against meta.platforms
# which contains a simple platform string.
#
# Avoiding an attrset allocation results in significant performance gains (~2-30) across the board in OfBorg
# because this is a hot path for nixpkgs.
if isString elem then
platform ? system && elem == platform.system
else
matchAttrs (
# Normalize platform attrset.
if elem ? parsed then elem else { parsed = elem; }
) platform
);
/**
Check if a package is available on a given platform.
A package is available on a platform if both
1. One of `meta.platforms` pattern matches the given
platform, or `meta.platforms` is not present.
2. None of `meta.badPlatforms` pattern matches the given platform.
# Inputs
`platform`
: 1\. Function argument
`pkg`
: 2\. Function argument
# Examples
:::{.example}
## `lib.meta.availableOn` usage example
```nix
lib.meta.availableOn { system = "aarch64-darwin"; } pkg.zsh
=> true
```
:::
*/
availableOn =
platform: pkg:
((!pkg ? meta.platforms) || any (platformMatch platform) pkg.meta.platforms)
&& all (elem: !platformMatch platform elem) (pkg.meta.badPlatforms or [ ]);
/**
Mapping of SPDX ID to the attributes in lib.licenses.
For SPDX IDs, see https://spdx.org/licenses.
Note that some SPDX licenses might be missing.
# Examples
:::{.example}
## `lib.meta.licensesSpdx` usage example
```nix
lib.licensesSpdx.MIT == lib.licenses.mit
=> true
lib.licensesSpdx."MY LICENSE"
=> error: attribute 'MY LICENSE' missing
```
:::
*/
licensesSpdx = mapAttrs' (_key: license: {
name = license.spdxId;
value = license;
}) (filterAttrs (_key: license: license ? spdxId) lib.licenses);
/**
Get the corresponding attribute in lib.licenses from the SPDX ID
or warn and fallback to `{ shortName = <license string>; }`.
For SPDX IDs, see https://spdx.org/licenses.
Note that some SPDX licenses might be missing.
# Type
```
getLicenseFromSpdxId :: str -> AttrSet
```
# Examples
:::{.example}
## `lib.meta.getLicenseFromSpdxId` usage example
```nix
lib.getLicenseFromSpdxId "MIT" == lib.licenses.mit
=> true
lib.getLicenseFromSpdxId "mIt" == lib.licenses.mit
=> true
lib.getLicenseFromSpdxId "MY LICENSE"
=> trace: warning: getLicenseFromSpdxId: No license matches the given SPDX ID: MY LICENSE
=> { shortName = "MY LICENSE"; }
```
:::
*/
getLicenseFromSpdxId =
licstr:
getLicenseFromSpdxIdOr licstr (
lib.warn "getLicenseFromSpdxId: No license matches the given SPDX ID: ${licstr}" {
shortName = licstr;
}
);
/**
Get the corresponding attribute in lib.licenses from the SPDX ID
or fallback to the given default value.
For SPDX IDs, see https://spdx.org/licenses.
Note that some SPDX licenses might be missing.
# Inputs
`licstr`
: 1\. SPDX ID string to find a matching license
`default`
: 2\. Fallback value when a match is not found
# Type
```
getLicenseFromSpdxIdOr :: str -> Any -> Any
```
# Examples
:::{.example}
## `lib.meta.getLicenseFromSpdxIdOr` usage example
```nix
lib.getLicenseFromSpdxIdOr "MIT" null == lib.licenses.mit
=> true
lib.getLicenseFromSpdxId "mIt" null == lib.licenses.mit
=> true
lib.getLicenseFromSpdxIdOr "MY LICENSE" lib.licenses.free == lib.licenses.free
=> true
lib.getLicenseFromSpdxIdOr "MY LICENSE" null
=> null
lib.getLicenseFromSpdxIdOr "MY LICENSE" (throw "No SPDX ID matches MY LICENSE")
=> error: No SPDX ID matches MY LICENSE
```
:::
*/
getLicenseFromSpdxIdOr =
let
lowercaseLicenses = lib.mapAttrs' (name: value: {
name = lib.toLower name;
inherit value;
}) licensesSpdx;
in
licstr: default: lowercaseLicenses.${lib.toLower licstr} or default;
/**
Get the path to the main program of a package based on meta.mainProgram
# Inputs
`x`
: 1\. Function argument
# Type
```
getExe :: package -> string
```
# Examples
:::{.example}
## `lib.meta.getExe` usage example
```nix
getExe pkgs.hello
=> "/nix/store/g124820p9hlv4lj8qplzxw1c44dxaw1k-hello-2.12/bin/hello"
getExe pkgs.mustache-go
=> "/nix/store/am9ml4f4ywvivxnkiaqwr0hyxka1xjsf-mustache-go-1.3.0/bin/mustache"
```
:::
*/
getExe =
x:
getExe' x (
x.meta.mainProgram or (
# This could be turned into an error when 23.05 is at end of life
lib.warn
"getExe: Package ${
lib.strings.escapeNixIdentifier x.meta.name or x.pname or x.name
} does not have the meta.mainProgram attribute. We'll assume that the main program has the same name for now, but this behavior is deprecated, because it leads to surprising errors when the assumption does not hold. If the package has a main program, please set `meta.mainProgram` in its definition to make this warning go away. Otherwise, if the package does not have a main program, or if you don't control its definition, use getExe' to specify the name to the program, such as lib.getExe' foo \"bar\"."
lib.getName
x
)
);
/**
Get the path of a program of a derivation.
# Inputs
`x`
: 1\. Function argument
`y`
: 2\. Function argument
# Type
```
getExe' :: derivation -> string -> string
```
# Examples
:::{.example}
## `lib.meta.getExe'` usage example
```nix
getExe' pkgs.hello "hello"
=> "/nix/store/g124820p9hlv4lj8qplzxw1c44dxaw1k-hello-2.12/bin/hello"
getExe' pkgs.imagemagick "convert"
=> "/nix/store/5rs48jamq7k6sal98ymj9l4k2bnwq515-imagemagick-7.1.1-15/bin/convert"
```
:::
*/
getExe' =
x: y:
assert assertMsg (isDerivation x)
"lib.meta.getExe': The first argument is of type ${typeOf x}, but it should be a derivation instead.";
assert assertMsg (isString y)
"lib.meta.getExe': The second argument is of type ${typeOf y}, but it should be a string instead.";
assert assertMsg (match ".*/.*" y == null)
"lib.meta.getExe': The second argument \"${y}\" is a nested path with a \"/\" character, but it should just be the name of the executable instead.";
"${getBin x}/bin/${y}";
/**
Generate [CPE parts](#var-meta-identifiers-cpeParts) from inputs. Copies `vendor` and `version` to the output, and sets `update` to `*`.
# Inputs
`vendor`
: package's vendor
`version`
: package's version
# Type
```
cpeFullVersionWithVendor :: string -> string -> AttrSet
```
# Examples
:::{.example}
## `lib.meta.cpeFullVersionWithVendor` usage example
```nix
lib.meta.cpeFullVersionWithVendor "gnu" "1.2.3"
=> {
vendor = "gnu";
version = "1.2.3";
update = "*";
}
```
:::
:::{.example}
## `lib.meta.cpeFullVersionWithVendor` usage in derivations
```nix
mkDerivation rec {
version = "1.2.3";
# ...
meta = {
# ...
identifiers.cpeParts = lib.meta.cpeFullVersionWithVendor "gnu" version;
};
}
```
:::
*/
cpeFullVersionWithVendor = vendor: version: {
inherit vendor version;
update = "*";
};
/**
Alternate version of [`lib.meta.cpePatchVersionInUpdateWithVendor`](#function-library-lib.meta.cpePatchVersionInUpdateWithVendor).
If `cpePatchVersionInUpdateWithVendor` succeeds, returns an attribute set with `success` set to `true` and `value` set to the result.
Otherwise, `success` is set to `false` and `error` is set to the string representation of the error.
# Inputs
`vendor`
: package's vendor
`version`
: package's version
# Type
```
tryCPEPatchVersionInUpdateWithVendor :: string -> string -> AttrSet
```
# Examples
:::{.example}
## `lib.meta.tryCPEPatchVersionInUpdateWithVendor` usage example
```nix
lib.meta.tryCPEPatchVersionInUpdateWithVendor "gnu" "1.2.3"
=> {
success = true;
value = {
vendor = "gnu";
version = "1.2";
update = "3";
};
}
```
:::
:::{.example}
## `lib.meta.cpePatchVersionInUpdateWithVendor` error example
```nix
lib.meta.tryCPEPatchVersionInUpdateWithVendor "gnu" "5.3p0"
=> {
success = false;
error = "version 5.3p0 doesn't match regex `([0-9]+\\.[0-9]+)\\.([0-9]+)`";
}
```
:::
*/
tryCPEPatchVersionInUpdateWithVendor =
vendor: version:
let
regex = "([0-9]+\\.[0-9]+)\\.([0-9]+)";
# we have to call toString here in case version is an attrset with __toString attribute
versionMatch = builtins.match regex (toString version);
in
if versionMatch == null then
{
success = false;
error = "version ${version} doesn't match regex `${regex}`";
}
else
{
success = true;
value = {
inherit vendor;
version = elemAt versionMatch 0;
update = elemAt versionMatch 1;
};
};
/**
Generate [CPE parts](#var-meta-identifiers-cpeParts) from inputs. Copies `vendor` to the result. When `version` matches `X.Y.Z` where all parts are numerical, sets `version` and `update` fields to `X.Y` and `Z`. Throws an error if the version doesn't match the expected template.
# Inputs
`vendor`
: package's vendor
`version`
: package's version
# Type
```
cpePatchVersionInUpdateWithVendor :: string -> string -> AttrSet
```
# Examples
:::{.example}
## `lib.meta.cpePatchVersionInUpdateWithVendor` usage example
```nix
lib.meta.cpePatchVersionInUpdateWithVendor "gnu" "1.2.3"
=> {
vendor = "gnu";
version = "1.2";
update = "3";
}
```
:::
:::{.example}
## `lib.meta.cpePatchVersionInUpdateWithVendor` usage in derivations
```nix
mkDerivation rec {
version = "1.2.3";
# ...
meta = {
# ...
identifiers.cpeParts = lib.meta.cpePatchVersionInUpdateWithVendor "gnu" version;
};
}
```
:::
*/
cpePatchVersionInUpdateWithVendor =
vendor: version:
let
result = tryCPEPatchVersionInUpdateWithVendor vendor version;
in
if result.success then result.value else throw result.error;
}

19
lib/minfeatures.nix Normal file
View File

@@ -0,0 +1,19 @@
let
features = [
{
description = "the `nixVersion` builtin";
condition = builtins ? nixVersion;
}
{
description = "`builtins.nixVersion` reports at least 2.18";
condition = builtins ? nixVersion && builtins.compareVersions "2.18" builtins.nixVersion != 1;
}
];
evaluated = builtins.partition ({ condition, ... }: condition) features;
in
{
all = features;
supported = evaluated.right;
missing = evaluated.wrong;
}

2209
lib/modules.nix Normal file
View File

File diff suppressed because it is too large Load Diff

112
lib/network/default.nix Normal file
View File

@@ -0,0 +1,112 @@
{ lib }:
let
inherit (import ./internal.nix { inherit lib; }) _ipv6;
inherit (lib.strings) match concatStringsSep toLower;
inherit (lib.trivial)
pipe
bitXor
fromHexString
toHexString
;
inherit (lib.lists) elemAt;
in
{
ipv6 = {
/**
Creates an `IPv6Address` object from an IPv6 address as a string. If
the prefix length is omitted, it defaults to 64. The parser is limited
to the first two versions of IPv6 addresses addressed in RFC 4291.
The form "x:x:x:x:x:x:d.d.d.d" is not yet implemented. Addresses are
NOT compressed, so they are not always the same as the canonical text
representation of IPv6 addresses defined in RFC 5952.
# Type
```
fromString :: String -> IPv6Address
```
# Examples
```nix
fromString "2001:DB8::ffff/32"
=> {
address = "2001:db8:0:0:0:0:0:ffff";
prefixLength = 32;
}
```
# Arguments
- [addr] An IPv6 address with optional prefix length.
*/
fromString =
addr:
let
splittedAddr = _ipv6.split addr;
addrInternal = splittedAddr.address;
prefixLength = splittedAddr.prefixLength;
address = _ipv6.toStringFromExpandedIp addrInternal;
in
{
inherit address prefixLength;
};
/**
Converts a 48-bit MAC address into a EUI-64 IPv6 address suffix.
# Example
```nix
mkEUI64Suffix "66:75:63:6B:20:75"
=> "6475:63ff:fe6b:2075"
```
# Type
```
mkEUI64Suffix :: String -> String
```
# Inputs
mac
: The MAC address (may contain these delimiters: `:`, `-` or `.` but it's not necessary)
*/
mkEUI64Suffix =
mac:
pipe mac [
# match mac address
(match "^([0-9A-Fa-f]{2})[-:.]?([0-9A-Fa-f]{2})[-:.]?([0-9A-Fa-f]{2})[-:.]?([0-9A-Fa-f]{2})[-:.]?([0-9A-Fa-f]{2})[-:.]?([0-9A-Fa-f]{2})$")
# check if there are matches
(
matches:
if matches == null then
throw ''"${mac}" is not a valid MAC address (expected 6 octets of hex digits)''
else
matches
)
# transform to result hextets
(octets: [
# combine 1st and 2nd octets into first hextet, flip U/L bit, 512 = 0x200
(toHexString (bitXor 512 (fromHexString ((elemAt octets 0) + (elemAt octets 1)))))
# combine 3rd and 4th octets, combine them, insert fffe pattern in between to get next two hextets
"${elemAt octets 2}ff"
"fe${elemAt octets 3}"
# combine 5th and 6th octets into the last hextet
((elemAt octets 4) + (elemAt octets 5))
])
# concat to result suffix
(concatStringsSep ":")
toLower
];
};
}

209
lib/network/internal.nix Normal file
View File

@@ -0,0 +1,209 @@
{
lib ? import ../.,
}:
let
inherit (builtins)
map
match
genList
length
concatMap
head
toString
;
inherit (lib) lists strings trivial;
inherit (lib.lists) last;
/**
IPv6 addresses are 128-bit identifiers. The preferred form is 'x:x:x:x:x:x:x:x',
where the 'x's are one to four hexadecimal digits of the eight 16-bit pieces of
the address. See RFC 4291.
*/
ipv6Bits = 128;
ipv6Pieces = 8; # 'x:x:x:x:x:x:x:x'
ipv6PieceBits = 16; # One piece in range from 0 to 0xffff.
ipv6PieceMaxValue = 65535; # 2^16 - 1
in
let
/**
Expand an IPv6 address by removing the "::" compression and padding them
with the necessary number of zeros. Converts an address from the string to
the list of strings which then can be parsed using `_parseExpanded`.
Throws an error when the address is malformed.
# Type: String -> [ String ]
# Example:
```nix
expandIpv6 "2001:DB8::ffff"
=> ["2001" "DB8" "0" "0" "0" "0" "0" "ffff"]
```
*/
expandIpv6 =
addr:
if match "^[0-9A-Fa-f:]+$" addr == null then
throw "${addr} contains malformed characters for IPv6 address"
else
let
pieces = strings.splitString ":" addr;
piecesNoEmpty = lists.remove "" pieces;
piecesNoEmptyLen = length piecesNoEmpty;
zeros = genList (_: "0") (ipv6Pieces - piecesNoEmptyLen);
hasPrefix = strings.hasPrefix "::" addr;
hasSuffix = strings.hasSuffix "::" addr;
hasInfix = strings.hasInfix "::" addr;
in
if addr == "::" then
zeros
else if
let
emptyCount = length pieces - piecesNoEmptyLen;
emptyExpected =
# splitString produces two empty pieces when "::" in the beginning
# or in the end, and only one when in the middle of an address.
if hasPrefix || hasSuffix then
2
else if hasInfix then
1
else
0;
in
emptyCount != emptyExpected
|| (hasInfix && piecesNoEmptyLen >= ipv6Pieces) # "::" compresses at least one group of zeros.
|| (!hasInfix && piecesNoEmptyLen != ipv6Pieces)
then
throw "${addr} is not a valid IPv6 address"
# Create a list of 8 elements, filling some of them with zeros depending
# on where the "::" was found.
else if hasPrefix then
zeros ++ piecesNoEmpty
else if hasSuffix then
piecesNoEmpty ++ zeros
else if hasInfix then
concatMap (piece: if piece == "" then zeros else [ piece ]) pieces
else
pieces;
/**
Parses an expanded IPv6 address (see `expandIpv6`), converting each part
from a string to an u16 integer. Returns an internal representation of IPv6
address (list of integers) that can be easily processed by other helper
functions.
Throws an error some element is not an u16 integer.
# Type: [ String ] -> IPv6
# Example:
```nix
parseExpandedIpv6 ["2001" "DB8" "0" "0" "0" "0" "0" "ffff"]
=> [8193 3512 0 0 0 0 0 65535]
```
*/
parseExpandedIpv6 =
addr:
assert lib.assertMsg (
length addr == ipv6Pieces
) "parseExpandedIpv6: expected list of integers with ${ipv6Pieces} elements";
let
u16FromHexStr =
hex:
let
parsed = trivial.fromHexString hex;
in
if 0 <= parsed && parsed <= ipv6PieceMaxValue then
parsed
else
throw "0x${hex} is not a valid u16 integer";
in
map (piece: u16FromHexStr piece) addr;
in
let
/**
Parses an IPv6 address from a string to the internal representation (list
of integers).
# Type: String -> IPv6
# Example:
```nix
parseIpv6FromString "2001:DB8::ffff"
=> [8193 3512 0 0 0 0 0 65535]
```
*/
parseIpv6FromString = addr: parseExpandedIpv6 (expandIpv6 addr);
in
{
/**
Internally, an IPv6 address is stored as a list of 16-bit integers with 8
elements. Wherever you see `IPv6` in internal functions docs, it means that
it is a list of integers produced by one of the internal parsers, such as
`parseIpv6FromString`
*/
_ipv6 = {
/**
Converts an internal representation of an IPv6 address (i.e, a list
of integers) to a string. The returned string is not a canonical
representation as defined in RFC 5952, i.e zeros are not compressed.
# Type: IPv6 -> String
# Example:
```nix
parseIpv6FromString [8193 3512 0 0 0 0 0 65535]
=> "2001:db8:0:0:0:0:0:ffff"
```
*/
toStringFromExpandedIp =
pieces: strings.concatMapStringsSep ":" (piece: strings.toLower (trivial.toHexString piece)) pieces;
/**
Extract an address and subnet prefix length from a string. The subnet
prefix length is optional and defaults to 128. The resulting address and
prefix length are validated and converted to an internal representation
that can be used by other functions.
# Type: String -> [ {address :: IPv6, prefixLength :: Int} ]
# Example:
```nix
split "2001:DB8::ffff/32"
=> {
address = [8193 3512 0 0 0 0 0 65535];
prefixLength = 32;
}
```
*/
split =
addr:
let
splitted = strings.splitString "/" addr;
splittedLength = length splitted;
in
if splittedLength == 1 then # [ ip ]
{
address = parseIpv6FromString addr;
prefixLength = ipv6Bits;
}
else if splittedLength == 2 then # [ ip subnet ]
{
address = parseIpv6FromString (head splitted);
prefixLength =
let
n = strings.toInt (last splitted);
in
if 1 <= n && n <= ipv6Bits then
n
else
throw "${addr} IPv6 subnet should be in range [1;${toString ipv6Bits}], got ${toString n}";
}
else
throw "${addr} is not a valid IPv6 address in CIDR notation";
};
}

834
lib/options.nix Normal file
View File

@@ -0,0 +1,834 @@
/**
Module System option handling.
*/
{ lib }:
let
inherit (lib)
all
collect
concatLists
concatMap
concatMapStringsSep
filter
foldl'
head
tail
isAttrs
isBool
isDerivation
isFunction
isInt
isList
isString
length
mapAttrs
optional
optionals
take
;
inherit (lib.attrsets)
attrByPath
optionalAttrs
showAttrPath
;
inherit (lib.strings)
concatMapStrings
concatStringsSep
;
inherit (lib.types)
mkOptionType
;
inherit (lib.lists)
last
toList
;
prioritySuggestion = ''
Use `lib.mkForce value` or `lib.mkDefault value` to change the priority on any of these definitions.
'';
in
rec {
/**
Returns true when the given argument `a` is an option
# Inputs
`a`
: Any value to check whether it is an option
# Examples
:::{.example}
## `lib.options.isOption` usage example
```nix
isOption 1 // => false
isOption (mkOption {}) // => true
```
:::
# Type
```
isOption :: a -> Bool
```
*/
isOption = lib.isType "option";
/**
Creates an Option attribute set. mkOption accepts an attribute set with the following keys:
# Inputs
Structured attribute set
: Attribute set containing none or some of the following attributes.
`default`
: Optional default value used when no definition is given in the configuration.
`defaultText`
: Substitute for documenting the `default`, if evaluating the default value during documentation rendering is not possible.
: Can be any nix value that evaluates.
: Usage with `lib.literalMD` or `lib.literalExpression` is supported
`example`
: Optional example value used in the manual.
: Can be any nix value that evaluates.
: Usage with `lib.literalMD` or `lib.literalExpression` is supported
`description`
: Optional string describing the option. This is required if option documentation is generated.
`relatedPackages`
: Optional related packages used in the manual (see `genRelatedPackages` in `../nixos/lib/make-options-doc/default.nix`).
`type`
: Optional option type, providing type-checking and value merging.
`apply`
: Optional function that converts the option value to something else.
`internal`
: Optional boolean indicating whether the option is for NixOS developers only.
`visible`
: Optional, whether the option and/or sub-options show up in the manual.
Use false to hide the option and any sub-options from submodules.
Use "shallow" to hide only sub-options.
Use "transparent" to hide this option, but not its sub-options.
Default: true.
`readOnly`
: Optional boolean indicating whether the option can be set only once.
`...` (any other attribute)
: Any other attribute is passed through to the resulting option attribute set.
# Examples
:::{.example}
## `lib.options.mkOption` usage example
```nix
mkOption { } // => { _type = "option"; }
mkOption { default = "foo"; } // => { _type = "option"; default = "foo"; }
```
:::
*/
mkOption =
{
default ? null,
defaultText ? null,
example ? null,
description ? null,
relatedPackages ? null,
type ? null,
apply ? null,
internal ? null,
visible ? null,
readOnly ? null,
}@attrs:
attrs // { _type = "option"; };
/**
Creates an option declaration with a default value of ´false´, and can be defined to ´true´.
# Inputs
`name`
: Name for the created option
# Examples
:::{.example}
## `lib.options.mkEnableOption` usage example
```nix
# module
let
eval = lib.evalModules {
modules = [
{
options.foo.enable = mkEnableOption "foo";
config.foo.enable = true;
}
];
};
in
eval.config
=> { foo.enable = true; }
```
:::
*/
mkEnableOption =
name:
mkOption {
default = false;
example = true;
description = "Whether to enable ${name}.";
type = lib.types.bool;
};
/**
Creates an Option attribute set for an option that specifies the
package a module should use for some purpose.
The package is specified in the third argument under `default` as a list of strings
representing its attribute path in nixpkgs (or another package set).
Because of this, you need to pass nixpkgs itself (usually `pkgs` in a module;
alternatively to nixpkgs itself, another package set) as the first argument.
If you pass another package set you should set the `pkgsText` option.
This option is used to display the expression for the package set. It is `"pkgs"` by default.
If your expression is complex you should parenthesize it, as the `pkgsText` argument
is usually immediately followed by an attribute lookup (`.`).
The second argument may be either a string or a list of strings.
It provides the display name of the package in the description of the generated option
(using only the last element if the passed value is a list)
and serves as the fallback value for the `default` argument.
To include extra information in the description, pass `extraDescription` to
append arbitrary text to the generated description.
You can also pass an `example` value, either a literal string or an attribute path.
The `default` argument can be omitted if the provided name is
an attribute of pkgs (if `name` is a string) or a valid attribute path in pkgs (if `name` is a list).
You can also set `default` to just a string in which case it is interpreted as an attribute name
(a singleton attribute path, if you will).
If you wish to explicitly provide no default, pass `null` as `default`.
If you want users to be able to set no package, pass `nullable = true`.
In this mode a `default = null` will not be interpreted as no default and is interpreted literally.
# Inputs
`pkgs`
: Package set (an instantiation of nixpkgs such as pkgs in modules or another package set)
`name`
: Name for the package, shown in option description
Structured function argument
: Attribute set containing the following attributes.
`nullable`
: Optional whether the package can be null, for example to disable installing a package altogether. Default: `false`
`default`
: Optional attribute path where the default package is located. Default: `name`
If omitted will be copied from `name`
`example`
: Optional string or an attribute path to use as an example. Default: `null`
`extraDescription`
: Optional additional text to include in the option description. Default: `""`
`pkgsText`
: Optional representation of the package set passed as pkgs. Default: `"pkgs"`
# Type
```
mkPackageOption :: pkgs -> (string|[string]) -> { nullable? :: bool, default? :: string|[string], example? :: null|string|[string], extraDescription? :: string, pkgsText? :: string } -> option
```
# Examples
:::{.example}
## `lib.options.mkPackageOption` usage example
```nix
mkPackageOption pkgs "hello" { }
=> { ...; default = pkgs.hello; defaultText = literalExpression "pkgs.hello"; description = "The hello package to use."; type = package; }
mkPackageOption pkgs "GHC" {
default = [ "ghc" ];
example = "pkgs.haskellPackages.ghc.withPackages (hkgs: [ hkgs.primes ])";
}
=> { ...; default = pkgs.ghc; defaultText = literalExpression "pkgs.ghc"; description = "The GHC package to use."; example = literalExpression "pkgs.haskellPackages.ghc.withPackages (hkgs: [ hkgs.primes ])"; type = package; }
mkPackageOption pkgs [ "python3Packages" "pytorch" ] {
extraDescription = "This is an example and doesn't actually do anything.";
}
=> { ...; default = pkgs.python3Packages.pytorch; defaultText = literalExpression "pkgs.python3Packages.pytorch"; description = "The pytorch package to use. This is an example and doesn't actually do anything."; type = package; }
mkPackageOption pkgs "nushell" {
nullable = true;
}
=> { ...; default = pkgs.nushell; defaultText = literalExpression "pkgs.nushell"; description = "The nushell package to use."; type = nullOr package; }
mkPackageOption pkgs "coreutils" {
default = null;
}
=> { ...; description = "The coreutils package to use."; type = package; }
mkPackageOption pkgs "dbus" {
nullable = true;
default = null;
}
=> { ...; default = null; description = "The dbus package to use."; type = nullOr package; }
mkPackageOption pkgs.javaPackages "OpenJFX" {
default = "openjfx20";
pkgsText = "pkgs.javaPackages";
}
=> { ...; default = pkgs.javaPackages.openjfx20; defaultText = literalExpression "pkgs.javaPackages.openjfx20"; description = "The OpenJFX package to use."; type = package; }
```
:::
*/
mkPackageOption =
pkgs: name:
{
nullable ? false,
default ? name,
example ? null,
extraDescription ? "",
pkgsText ? "pkgs",
}:
let
name' = if isList name then last name else name;
default' = toList default;
defaultText = showAttrPath default';
defaultValue = attrByPath default' (throw "${defaultText} cannot be found in ${pkgsText}") pkgs;
defaults =
if default != null then
{
default = defaultValue;
defaultText = literalExpression "${pkgsText}.${defaultText}";
}
else
optionalAttrs nullable {
default = null;
};
in
mkOption (
defaults
// {
description =
"The ${name'} package to use." + (if extraDescription == "" then "" else " ") + extraDescription;
type = with lib.types; (if nullable then nullOr else lib.id) package;
}
// optionalAttrs (example != null) {
example = literalExpression (
if isList example then "${pkgsText}.${showAttrPath example}" else example
);
}
);
/**
This option accepts arbitrary definitions, but it does not produce an option value.
This is useful for sharing a module across different module sets
without having to implement similar features as long as the
values of the options are not accessed.
# Inputs
`attrs`
: Attribute set whose attributes override the argument to `mkOption`.
*/
mkSinkUndeclaredOptions =
attrs:
mkOption (
{
internal = true;
visible = false;
default = false;
description = "Sink for option definitions.";
type = mkOptionType {
name = "sink";
check = x: true;
merge = loc: defs: false;
};
apply = x: throw "Option value is not readable because the option is not declared.";
}
// attrs
);
/**
A merge function that merges multiple definitions of an option into a single value
:::{.caution}
This function is used as the default merge operation in `lib.types.mkOptionType`. In most cases, explicit usage of this function is unnecessary.
:::
# Inputs
`loc`
: location of the option in the configuration as a list of strings.
e.g. `["boot" "loader "grub" "enable"]`
`defs`
: list of definition values and locations.
e.g. `[ { file = "/foo.nix"; value = 1; } { file = "/bar.nix"; value = 2 } ]`
# Example
:::{.example}
## `lib.options.mergeDefaultOption` usage example
```nix
myType = mkOptionType {
name = "myType";
merge = mergeDefaultOption; # <- This line is redundant. It is the default already.
};
```
:::
# Merge behavior
Merging requires all definition values to have the same type.
- If all definitions are booleans, the result of a `foldl'` with the `or` operation is returned.
- If all definitions are strings, they are concatenated. (`lib.concatStrings`)
- If all definitions are integers and all are equal, the first one is returned.
- If all definitions are lists, they are concatenated. (`++`)
- If all definitions are attribute sets, they are merged. (`lib.mergeAttrs`)
- If all definitions are functions, the first function is applied to the result of the second function. (`f -> x: f x`)
- Otherwise, an error is thrown.
*/
mergeDefaultOption =
loc: defs:
let
list = getValues defs;
in
if length list == 1 then
head list
else if all isFunction list then
x: mergeDefaultOption loc (map (f: f x) list)
else if all isList list then
concatLists list
else if all isAttrs list then
foldl' lib.mergeAttrs { } list
else if all isBool list then
foldl' lib.or false list
else if all isString list then
lib.concatStrings list
else if all isInt list && all (x: x == head list) list then
head list
else
throw "Cannot merge definitions of `${showOption loc}'. Definition values:${showDefs defs}";
/**
Require a single definition.
WARNING: Does not perform nested checks, as this does not run the merge function!
*/
mergeOneOption = mergeUniqueOption { message = ""; };
/**
Require a single definition.
NOTE: When the type is not checked completely by check, pass a merge function for further checking (of sub-attributes, etc).
# Inputs
`loc`
: 2\. Function argument
`defs`
: 3\. Function argument
*/
mergeUniqueOption =
args@{
message,
# WARNING: the default merge function assumes that the definition is a valid (option) value. You MUST pass a merge function if the return value needs to be
# - type checked beyond what .check does (which should be very little; only on the value head; not attribute values, etc)
# - if you want attribute values to be checked, or list items
# - if you want coercedTo-like behavior to work
merge ? loc: defs: (head defs).value,
}:
loc: defs:
if length defs == 1 then
merge loc defs
else
assert length defs > 1;
throw "The option `${showOption loc}' is defined multiple times while it's expected to be unique.\n${message}\nDefinition values:${showDefs defs}\n${prioritySuggestion}";
/**
"Merge" option definitions by checking that they all have the same value.
# Inputs
`loc`
: 1\. Function argument
`defs`
: 2\. Function argument
*/
mergeEqualOption =
loc: defs:
if defs == [ ] then
abort "This case should never happen."
# Returns early if we only have one element
# This also makes it work for functions, because the foldl' below would try
# to compare the first element with itself, which is false for functions
else if length defs == 1 then
(head defs).value
else
(foldl' (
first: def:
if def.value != first.value then
throw "The option `${showOption loc}' has conflicting definition values:${
showDefs [
first
def
]
}\n${prioritySuggestion}"
else
first
) (head defs) (tail defs)).value;
/**
Extracts values of all "value" keys of the given list.
# Type
```
getValues :: [ { value :: a; } ] -> [a]
```
# Examples
:::{.example}
## `getValues` usage example
```nix
getValues [ { value = 1; } { value = 2; } ] // => [ 1 2 ]
getValues [ ] // => [ ]
```
:::
*/
getValues = map (x: x.value);
/**
Extracts values of all "file" keys of the given list
# Type
```
getFiles :: [ { file :: a; } ] -> [a]
```
# Examples
:::{.example}
## `getFiles` usage example
```nix
getFiles [ { file = "file1"; } { file = "file2"; } ] // => [ "file1" "file2" ]
getFiles [ ] // => [ ]
```
:::
*/
getFiles = map (x: x.file);
# Generate documentation template from the list of option declaration like
# the set generated with filterOptionSets.
optionAttrSetToDocList = optionAttrSetToDocList' [ ];
optionAttrSetToDocList' =
_: options:
concatMap (
opt:
let
name = showOption opt.loc;
visible = opt.visible or true;
docOption = {
loc = opt.loc;
inherit name;
description = opt.description or null;
declarations = filter (x: x != unknownModule) opt.declarations;
internal = opt.internal or false;
visible = if isBool visible then visible else visible == "shallow";
readOnly = opt.readOnly or false;
type = opt.type.description or "unspecified";
}
// optionalAttrs (opt ? example) {
example = builtins.addErrorContext "while evaluating the example of option `${name}`" (
renderOptionValue opt.example
);
}
// optionalAttrs (opt ? defaultText || opt ? default) {
default = builtins.addErrorContext "while evaluating the ${
if opt ? defaultText then "defaultText" else "default value"
} of option `${name}`" (renderOptionValue (opt.defaultText or opt.default));
}
// optionalAttrs (opt ? relatedPackages && opt.relatedPackages != null) {
inherit (opt) relatedPackages;
};
subOptions =
let
ss = opt.type.getSubOptions opt.loc;
in
if ss != { } then optionAttrSetToDocList' opt.loc ss else [ ];
subOptionsVisible = if isBool visible then visible else visible == "transparent";
in
# To find infinite recursion in NixOS option docs:
# builtins.trace opt.loc
[ docOption ] ++ optionals subOptionsVisible subOptions
) (collect isOption options);
/**
This function recursively removes all derivation attributes from
`x` except for the `name` attribute.
This is to make the generation of `options.xml` much more
efficient: the XML representation of derivations is very large
(on the order of megabytes) and is not actually used by the
manual generator.
This function was made obsolete by renderOptionValue and is kept for
compatibility with out-of-tree code.
# Inputs
`x`
: 1\. Function argument
*/
scrubOptionValue =
x:
if isDerivation x then
{
type = "derivation";
drvPath = x.name;
outPath = x.name;
name = x.name;
}
else if isList x then
map scrubOptionValue x
else if isAttrs x then
mapAttrs (n: v: scrubOptionValue v) (removeAttrs x [ "_args" ])
else
x;
/**
Ensures that the given option value (default or example) is a `_type`d string
by rendering Nix values to `literalExpression`s.
# Inputs
`v`
: 1\. Function argument
*/
renderOptionValue =
v:
if v ? _type && v ? text then
v
else
literalExpression (
lib.generators.toPretty {
multiline = true;
allowPrettyValues = true;
} v
);
/**
For use in the `defaultText` and `example` option attributes. Causes the
given string to be rendered verbatim in the documentation as Nix code. This
is necessary for complex values, e.g. functions, or values that depend on
other values or packages.
# Inputs
`text`
: 1\. Function argument
*/
literalExpression =
text:
if !isString text then
throw "literalExpression expects a string."
else
{
_type = "literalExpression";
inherit text;
};
literalExample = lib.warn "lib.literalExample is deprecated, use lib.literalExpression instead, or use lib.literalMD for a non-Nix description." literalExpression;
/**
For use in the `defaultText` and `example` option attributes. Causes the
given MD text to be inserted verbatim in the documentation, for when
a `literalExpression` would be too hard to read.
# Inputs
`text`
: 1\. Function argument
*/
literalMD =
text:
if !isString text then
throw "literalMD expects a string."
else
{
_type = "literalMD";
inherit text;
};
# Helper functions.
/**
Convert an option, described as a list of the option parts to a
human-readable version.
# Inputs
`parts`
: 1\. Function argument
# Examples
:::{.example}
## `showOption` usage example
```nix
(showOption ["foo" "bar" "baz"]) == "foo.bar.baz"
(showOption ["foo" "bar.baz" "tux"]) == "foo.\"bar.baz\".tux"
(showOption ["windowManager" "2bwm" "enable"]) == "windowManager.\"2bwm\".enable"
Placeholders will not be quoted as they are not actual values:
(showOption ["foo" "*" "bar"]) == "foo.*.bar"
(showOption ["foo" "<name>" "bar"]) == "foo.<name>.bar"
(showOption ["foo" "<myPlaceholder>" "bar"]) == "foo.<myPlaceholder>.bar"
```
:::
*/
showOption =
parts:
let
# If the part is a named placeholder of the form "<...>" don't escape it.
# It may cause misleading escaping if somebody uses literally "<...>" in their option names.
# This is the trade-off to allow for placeholders in option names.
isNamedPlaceholder = builtins.match "<(.*)>";
escapeOptionPart =
part:
if part == "*" || isNamedPlaceholder part != null then
part
else
lib.strings.escapeNixIdentifier part;
in
(concatStringsSep ".") (map escapeOptionPart parts);
showFiles = files: concatStringsSep " and " (map (f: "`${f}'") files);
showDefs =
defs:
concatMapStrings (
def:
let
# Pretty print the value for display, if successful
prettyEval = builtins.tryEval (
lib.generators.toPretty { } (
lib.generators.withRecursion {
depthLimit = 10;
throwOnDepthLimit = false;
} def.value
)
);
# Split it into its lines
lines = filter (v: !isList v) (builtins.split "\n" prettyEval.value);
# Only display the first 5 lines, and indent them for better visibility
value = concatStringsSep "\n " (take 5 lines ++ optional (length lines > 5) "...");
result =
# Don't print any value if evaluating the value strictly fails
if !prettyEval.success then
""
# Put it on a new line if it consists of multiple
else if length lines > 1 then
":\n " + value
else
": " + value;
in
"\n- In `${def.file}'${result}"
) defs;
/**
Pretty prints all option definition locations
# Inputs
`option`
: The option to pretty print
# Examples
:::{.example}
## `lib.options.showOptionWithDefLocs` usage example
```nix
showOptionWithDefLocs { loc = ["x" "y" ]; files = [ "foo.nix" "bar.nix" ]; }
"x.y, with values defined in:\n - foo.nix\n - bar.nix\n"
```
```nix
nix-repl> eval = lib.evalModules {
modules = [
{
options = {
foo = lib.mkEnableOption "foo";
};
}
];
}
nix-repl> lib.options.showOptionWithDefLocs eval.options.foo
"foo, with values defined in:\n - <unknown-file>\n"
```
:::
# Type
```
showDefsSep :: { files :: [ String ]; loc :: [ String ]; ... } -> string
```
*/
showOptionWithDefLocs = opt: ''
${showOption opt.loc}, with values defined in:
${concatMapStringsSep "\n" (defFile: " - ${defFile}") opt.files}
'';
unknownModule = "<unknown-file>";
}

240
lib/path/README.md Normal file
View File

@@ -0,0 +1,240 @@
# Path library
This document explains why the `lib.path` library is designed the way it is.
The purpose of this library is to process [filesystem paths].
It does not read files from the filesystem.
It exists to support the native Nix [path value type] with extra functionality.
[filesystem paths]: https://en.m.wikipedia.org/wiki/Path_(computing)
[path value type]: https://nixos.org/manual/nix/stable/language/values.html#type-path
As an extension of the path value type, it inherits the same intended use cases and limitations:
- Only use paths to access files at evaluation time, such as the local project source.
- Paths cannot point to derivations, so they are unfit to represent dependencies.
- A path implicitly imports the referenced files into the Nix store when interpolated to a string.
Therefore paths are not suitable to access files at build- or run-time, as you risk importing the path from the evaluation system instead.
Overall, this library works with two types of paths:
- Absolute paths are represented with the Nix [path value type].
Nix automatically normalises these paths.
- Subpaths are represented with the [string value type] since path value types don't support relative paths.
This library normalises these paths as safely as possible.
Absolute paths in strings are not supported.
A subpath refers to a specific file or directory within an absolute base directory.
It is a stricter form of a relative path, notably [without support for `..` components][parents] since those could escape the base directory.
[string value type]: https://nixos.org/manual/nix/stable/language/values.html#type-string
This library is designed to be as safe and intuitive as possible, throwing errors when operations are attempted that would produce surprising results, and giving the expected result otherwise.
This library is designed to work well as a dependency for the `lib.filesystem` and `lib.sources` library components.
Contrary to these library components, `lib.path` does not read any paths from the filesystem.
This library makes only these assumptions about paths and no others:
- `dirOf path` returns the path to the parent directory of `path`, unless `path` is the filesystem root, in which case `path` is returned.
- There can be multiple filesystem roots: `p == dirOf p` and `q == dirOf q` does not imply `p == q`.
- While there's only a single filesystem root in stable Nix, the [lazy trees feature](https://github.com/NixOS/nix/pull/6530) introduces [additional filesystem roots](https://github.com/NixOS/nix/pull/6530#discussion_r1041442173).
- `path + ("/" + string)` returns the path to the `string` subdirectory in `path`.
- If `string` contains no `/` characters, then `dirOf (path + ("/" + string)) == path`.
- If `string` contains no `/` characters, then `baseNameOf (path + ("/" + string)) == string`.
- `path1 == path2` returns `true` only if `path1` points to the same filesystem path as `path2`.
Notably we do not make the assumption that we can turn paths into strings using `toString path`.
## Design decisions
Each subsection here contains a decision along with arguments and counter-arguments for (+) and against (-) that decision.
### Leading dots for relative paths
[leading-dots]: #leading-dots-for-relative-paths
Observing: Since subpaths are a form of relative paths, they can have a leading `./` to indicate it being a relative path, this is generally not necessary for tools though.
Considering: Paths should be as explicit, consistent and unambiguous as possible.
Decision: Returned subpaths should always have a leading `./`.
<details>
<summary>Arguments</summary>
- (+) In shells, just running `foo` as a command wouldn't execute the file `foo`, whereas `./foo` would execute the file.
In contrast, `foo/bar` does execute that file without the need for `./`.
This can lead to confusion about when a `./` needs to be prefixed.
If a `./` is always included, this becomes a non-issue.
This effectively then means that paths don't overlap with command names.
- (+) Prepending with `./` makes the subpaths always valid as relative Nix path expressions.
- (+) Using paths in command line arguments could give problems if not escaped properly, e.g. if a path was `--version`.
This is not a problem with `./--version`.
This effectively then means that paths don't overlap with GNU-style command line options.
- (-) `./` is not required to resolve relative paths, resolution always has an implicit `./` as prefix.
- (-) It's less noisy without the `./`, e.g. in error messages.
- (+) But similarly, it could be confusing whether something was even a path.
e.g. `foo` could be anything, but `./foo` is more clearly a path.
- (+) Makes it more uniform with absolute paths (those always start with `/`).
- (-) That is not relevant for practical purposes.
- (+) `find` also outputs results with `./`.
- (-) But only if you give it an argument of `.`.
If you give it the argument `some-directory`, it won't prefix that.
- (-) `realpath --relative-to` doesn't prefix relative paths with `./`.
- (+) There is no need to return the same result as `realpath`.
</details>
### Representation of the current directory
[curdir]: #representation-of-the-current-directory
Observing: The subpath that produces the base directory can be represented with `.` or `./` or `./.`.
Considering: Paths should be as consistent and unambiguous as possible.
Decision: It should be `./.`.
<details>
<summary>Arguments</summary>
- (+) `./` would be inconsistent with [the decision to not persist trailing slashes][trailing-slashes].
- (-) `.` is how `realpath` normalises paths.
- (+) `.` can be interpreted as a shell command (it's a builtin for sourcing files in `bash` and `zsh`).
- (+) `.` would be the only path without a `/`.
It could not be used as a Nix path expression, since those require at least one `/` to be parsed as such.
- (-) `./.` is rather long.
- (-) We don't require users to type this though, as it's only output by the library.
As inputs all three variants are supported for subpaths (and we can't do anything about absolute paths)
- (-) `builtins.dirOf "foo" == "."`, so `.` would be consistent with that.
- (+) `./.` is consistent with the [decision to have leading `./`][leading-dots].
- (+) `./.` is a valid Nix path expression, although this property does not hold for every relative path or subpath.
</details>
### Subpath representation
[relrepr]: #subpath-representation
Observing: Subpaths such as `foo/bar` can be represented in various ways:
- string: `"foo/bar"`
- list with all the components: `[ "foo" "bar" ]`
- attribute set: `{ type = "relative-path"; components = [ "foo" "bar" ]; }`
Considering: Paths should be as safe to use as possible.
We should generate string outputs in the library and not encourage users to do that themselves.
Decision: Paths are represented as strings.
<details>
<summary>Arguments</summary>
- (+) It's simpler for the users of the library.
One doesn't have to convert a path a string before it can be used.
- (+) Naively converting the list representation to a string with `concatStringsSep "/"` would break for `[]`, requiring library users to be more careful.
- (+) It doesn't encourage people to do their own path processing and instead use the library.
With a list representation it would seem easy to just use `lib.lists.init` to get the parent directory, but then it breaks for `.`, which would be represented as `[ ]`.
- (+) `+` is convenient and doesn't work on lists and attribute sets.
- (-) Shouldn't use `+` anyways, we export safer functions for path manipulation.
</details>
### Parent directory
[parents]: #parent-directory
Observing: Relative paths can have `..` components, which refer to the parent directory.
Considering: Paths should be as safe and unambiguous as possible.
Decision: `..` path components in string paths are not supported, neither as inputs nor as outputs.
Hence, string paths are called subpaths, rather than relative paths.
<details>
<summary>Arguments</summary>
- (+) If we wanted relative paths to behave according to the "physical" interpretation (as a directory tree with relations between nodes), it would require resolving symlinks, since e.g. `foo/..` would not be the same as `.` if `foo` is a symlink.
- (-) The "logical" interpretation is also valid (treating paths as a sequence of names), and is used by some software.
It is simpler, and not using symlinks at all is safer.
- (+) Mixing both models can lead to surprises.
- (+) We can't resolve symlinks without filesystem access.
- (+) Nix also doesn't support reading symlinks at evaluation time.
- (-) We could just not handle such cases, e.g. `equals "foo" "foo/bar/.. == false`.
The paths are different, we don't need to check whether the paths point to the same thing.
- (+) Assume we said `relativeTo /foo /bar == "../bar"`.
If this is used like `/bar/../foo` in the end, and `bar` turns out to be a symlink to somewhere else, this won't be accurate.
- (-) We could decide to not support such ambiguous operations, or mark them as such, e.g. the normal `relativeTo` will error on such a case, but there could be `extendedRelativeTo` supporting that.
- (-) `..` are a part of paths, a path library should therefore support it.
- (+) If we can convincingly argue that all such use cases are better done e.g. with runtime tools, the library not supporting it can nudge people towards using those.
- (-) We could allow "..", but only in the prefix.
- (+) Then we'd have to throw an error for doing `append /some/path "../foo"`, making it non-composable.
- (+) The same is for returning paths with `..`: `relativeTo /foo /bar => "../bar"` would produce a non-composable path.
- (+) We argue that `..` is not needed at the Nix evaluation level, since we'd always start evaluation from the project root and don't go up from there.
- (+) `..` is supported in Nix paths, turning them into absolute paths.
- (-) This is ambiguous in the presence of symlinks.
- (+) If you need `..` for building or runtime, you can use build-/run-time tooling to create those (e.g. `realpath` with `--relative-to`), or use absolute paths instead.
This also gives you the ability to correctly handle symlinks.
</details>
### Trailing slashes
[trailing-slashes]: #trailing-slashes
Observing: Subpaths can contain trailing slashes, like `foo/`, indicating that the path points to a directory and not a file.
Considering: Paths should be as consistent as possible, there should only be a single normalisation for the same path.
Decision: All functions remove trailing slashes in their results.
<details>
<summary>Arguments</summary>
- (+) It allows normalisations to be unique, in that there's only a single normalisation for the same path.
If trailing slashes were preserved, both `foo/bar` and `foo/bar/` would be valid but different normalisations for the same path.
- Comparison to other frameworks to figure out the least surprising behavior:
- (+) Nix itself doesn't support trailing slashes when parsing and doesn't preserve them when appending paths.
- (-) [Rust's std::path](https://doc.rust-lang.org/std/path/index.html) does preserve them during [construction](https://doc.rust-lang.org/std/path/struct.Path.html#method.new).
- (+) Doesn't preserve them when returning individual [components](https://doc.rust-lang.org/std/path/struct.Path.html#method.components).
- (+) Doesn't preserve them when [canonicalizing](https://doc.rust-lang.org/std/path/struct.Path.html#method.canonicalize).
- (+) [Python 3's pathlib](https://docs.python.org/3/library/pathlib.html#module-pathlib) doesn't preserve them during [construction](https://docs.python.org/3/library/pathlib.html#pathlib.PurePath).
- Notably it represents the individual components as a list internally.
- (-) [Haskell's filepath](https://hackage.haskell.org/package/filepath-1.4.100.0) has [explicit support](https://hackage.haskell.org/package/filepath-1.4.100.0/docs/System-FilePath.html#g:6) for handling trailing slashes.
- (-) Does preserve them for [normalisation](https://hackage.haskell.org/package/filepath-1.4.100.0/docs/System-FilePath.html#v:normalise).
- (-) [NodeJS's Path library](https://nodejs.org/api/path.html) preserves trailing slashes for [normalisation](https://nodejs.org/api/path.html#pathnormalizepath).
- (+) For [parsing a path](https://nodejs.org/api/path.html#pathparsepath) into its significant elements, trailing slashes are not preserved.
- (+) Nix's builtin function `dirOf` gives an unexpected result for paths with trailing slashes: `dirOf "foo/bar/" == "foo/bar"`.
Inconsistently, `baseNameOf` works correctly though: `baseNameOf "foo/bar/" == "bar"`.
- (-) We are writing a path library to improve handling of paths though, so we shouldn't use these functions and discourage their use.
- (-) Unexpected result when normalising intermediate paths, like `relative.normalise ("foo" + "/") + "bar" == "foobar"`.
- (+) This is not a practical use case though.
- (+) Don't use `+` to append paths, this library has a `join` function for that.
- (-) Users might use `+` out of habit though.
- (+) The `realpath` command also removes trailing slashes.
- (+) Even with a trailing slash, the path is the same, it's only an indication that it's a directory.
</details>
### Prefer returning subpaths over components
[subpath-preference]: #prefer-returning-subpaths-over-components
Observing: Functions could return subpaths or lists of path component strings.
Considering: Subpaths are used as inputs for some functions.
Using them for outputs, too, makes the library more consistent and composable.
Decision: Subpaths should be preferred over list of path component strings.
<details>
<summary>Arguments</summary>
- (+) It is consistent with functions accepting subpaths, making the library more composable
- (-) It is less efficient when the components are needed, because after creating the normalised subpath string, it will have to be parsed into components again
- (+) If necessary, we can still make it faster by adding builtins to Nix
- (+) Alternatively if necessary, versions of these functions that return components could later still be introduced.
- (+) It makes the path library simpler because there's only two types (paths and subpaths).
Only `lib.path.subpath.components` can be used to get a list of components.
And once we have a list of component strings, `lib.lists` and `lib.strings` can be used to operate on them.
For completeness, `lib.path.subpath.join` allows converting the list of components back to a subpath.
</details>
## Other implementations and references
- [Rust](https://doc.rust-lang.org/std/path/struct.Path.html)
- [Python](https://docs.python.org/3/library/pathlib.html)
- [Haskell](https://hackage.haskell.org/package/filepath-1.4.100.0/docs/System-FilePath.html)
- [Nodejs](https://nodejs.org/api/path.html)
- [POSIX.1-2017](https://pubs.opengroup.org/onlinepubs/9699919799/nframe.html)

807
lib/path/default.nix Normal file
View File

@@ -0,0 +1,807 @@
# Functions for working with path values.
# See ./README.md for internal docs
{ lib }:
let
inherit (builtins)
isString
isPath
split
match
typeOf
storeDir
;
inherit (lib.lists)
length
head
last
genList
elemAt
all
concatMap
foldl'
take
drop
;
listHasPrefix = lib.lists.hasPrefix;
inherit (lib.strings)
concatStringsSep
substring
;
inherit (lib.asserts)
assertMsg
;
inherit (lib.path.subpath)
isValid
;
# Returns the reason why a subpath is invalid, or `null` if it's valid
subpathInvalidReason =
value:
if !isString value then
"The given value is of type ${builtins.typeOf value}, but a string was expected"
else if value == "" then
"The given string is empty"
else if substring 0 1 value == "/" then
"The given string \"${value}\" starts with a `/`, representing an absolute path"
# We don't support ".." components, see ./path.md#parent-directory
else if match "(.*/)?\\.\\.(/.*)?" value != null then
"The given string \"${value}\" contains a `..` component, which is not allowed in subpaths"
else
null;
# Split and normalise a relative path string into its components.
# Error for ".." components and doesn't include "." components
splitRelPath =
path:
let
# Split the string into its parts using regex for efficiency. This regex
# matches patterns like "/", "/./", "/././", with arbitrarily many "/"s
# together. These are the main special cases:
# - Leading "./" gets split into a leading "." part
# - Trailing "/." or "/" get split into a trailing "." or ""
# part respectively
#
# These are the only cases where "." and "" parts can occur
parts = split "/+(\\./+)*" path;
# `split` creates a list of 2 * k + 1 elements, containing the k +
# 1 parts, interleaved with k matches where k is the number of
# (non-overlapping) matches. This calculation here gets the number of parts
# back from the list length
# floor( (2 * k + 1) / 2 ) + 1 == floor( k + 1/2 ) + 1 == k + 1
partCount = length parts / 2 + 1;
# To assemble the final list of components we want to:
# - Skip a potential leading ".", normalising "./foo" to "foo"
# - Skip a potential trailing "." or "", normalising "foo/" and "foo/." to
# "foo". See ./path.md#trailing-slashes
skipStart = if head parts == "." then 1 else 0;
skipEnd = if last parts == "." || last parts == "" then 1 else 0;
# We can now know the length of the result by removing the number of
# skipped parts from the total number
componentCount = partCount - skipEnd - skipStart;
in
# Special case of a single "." path component. Such a case leaves a
# componentCount of -1 due to the skipStart/skipEnd not verifying that
# they don't refer to the same character
if path == "." then
[ ]
# Generate the result list directly. This is more efficient than a
# combination of `filter`, `init` and `tail`, because here we don't
# allocate any intermediate lists
else
genList (
index:
# To get to the element we need to add the number of parts we skip and
# multiply by two due to the interleaved layout of `parts`
elemAt parts ((skipStart + index) * 2)
) componentCount;
# Join relative path components together
joinRelPath =
components:
# Always return relative paths with `./` as a prefix (./path.md#leading-dots-for-relative-paths)
"./"
+
# An empty string is not a valid relative path, so we need to return a `.` when we have no components
(if components == [ ] then "." else concatStringsSep "/" components);
# Type: Path -> { root :: Path, components :: [ String ] }
#
# Deconstruct a path value type into:
# - root: The filesystem root of the path, generally `/`
# - components: All the path's components
#
# This is similar to `splitString "/" (toString path)` but safer
# because it can distinguish different filesystem roots
deconstructPath =
let
recurse =
components: base:
# If the parent of a path is the path itself, then it's a filesystem root
if base == dirOf base then
{
root = base;
inherit components;
}
else
recurse ([ (baseNameOf base) ] ++ components) (dirOf base);
in
recurse [ ];
# The components of the store directory, typically [ "nix" "store" ]
storeDirComponents = splitRelPath ("./" + storeDir);
# The number of store directory components, typically 2
storeDirLength = length storeDirComponents;
# Type: [ String ] -> Bool
#
# Whether path components have a store path as a prefix, according to
# https://nixos.org/manual/nix/stable/store/store-path.html#store-path.
componentsHaveStorePathPrefix =
components:
# path starts with the store directory (typically /nix/store)
listHasPrefix storeDirComponents components
# is not the store directory itself, meaning there's at least one extra component
&& storeDirComponents != components
# and the first component after the store directory has the expected format.
# NOTE: We could change the hash regex to be [0-9a-df-np-sv-z],
# because these are the actual ASCII characters used by Nix's base32 implementation,
# but this is not fully specified, so let's tie this too much to the currently implemented concept of store paths.
# Similar reasoning applies to the validity of the name part.
# We care more about discerning store path-ness on realistic values. Making it airtight would be fragile and slow.
&& match ".{32}-.+" (elemAt components storeDirLength) != null
# alternatively match content‐addressed derivations, which _currently_ do
# not have a store directory prefix.
# This is a workaround for https://github.com/NixOS/nix/issues/12361 which
# was needed during the experimental phase of ca-derivations and should be
# removed once the issue has been resolved.
|| components != [ ] && match "[0-9a-z]{52}" (head components) != null;
in
# No rec! Add dependencies on this file at the top.
{
/**
Append a subpath string to a path.
Like `path + ("/" + string)` but safer, because it errors instead of returning potentially surprising results.
More specifically, it checks that the first argument is a [path value type](https://nixos.org/manual/nix/stable/language/values.html#type-path"),
and that the second argument is a [valid subpath string](#function-library-lib.path.subpath.isValid).
Laws:
- Not influenced by subpath [normalisation](#function-library-lib.path.subpath.normalise):
append p s == append p (subpath.normalise s)
# Inputs
`path`
: The absolute path to append to
`subpath`
: The subpath string to append
# Type
```
append :: Path -> String -> Path
```
# Examples
:::{.example}
## `append` usage example
```nix
append /foo "bar/baz"
=> /foo/bar/baz
# subpaths don't need to be normalised
append /foo "./bar//baz/./"
=> /foo/bar/baz
# can append to root directory
append /. "foo/bar"
=> /foo/bar
# first argument needs to be a path value type
append "/foo" "bar"
=> <error>
# second argument needs to be a valid subpath string
append /foo /bar
=> <error>
append /foo ""
=> <error>
append /foo "/bar"
=> <error>
append /foo "../bar"
=> <error>
```
:::
*/
append =
# The absolute path to append to
path:
# The subpath string to append
subpath:
assert assertMsg (isPath path)
''lib.path.append: The first argument is of type ${builtins.typeOf path}, but a path was expected'';
assert assertMsg (isValid subpath) ''
lib.path.append: Second argument is not a valid subpath string:
${subpathInvalidReason subpath}'';
path + ("/" + subpath);
/**
Whether the first path is a component-wise prefix of the second path.
Laws:
- `hasPrefix p q` is only true if [`q == append p s`](#function-library-lib.path.append) for some [subpath](#function-library-lib.path.subpath.isValid) `s`.
- `hasPrefix` is a [non-strict partial order](https://en.wikipedia.org/wiki/Partially_ordered_set#Non-strict_partial_order) over the set of all path values.
# Inputs
`path1`
: 1\. Function argument
# Type
```
hasPrefix :: Path -> Path -> Bool
```
# Examples
:::{.example}
## `hasPrefix` usage example
```nix
hasPrefix /foo /foo/bar
=> true
hasPrefix /foo /foo
=> true
hasPrefix /foo/bar /foo
=> false
hasPrefix /. /foo
=> true
```
:::
*/
hasPrefix =
path1:
assert assertMsg (isPath path1)
"lib.path.hasPrefix: First argument is of type ${typeOf path1}, but a path was expected";
let
path1Deconstructed = deconstructPath path1;
in
path2:
assert assertMsg (isPath path2)
"lib.path.hasPrefix: Second argument is of type ${typeOf path2}, but a path was expected";
let
path2Deconstructed = deconstructPath path2;
in
assert assertMsg (path1Deconstructed.root == path2Deconstructed.root) ''
lib.path.hasPrefix: Filesystem roots must be the same for both paths, but paths with different roots were given:
first argument: "${toString path1}" with root "${toString path1Deconstructed.root}"
second argument: "${toString path2}" with root "${toString path2Deconstructed.root}"'';
take (length path1Deconstructed.components) path2Deconstructed.components
== path1Deconstructed.components;
/**
Remove the first path as a component-wise prefix from the second path.
The result is a [normalised subpath string](#function-library-lib.path.subpath.normalise).
Laws:
- Inverts [`append`](#function-library-lib.path.append) for [normalised subpath string](#function-library-lib.path.subpath.normalise):
removePrefix p (append p s) == subpath.normalise s
# Inputs
`path1`
: 1\. Function argument
# Type
```
removePrefix :: Path -> Path -> String
```
# Examples
:::{.example}
## `removePrefix` usage example
```nix
removePrefix /foo /foo/bar/baz
=> "./bar/baz"
removePrefix /foo /foo
=> "./."
removePrefix /foo/bar /foo
=> <error>
removePrefix /. /foo
=> "./foo"
```
:::
*/
removePrefix =
path1:
assert assertMsg (isPath path1)
"lib.path.removePrefix: First argument is of type ${typeOf path1}, but a path was expected.";
let
path1Deconstructed = deconstructPath path1;
path1Length = length path1Deconstructed.components;
in
path2:
assert assertMsg (isPath path2)
"lib.path.removePrefix: Second argument is of type ${typeOf path2}, but a path was expected.";
let
path2Deconstructed = deconstructPath path2;
success = take path1Length path2Deconstructed.components == path1Deconstructed.components;
components =
if success then
drop path1Length path2Deconstructed.components
else
throw ''lib.path.removePrefix: The first path argument "${toString path1}" is not a component-wise prefix of the second path argument "${toString path2}".'';
in
assert assertMsg (path1Deconstructed.root == path2Deconstructed.root) ''
lib.path.removePrefix: Filesystem roots must be the same for both paths, but paths with different roots were given:
first argument: "${toString path1}" with root "${toString path1Deconstructed.root}"
second argument: "${toString path2}" with root "${toString path2Deconstructed.root}"'';
joinRelPath components;
/**
Split the filesystem root from a [path](https://nixos.org/manual/nix/stable/language/values.html#type-path).
The result is an attribute set with these attributes:
- `root`: The filesystem root of the path, meaning that this directory has no parent directory.
- `subpath`: The [normalised subpath string](#function-library-lib.path.subpath.normalise) that when [appended](#function-library-lib.path.append) to `root` returns the original path.
Laws:
- [Appending](#function-library-lib.path.append) the `root` and `subpath` gives the original path:
p ==
append
(splitRoot p).root
(splitRoot p).subpath
- Trying to get the parent directory of `root` using [`dirOf`](https://nixos.org/manual/nix/stable/language/builtins.html#builtins-dirOf) returns `root` itself:
dirOf (splitRoot p).root == (splitRoot p).root
# Inputs
`path`
: The path to split the root off of
# Type
```
splitRoot :: Path -> { root :: Path, subpath :: String }
```
# Examples
:::{.example}
## `splitRoot` usage example
```nix
splitRoot /foo/bar
=> { root = /.; subpath = "./foo/bar"; }
splitRoot /.
=> { root = /.; subpath = "./."; }
# Nix neutralises `..` path components for all path values automatically
splitRoot /foo/../bar
=> { root = /.; subpath = "./bar"; }
splitRoot "/foo/bar"
=> <error>
```
:::
*/
splitRoot =
# The path to split the root off of
path:
assert assertMsg (isPath path)
"lib.path.splitRoot: Argument is of type ${typeOf path}, but a path was expected";
let
deconstructed = deconstructPath path;
in
{
root = deconstructed.root;
subpath = joinRelPath deconstructed.components;
};
/**
Whether a [path](https://nixos.org/manual/nix/stable/language/values.html#type-path)
has a [store path](https://nixos.org/manual/nix/stable/store/store-path.html#store-path)
as a prefix.
:::{.note}
As with all functions of this `lib.path` library, it does not work on paths in strings,
which is how you'd typically get store paths.
Instead, this function only handles path values themselves,
which occur when Nix files in the store use relative path expressions.
:::
# Inputs
`path`
: 1\. Function argument
# Type
```
hasStorePathPrefix :: Path -> Bool
```
# Examples
:::{.example}
## `hasStorePathPrefix` usage example
```nix
# Subpaths of derivation outputs have a store path as a prefix
hasStorePathPrefix /nix/store/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo/bar/baz
=> true
# The store directory itself is not a store path
hasStorePathPrefix /nix/store
=> false
# Derivation outputs are store paths themselves
hasStorePathPrefix /nix/store/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo
=> true
# Paths outside the Nix store don't have a store path prefix
hasStorePathPrefix /home/user
=> false
# Not all paths under the Nix store are store paths
hasStorePathPrefix /nix/store/.links/10gg8k3rmbw8p7gszarbk7qyd9jwxhcfq9i6s5i0qikx8alkk4hq
=> false
# Store derivations are also store paths themselves
hasStorePathPrefix /nix/store/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo.drv
=> true
```
:::
*/
hasStorePathPrefix =
path:
let
deconstructed = deconstructPath path;
in
assert assertMsg (isPath path)
"lib.path.hasStorePathPrefix: Argument is of type ${typeOf path}, but a path was expected";
assert assertMsg
# This function likely breaks or needs adjustment if used with other filesystem roots, if they ever get implemented.
# Let's try to error nicely in such a case, though it's unclear how an implementation would work even and whether this could be detected.
# See also https://github.com/NixOS/nix/pull/6530#discussion_r1422843117
(deconstructed.root == /. && toString deconstructed.root == "/")
"lib.path.hasStorePathPrefix: Argument has a filesystem root (${toString deconstructed.root}) that's not /, which is currently not supported.";
componentsHaveStorePathPrefix deconstructed.components;
/**
Whether a value is a valid subpath string.
A subpath string points to a specific file or directory within an absolute base directory.
It is a stricter form of a relative path that excludes `..` components, since those could escape the base directory.
- The value is a string.
- The string is not empty.
- The string doesn't start with a `/`.
- The string doesn't contain any `..` path components.
# Inputs
`value`
: The value to check
# Type
```
subpath.isValid :: String -> Bool
```
# Examples
:::{.example}
## `subpath.isValid` usage example
```nix
# Not a string
subpath.isValid null
=> false
# Empty string
subpath.isValid ""
=> false
# Absolute path
subpath.isValid "/foo"
=> false
# Contains a `..` path component
subpath.isValid "../foo"
=> false
# Valid subpath
subpath.isValid "foo/bar"
=> true
# Doesn't need to be normalised
subpath.isValid "./foo//bar/"
=> true
```
:::
*/
subpath.isValid =
# The value to check
value: subpathInvalidReason value == null;
/**
Join subpath strings together using `/`, returning a normalised subpath string.
Like `concatStringsSep "/"` but safer, specifically:
- All elements must be [valid subpath strings](#function-library-lib.path.subpath.isValid).
- The result gets [normalised](#function-library-lib.path.subpath.normalise).
- The edge case of an empty list gets properly handled by returning the neutral subpath `"./."`.
Laws:
- Associativity:
subpath.join [ x (subpath.join [ y z ]) ] == subpath.join [ (subpath.join [ x y ]) z ]
- Identity - `"./."` is the neutral element for normalised paths:
subpath.join [ ] == "./."
subpath.join [ (subpath.normalise p) "./." ] == subpath.normalise p
subpath.join [ "./." (subpath.normalise p) ] == subpath.normalise p
- Normalisation - the result is [normalised](#function-library-lib.path.subpath.normalise):
subpath.join ps == subpath.normalise (subpath.join ps)
- For non-empty lists, the implementation is equivalent to [normalising](#function-library-lib.path.subpath.normalise) the result of `concatStringsSep "/"`.
Note that the above laws can be derived from this one:
ps != [] -> subpath.join ps == subpath.normalise (concatStringsSep "/" ps)
# Inputs
`subpaths`
: The list of subpaths to join together
# Type
```
subpath.join :: [ String ] -> String
```
# Examples
:::{.example}
## `subpath.join` usage example
```nix
subpath.join [ "foo" "bar/baz" ]
=> "./foo/bar/baz"
# normalise the result
subpath.join [ "./foo" "." "bar//./baz/" ]
=> "./foo/bar/baz"
# passing an empty list results in the current directory
subpath.join [ ]
=> "./."
# elements must be valid subpath strings
subpath.join [ /foo ]
=> <error>
subpath.join [ "" ]
=> <error>
subpath.join [ "/foo" ]
=> <error>
subpath.join [ "../foo" ]
=> <error>
```
:::
*/
subpath.join =
# The list of subpaths to join together
subpaths:
# Fast in case all paths are valid
if all isValid subpaths then
joinRelPath (concatMap splitRelPath subpaths)
else
# Otherwise we take our time to gather more info for a better error message
# Strictly go through each path, throwing on the first invalid one
# Tracks the list index in the fold accumulator
foldl' (
i: path:
if isValid path then
i + 1
else
throw ''
lib.path.subpath.join: Element at index ${toString i} is not a valid subpath string:
${subpathInvalidReason path}''
) 0 subpaths;
/**
Split [a subpath](#function-library-lib.path.subpath.isValid) into its path component strings.
Throw an error if the subpath isn't valid.
Note that the returned path components are also [valid subpath strings](#function-library-lib.path.subpath.isValid), though they are intentionally not [normalised](#function-library-lib.path.subpath.normalise).
Laws:
- Splitting a subpath into components and [joining](#function-library-lib.path.subpath.join) the components gives the same subpath but [normalised](#function-library-lib.path.subpath.normalise):
subpath.join (subpath.components s) == subpath.normalise s
# Inputs
`subpath`
: The subpath string to split into components
# Type
```
subpath.components :: String -> [ String ]
```
# Examples
:::{.example}
## `subpath.components` usage example
```nix
subpath.components "."
=> [ ]
subpath.components "./foo//bar/./baz/"
=> [ "foo" "bar" "baz" ]
subpath.components "/foo"
=> <error>
```
:::
*/
subpath.components =
# The subpath string to split into components
subpath:
assert assertMsg (isValid subpath) ''
lib.path.subpath.components: Argument is not a valid subpath string:
${subpathInvalidReason subpath}'';
splitRelPath subpath;
/**
Normalise a subpath. Throw an error if the subpath isn't [valid](#function-library-lib.path.subpath.isValid).
- Limit repeating `/` to a single one.
- Remove redundant `.` components.
- Remove trailing `/` and `/.`.
- Add leading `./`.
Laws:
- Idempotency - normalising multiple times gives the same result:
subpath.normalise (subpath.normalise p) == subpath.normalise p
- Uniqueness - there's only a single normalisation for the paths that lead to the same file system node:
subpath.normalise p != subpath.normalise q -> $(realpath ${p}) != $(realpath ${q})
- Don't change the result when [appended](#function-library-lib.path.append) to a Nix path value:
append base p == append base (subpath.normalise p)
- Don't change the path according to `realpath`:
$(realpath ${p}) == $(realpath ${subpath.normalise p})
- Only error on [invalid subpaths](#function-library-lib.path.subpath.isValid):
builtins.tryEval (subpath.normalise p)).success == subpath.isValid p
# Inputs
`subpath`
: The subpath string to normalise
# Type
```
subpath.normalise :: String -> String
```
# Examples
:::{.example}
## `subpath.normalise` usage example
```nix
# limit repeating `/` to a single one
subpath.normalise "foo//bar"
=> "./foo/bar"
# remove redundant `.` components
subpath.normalise "foo/./bar"
=> "./foo/bar"
# add leading `./`
subpath.normalise "foo/bar"
=> "./foo/bar"
# remove trailing `/`
subpath.normalise "foo/bar/"
=> "./foo/bar"
# remove trailing `/.`
subpath.normalise "foo/bar/."
=> "./foo/bar"
# Return the current directory as `./.`
subpath.normalise "."
=> "./."
# error on `..` path components
subpath.normalise "foo/../bar"
=> <error>
# error on empty string
subpath.normalise ""
=> <error>
# error on absolute path
subpath.normalise "/foo"
=> <error>
```
:::
*/
subpath.normalise =
# The subpath string to normalise
subpath:
assert assertMsg (isValid subpath) ''
lib.path.subpath.normalise: Argument is not a valid subpath string:
${subpathInvalidReason subpath}'';
joinRelPath (splitRelPath subpath);
}

48
lib/path/tests/default.nix Normal file
View File

@@ -0,0 +1,48 @@
{
nixpkgs ? ../../..,
system ? builtins.currentSystem,
pkgs ? import nixpkgs {
config = { };
overlays = [ ];
inherit system;
},
nixVersions ? import ../../tests/nix-for-tests.nix { inherit pkgs; },
libpath ? ../..,
# Random seed
seed ? null,
}:
pkgs.runCommand "lib-path-tests"
{
nativeBuildInputs = [
nixVersions.stable
]
++ (with pkgs; [
jq
bc
]);
}
''
# Needed to make Nix evaluation work
export TEST_ROOT=$(pwd)/test-tmp
export NIX_BUILD_HOOK=
export NIX_CONF_DIR=$TEST_ROOT/etc
export NIX_LOCALSTATE_DIR=$TEST_ROOT/var
export NIX_LOG_DIR=$TEST_ROOT/var/log/nix
export NIX_STATE_DIR=$TEST_ROOT/var/nix
export NIX_STORE_DIR=$TEST_ROOT/store
export PAGER=cat
cp -r ${libpath} lib
export TEST_LIB=$PWD/lib
echo "Running unit tests lib/path/tests/unit.nix"
nix-instantiate --eval --show-trace \
--argstr libpath "$TEST_LIB" \
lib/path/tests/unit.nix
echo "Running property tests lib/path/tests/prop.sh"
bash lib/path/tests/prop.sh ${toString seed}
touch $out
''

64
lib/path/tests/generate.awk Normal file
View File

@@ -0,0 +1,64 @@
# Generate random path-like strings, separated by null characters.
#
# Invocation:
#
# awk -f ./generate.awk -v <variable>=<value> | tr '\0' '\n'
#
# Customizable variables (all default to 0):
# - seed: Deterministic random seed to use for generation
# - count: Number of paths to generate
# - extradotweight: Give extra weight to dots being generated
# - extraslashweight: Give extra weight to slashes being generated
# - extranullweight: Give extra weight to null being generated, making paths shorter
BEGIN {
# Random seed, passed explicitly for reproducibility
srand(seed)
# Don't include special characters below 32
minascii = 32
# Don't include DEL at 128
maxascii = 127
upperascii = maxascii - minascii
# add extra weight for ., in addition to the one weight from the ascii range
upperdot = upperascii + extradotweight
# add extra weight for /, in addition to the one weight from the ascii range
upperslash = upperdot + extraslashweight
# add extra weight for null, indicating the end of the string
# Must be at least 1 to have strings end at all
total = upperslash + 1 + extranullweight
# new=1 indicates that it's a new string
new=1
while (count > 0) {
# Random integer between [0, total)
value = int(rand() * total)
if (value < upperascii) {
# Ascii range
printf("%c", value + minascii)
new=0
} else if (value < upperdot) {
# Dot range
printf "."
new=0
} else if (value < upperslash) {
# If it's the start of a new path, only generate a / in 10% of cases
# This is always an invalid subpath, which is not a very interesting case
if (new && rand() > 0.1) continue
printf "/"
} else {
# Do not generate empty strings
if (new) continue
printf "\x00"
count--
new=1
}
}
}

60
lib/path/tests/prop.nix Normal file
View File

@@ -0,0 +1,60 @@
# Given a list of path-like strings, check some properties of the path library
# using those paths and return a list of attribute sets of the following form:
#
# { <string> = <lib.path.subpath.normalise string>; }
#
# If `normalise` fails to evaluate, the attribute value is set to `""`.
# If not, the resulting value is normalised again and an appropriate attribute set added to the output list.
{
# The path to the nixpkgs lib to use
libpath,
# A flat directory containing files with randomly-generated
# path-like values
dir,
}:
let
lib = import libpath;
# read each file into a string
strings = map (name: builtins.readFile (dir + "/${name}")) (
builtins.attrNames (builtins.readDir dir)
);
inherit (lib.path.subpath) normalise isValid;
inherit (lib.asserts) assertMsg;
normaliseAndCheck =
str:
let
originalValid = isValid str;
tryOnce = builtins.tryEval (normalise str);
tryTwice = builtins.tryEval (normalise tryOnce.value);
absConcatOrig = /. + ("/" + str);
absConcatNormalised = /. + ("/" + tryOnce.value);
in
# Check the lib.path.subpath.normalise property to only error on invalid subpaths
assert assertMsg (
originalValid -> tryOnce.success
) "Even though string \"${str}\" is valid as a subpath, the normalisation for it failed";
assert assertMsg (
!originalValid -> !tryOnce.success
) "Even though string \"${str}\" is invalid as a subpath, the normalisation for it succeeded";
# Check normalisation idempotency
assert assertMsg (
originalValid -> tryTwice.success
) "For valid subpath \"${str}\", the normalisation \"${tryOnce.value}\" was not a valid subpath";
assert assertMsg (originalValid -> tryOnce.value == tryTwice.value)
"For valid subpath \"${str}\", normalising it once gives \"${tryOnce.value}\" but normalising it twice gives a different result: \"${tryTwice.value}\"";
# Check that normalisation doesn't change a string when appended to an absolute Nix path value
assert assertMsg (originalValid -> absConcatOrig == absConcatNormalised)
"For valid subpath \"${str}\", appending to an absolute Nix path value gives \"${absConcatOrig}\", but appending the normalised result \"${tryOnce.value}\" gives a different value \"${absConcatNormalised}\"";
# Return an empty string when failed
if tryOnce.success then tryOnce.value else "";
in
lib.genAttrs strings normaliseAndCheck

182
lib/path/tests/prop.sh Executable file
View File

@@ -0,0 +1,182 @@
#!/usr/bin/env bash
# Property tests for lib/path/default.nix
# It generates random path-like strings and runs the functions on
# them, checking that the expected laws of the functions hold
# Run:
# [nixpkgs]$ lib/path/tests/prop.sh
# or:
# [nixpkgs]$ nix-build lib/tests/release.nix
set -euo pipefail
shopt -s inherit_errexit
# https://stackoverflow.com/a/246128
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
if test -z "${TEST_LIB:-}"; then
TEST_LIB=$SCRIPT_DIR/../..
fi
tmp="$(mktemp -d)"
clean_up() {
rm -rf "$tmp"
}
trap clean_up EXIT
mkdir -p "$tmp/work"
cd "$tmp/work"
# Defaulting to a random seed but the first argument can override this
seed=${1:-$RANDOM}
echo >&2 "Using seed $seed, use \`lib/path/tests/prop.sh $seed\` to reproduce this result"
# The number of random paths to generate. This specific number was chosen to
# be fast enough while still generating enough variety to detect bugs.
count=500
debug=0
# debug=1 # print some extra info
# debug=2 # print generated values
# Fine tuning parameters to balance the number of generated invalid paths
# to the variance in generated paths.
extradotweight=64 # Larger value: more dots
extraslashweight=64 # Larger value: more slashes
extranullweight=16 # Larger value: shorter strings
die() {
echo >&2 "test case failed: " "$@"
exit 1
}
if [[ "$debug" -ge 1 ]]; then
echo >&2 "Generating $count random path-like strings"
fi
# Read stream of null-terminated strings entry-by-entry into bash,
# write it to a file and the `strings` array.
declare -a strings=()
mkdir -p "$tmp/strings"
while IFS= read -r -d $'\0' str; do
printf "%s" "$str" > "$tmp/strings/${#strings[@]}"
strings+=("$str")
done < <(awk \
-f "$SCRIPT_DIR"/generate.awk \
-v seed="$seed" \
-v count="$count" \
-v extradotweight="$extradotweight" \
-v extraslashweight="$extraslashweight" \
-v extranullweight="$extranullweight")
if [[ "$debug" -ge 1 ]]; then
echo >&2 "Trying to normalise the generated path-like strings with Nix"
fi
# Precalculate all normalisations with a single Nix call. Calling Nix for each
# string individually would take way too long
nix-instantiate --eval --strict --json --show-trace \
--argstr libpath "$TEST_LIB" \
--argstr dir "$tmp/strings" \
"$SCRIPT_DIR"/prop.nix \
>"$tmp/result.json"
# Uses some jq magic to turn the resulting attribute set into an associative
# bash array assignment
declare -A normalised_result="($(jq '
to_entries
| map("[\(.key | @sh)]=\(.value | @sh)")
| join(" \n")' -r < "$tmp/result.json"))"
# Looks up a normalisation result for a string
# Checks that the normalisation is only failing iff it's an invalid subpath
# For valid subpaths, returns 0 and prints the normalisation result
# For invalid subpaths, returns 1
normalise() {
local str=$1
# Uses the same check for validity as in the library implementation
if [[ "$str" == "" || "$str" == /* || "$str" =~ ^(.*/)?\.\.(/.*)?$ ]]; then
valid=
else
valid=1
fi
normalised=${normalised_result[$str]}
# An empty string indicates failure, this is encoded in ./prop.nix
if [[ -n "$normalised" ]]; then
if [[ -n "$valid" ]]; then
echo "$normalised"
else
die "For invalid subpath \"$str\", lib.path.subpath.normalise returned this result: \"$normalised\""
fi
else
if [[ -n "$valid" ]]; then
die "For valid subpath \"$str\", lib.path.subpath.normalise failed"
else
if [[ "$debug" -ge 2 ]]; then
echo >&2 "String \"$str\" is not a valid subpath"
fi
# Invalid and it correctly failed, we let the caller continue if they catch the exit code
return 1
fi
fi
}
# Intermediate result populated by test_idempotency_realpath
# and used in test_normalise_uniqueness
#
# Contains a mapping from a normalised subpath to the realpath result it represents
declare -A norm_to_real
test_idempotency_realpath() {
if [[ "$debug" -ge 1 ]]; then
echo >&2 "Checking idempotency of each result and making sure the realpath result isn't changed"
fi
# Count invalid subpaths to display stats
invalid=0
for str in "${strings[@]}"; do
if ! result=$(normalise "$str"); then
((invalid++)) || true
continue
fi
# Check the law that it doesn't change the result of a realpath
mkdir -p -- "$str" "$result"
real_orig=$(realpath -- "$str")
real_norm=$(realpath -- "$result")
if [[ "$real_orig" != "$real_norm" ]]; then
die "realpath of the original string \"$str\" (\"$real_orig\") is not the same as realpath of the normalisation \"$result\" (\"$real_norm\")"
fi
if [[ "$debug" -ge 2 ]]; then
echo >&2 "String \"$str\" gets normalised to \"$result\" and file path \"$real_orig\""
fi
norm_to_real["$result"]="$real_orig"
done
if [[ "$debug" -ge 1 ]]; then
echo >&2 "$(bc <<< "scale=1; 100 / $count * $invalid")% of the total $count generated strings were invalid subpath strings, and were therefore ignored"
fi
}
test_normalise_uniqueness() {
if [[ "$debug" -ge 1 ]]; then
echo >&2 "Checking for the uniqueness law"
fi
for norm_p in "${!norm_to_real[@]}"; do
real_p=${norm_to_real["$norm_p"]}
for norm_q in "${!norm_to_real[@]}"; do
real_q=${norm_to_real["$norm_q"]}
# Checks normalisation uniqueness law for each pair of values
if [[ "$norm_p" != "$norm_q" && "$real_p" == "$real_q" ]]; then
die "Normalisations \"$norm_p\" and \"$norm_q\" are different, but the realpath of them is the same: \"$real_p\""
fi
done
done
}
test_idempotency_realpath
test_normalise_uniqueness
echo >&2 tests ok

332
lib/path/tests/unit.nix Normal file
View File

@@ -0,0 +1,332 @@
# Unit tests for lib.path functions. Use `nix-build` in this directory to
# run these
{ libpath }:
let
lib = import libpath;
inherit (lib.path)
hasPrefix
removePrefix
append
splitRoot
hasStorePathPrefix
subpath
;
# This is not allowed generally, but we're in the tests here, so we'll allow ourselves.
storeDirPath = /. + builtins.storeDir;
cases = lib.runTests {
# Test examples from the lib.path.append documentation
testAppendExample1 = {
expr = append /foo "bar/baz";
expected = /foo/bar/baz;
};
testAppendExample2 = {
expr = append /foo "./bar//baz/./";
expected = /foo/bar/baz;
};
testAppendExample3 = {
expr = append /. "foo/bar";
expected = /foo/bar;
};
testAppendExample4 = {
expr = (builtins.tryEval (append "/foo" "bar")).success;
expected = false;
};
testAppendExample5 = {
expr = (builtins.tryEval (append /foo /bar)).success;
expected = false;
};
testAppendExample6 = {
expr = (builtins.tryEval (append /foo "")).success;
expected = false;
};
testAppendExample7 = {
expr = (builtins.tryEval (append /foo "/bar")).success;
expected = false;
};
testAppendExample8 = {
expr = (builtins.tryEval (append /foo "../bar")).success;
expected = false;
};
testHasPrefixExample1 = {
expr = hasPrefix /foo /foo/bar;
expected = true;
};
testHasPrefixExample2 = {
expr = hasPrefix /foo /foo;
expected = true;
};
testHasPrefixExample3 = {
expr = hasPrefix /foo/bar /foo;
expected = false;
};
testHasPrefixExample4 = {
expr = hasPrefix /. /foo;
expected = true;
};
testRemovePrefixExample1 = {
expr = removePrefix /foo /foo/bar/baz;
expected = "./bar/baz";
};
testRemovePrefixExample2 = {
expr = removePrefix /foo /foo;
expected = "./.";
};
testRemovePrefixExample3 = {
expr = (builtins.tryEval (removePrefix /foo/bar /foo)).success;
expected = false;
};
testRemovePrefixExample4 = {
expr = removePrefix /. /foo;
expected = "./foo";
};
testSplitRootExample1 = {
expr = splitRoot /foo/bar;
expected = {
root = /.;
subpath = "./foo/bar";
};
};
testSplitRootExample2 = {
expr = splitRoot /.;
expected = {
root = /.;
subpath = "./.";
};
};
testSplitRootExample3 = {
expr = splitRoot /foo/../bar;
expected = {
root = /.;
subpath = "./bar";
};
};
testSplitRootExample4 = {
expr = (builtins.tryEval (splitRoot "/foo/bar")).success;
expected = false;
};
# Root path (empty path components list)
testHasStorePathPrefixRoot = {
expr = hasStorePathPrefix /.;
expected = false;
};
testHasStorePathPrefixExample1 = {
expr = hasStorePathPrefix (storeDirPath + "/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo/bar/baz");
expected = true;
};
testHasStorePathPrefixExample2 = {
expr = hasStorePathPrefix storeDirPath;
expected = false;
};
testHasStorePathPrefixExample3 = {
expr = hasStorePathPrefix (storeDirPath + "/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo");
expected = true;
};
testHasStorePathPrefixExample4 = {
expr = hasStorePathPrefix /home/user;
expected = false;
};
testHasStorePathPrefixExample5 = {
expr = hasStorePathPrefix (
storeDirPath + "/.links/10gg8k3rmbw8p7gszarbk7qyd9jwxhcfq9i6s5i0qikx8alkk4hq"
);
expected = false;
};
testHasStorePathPrefixExample6 = {
expr = hasStorePathPrefix (storeDirPath + "/nvl9ic0pj1fpyln3zaqrf4cclbqdfn1j-foo.drv");
expected = true;
};
# Test paths for content‐addressed derivations
testHasStorePathPrefixExample7 = {
expr = hasStorePathPrefix (/. + "/1121rp0gvr1qya7hvy925g5kjwg66acz6sn1ra1hca09f1z5dsab");
expected = true;
};
testHasStorePathPrefixExample8 = {
expr = hasStorePathPrefix (/. + "/1121rp0gvr1qya7hvy925g5kjwg66acz6sn1ra1hca09f1z5dsab/foo/bar");
expected = true;
};
# Test examples from the lib.path.subpath.isValid documentation
testSubpathIsValidExample1 = {
expr = subpath.isValid null;
expected = false;
};
testSubpathIsValidExample2 = {
expr = subpath.isValid "";
expected = false;
};
testSubpathIsValidExample3 = {
expr = subpath.isValid "/foo";
expected = false;
};
testSubpathIsValidExample4 = {
expr = subpath.isValid "../foo";
expected = false;
};
testSubpathIsValidExample5 = {
expr = subpath.isValid "foo/bar";
expected = true;
};
testSubpathIsValidExample6 = {
expr = subpath.isValid "./foo//bar/";
expected = true;
};
# Some extra tests
testSubpathIsValidTwoDotsEnd = {
expr = subpath.isValid "foo/..";
expected = false;
};
testSubpathIsValidTwoDotsMiddle = {
expr = subpath.isValid "foo/../bar";
expected = false;
};
testSubpathIsValidTwoDotsPrefix = {
expr = subpath.isValid "..foo";
expected = true;
};
testSubpathIsValidTwoDotsSuffix = {
expr = subpath.isValid "foo..";
expected = true;
};
testSubpathIsValidTwoDotsPrefixComponent = {
expr = subpath.isValid "foo/..bar/baz";
expected = true;
};
testSubpathIsValidTwoDotsSuffixComponent = {
expr = subpath.isValid "foo/bar../baz";
expected = true;
};
testSubpathIsValidThreeDots = {
expr = subpath.isValid "...";
expected = true;
};
testSubpathIsValidFourDots = {
expr = subpath.isValid "....";
expected = true;
};
testSubpathIsValidThreeDotsComponent = {
expr = subpath.isValid "foo/.../bar";
expected = true;
};
testSubpathIsValidFourDotsComponent = {
expr = subpath.isValid "foo/..../bar";
expected = true;
};
# Test examples from the lib.path.subpath.join documentation
testSubpathJoinExample1 = {
expr = subpath.join [
"foo"
"bar/baz"
];
expected = "./foo/bar/baz";
};
testSubpathJoinExample2 = {
expr = subpath.join [
"./foo"
"."
"bar//./baz/"
];
expected = "./foo/bar/baz";
};
testSubpathJoinExample3 = {
expr = subpath.join [ ];
expected = "./.";
};
testSubpathJoinExample4 = {
expr = (builtins.tryEval (subpath.join [ /foo ])).success;
expected = false;
};
testSubpathJoinExample5 = {
expr = (builtins.tryEval (subpath.join [ "" ])).success;
expected = false;
};
testSubpathJoinExample6 = {
expr = (builtins.tryEval (subpath.join [ "/foo" ])).success;
expected = false;
};
testSubpathJoinExample7 = {
expr = (builtins.tryEval (subpath.join [ "../foo" ])).success;
expected = false;
};
# Test examples from the lib.path.subpath.normalise documentation
testSubpathNormaliseExample1 = {
expr = subpath.normalise "foo//bar";
expected = "./foo/bar";
};
testSubpathNormaliseExample2 = {
expr = subpath.normalise "foo/./bar";
expected = "./foo/bar";
};
testSubpathNormaliseExample3 = {
expr = subpath.normalise "foo/bar";
expected = "./foo/bar";
};
testSubpathNormaliseExample4 = {
expr = subpath.normalise "foo/bar/";
expected = "./foo/bar";
};
testSubpathNormaliseExample5 = {
expr = subpath.normalise "foo/bar/.";
expected = "./foo/bar";
};
testSubpathNormaliseExample6 = {
expr = subpath.normalise ".";
expected = "./.";
};
testSubpathNormaliseExample7 = {
expr = (builtins.tryEval (subpath.normalise "foo/../bar")).success;
expected = false;
};
testSubpathNormaliseExample8 = {
expr = (builtins.tryEval (subpath.normalise "")).success;
expected = false;
};
testSubpathNormaliseExample9 = {
expr = (builtins.tryEval (subpath.normalise "/foo")).success;
expected = false;
};
# Some extra tests
testSubpathNormaliseIsValidDots = {
expr = subpath.normalise "./foo/.bar/.../baz...qux";
expected = "./foo/.bar/.../baz...qux";
};
testSubpathNormaliseWrongType = {
expr = (builtins.tryEval (subpath.normalise null)).success;
expected = false;
};
testSubpathNormaliseTwoDots = {
expr = (builtins.tryEval (subpath.normalise "..")).success;
expected = false;
};
testSubpathComponentsExample1 = {
expr = subpath.components ".";
expected = [ ];
};
testSubpathComponentsExample2 = {
expr = subpath.components "./foo//bar/./baz/";
expected = [
"foo"
"bar"
"baz"
];
};
testSubpathComponentsExample3 = {
expr = (builtins.tryEval (subpath.components "/foo")).success;
expected = false;
};
};
in
if cases == [ ] then
"Unit tests successful"
else
throw "Path unit tests failed: ${lib.generators.toPretty { } cases}"

20
lib/source-types.nix Normal file
View File

@@ -0,0 +1,20 @@
{ lib }:
let
defaultSourceType = tname: {
shortName = tname;
isSource = false;
};
in
lib.mapAttrs (tname: tset: defaultSourceType tname // tset) {
fromSource = {
isSource = true;
};
binaryNativeCode = { };
binaryBytecode = { };
binaryFirmware = { };
}

548
lib/sources.nix Normal file
View File

@@ -0,0 +1,548 @@
# Functions for copying sources to the Nix store.
{ lib }:
# Tested in lib/tests/sources.sh
let
inherit (lib.strings)
match
split
storeDir
;
inherit (lib)
boolToString
filter
isString
readFile
;
inherit (lib.filesystem)
pathIsRegularFile
;
/**
A basic filter for `cleanSourceWith` that removes
directories of version control system, backup files (*~)
and some generated files.
# Inputs
`name`
: 1\. Function argument
`type`
: 2\. Function argument
*/
cleanSourceFilter =
name: type:
let
baseName = baseNameOf (toString name);
in
!(
# Filter out version control software files/directories
(
baseName == ".git"
||
type == "directory"
&& (
baseName == ".svn"
|| baseName == "CVS"
|| baseName == ".hg"
|| baseName == ".jj"
|| baseName == ".pijul"
|| baseName == "_darcs"
)
)
||
# Filter out editor backup / swap files.
lib.hasSuffix "~" baseName
|| match "^\\.sw[a-z]$" baseName != null
|| match "^\\..*\\.sw[a-z]$" baseName != null
||
# Filter out generates files.
lib.hasSuffix ".o" baseName
|| lib.hasSuffix ".so" baseName
||
# Filter out nix-build result symlinks
(type == "symlink" && lib.hasPrefix "result" baseName)
||
# Filter out sockets and other types of files we can't have in the store.
(type == "unknown")
);
/**
Filters a source tree removing version control files and directories using cleanSourceFilter.
# Inputs
`src`
: 1\. Function argument
# Examples
:::{.example}
## `cleanSource` usage example
```nix
cleanSource ./.
```
:::
*/
cleanSource =
src:
cleanSourceWith {
filter = cleanSourceFilter;
inherit src;
};
/**
Like `builtins.filterSource`, except it will compose with itself,
allowing you to chain multiple calls together without any
intermediate copies being put in the nix store.
# Examples
:::{.example}
## `cleanSourceWith` usage example
```nix
lib.cleanSourceWith {
filter = f;
src = lib.cleanSourceWith {
filter = g;
src = ./.;
};
}
# Succeeds!
builtins.filterSource f (builtins.filterSource g ./.)
# Fails!
```
:::
*/
cleanSourceWith =
{
# A path or cleanSourceWith result to filter and/or rename.
src,
# Optional with default value: constant true (include everything)
# The function will be combined with the && operator such
# that src.filter is called lazily.
# For implementing a filter, see
# https://nixos.org/nix/manual/#builtin-filterSource
# Type: A function (path -> type -> bool)
filter ? _path: _type: true,
# Optional name to use as part of the store path.
# This defaults to `src.name` or otherwise `"source"`.
name ? null,
}:
let
orig = toSourceAttributes src;
in
fromSourceAttributes {
inherit (orig) origSrc;
filter = path: type: filter path type && orig.filter path type;
name = if name != null then name else orig.name;
};
/**
Add logging to a source, for troubleshooting the filtering behavior.
# Inputs
`src`
: Source to debug. The returned source will behave like this source, but also log its filter invocations.
# Type
```
sources.trace :: sourceLike -> Source
```
*/
trace =
# Source to debug. The returned source will behave like this source, but also log its filter invocations.
src:
let
attrs = toSourceAttributes src;
in
fromSourceAttributes (
attrs
// {
filter =
path: type:
let
r = attrs.filter path type;
in
builtins.trace "${attrs.name}.filter ${path} = ${boolToString r}" r;
}
)
// {
satisfiesSubpathInvariant = src ? satisfiesSubpathInvariant && src.satisfiesSubpathInvariant;
};
/**
Filter sources by a list of regular expressions.
# Inputs
`src`
: 1\. Function argument
`regexes`
: 2\. Function argument
# Examples
:::{.example}
## `sourceByRegex` usage example
```nix
src = sourceByRegex ./my-subproject [".*\.py$" "^database.sql$"]
```
:::
*/
sourceByRegex =
src: regexes:
let
isFiltered = src ? _isLibCleanSourceWith;
origSrc = if isFiltered then src.origSrc else src;
in
lib.cleanSourceWith {
filter = (
path: type:
let
relPath = lib.removePrefix (toString origSrc + "/") (toString path);
in
lib.any (re: match re relPath != null) regexes
);
inherit src;
};
/**
Get all files ending with the specified suffices from the given
source directory or its descendants, omitting files that do not match
any suffix. The result of the example below will include files like
`./dir/module.c` and `./dir/subdir/doc.xml` if present.
# Inputs
`src`
: Path or source containing the files to be returned
`exts`
: A list of file suffix strings
# Type
```
sourceLike -> [String] -> Source
```
# Examples
:::{.example}
## `sourceFilesBySuffices` usage example
```nix
sourceFilesBySuffices ./. [ ".xml" ".c" ]
```
:::
*/
sourceFilesBySuffices =
# Path or source containing the files to be returned
src:
# A list of file suffix strings
exts:
let
filter =
name: type:
let
base = baseNameOf (toString name);
in
type == "directory" || lib.any (ext: lib.hasSuffix ext base) exts;
in
cleanSourceWith { inherit filter src; };
pathIsGitRepo = path: (_commitIdFromGitRepoOrError path) ? value;
/**
Get the commit id of a git repo.
# Inputs
`path`
: 1\. Function argument
# Examples
:::{.example}
## `commitIdFromGitRepo` usage example
```nix
commitIdFromGitRepo <nixpkgs/.git>
```
:::
*/
commitIdFromGitRepo =
path:
let
commitIdOrError = _commitIdFromGitRepoOrError path;
in
commitIdOrError.value or (throw commitIdOrError.error);
# Get the commit id of a git repo.
# Returns `{ value = commitHash }` or `{ error = "... message ..." }`.
# Example: commitIdFromGitRepo <nixpkgs/.git>
# not exported, used for commitIdFromGitRepo
_commitIdFromGitRepoOrError =
let
readCommitFromFile =
file: path:
let
fileName = path + "/${file}";
packedRefsName = path + "/packed-refs";
absolutePath =
base: path: if lib.hasPrefix "/" path then path else toString (/. + "${base}/${path}");
in
if
pathIsRegularFile path
# Resolve git worktrees. See gitrepository-layout(5)
then
let
m = match "^gitdir: (.*)$" (lib.fileContents path);
in
if m == null then
{ error = "File contains no gitdir reference: " + path; }
else
let
gitDir = absolutePath (dirOf path) (lib.head m);
commonDir'' =
if pathIsRegularFile "${gitDir}/commondir" then lib.fileContents "${gitDir}/commondir" else gitDir;
commonDir' = lib.removeSuffix "/" commonDir'';
commonDir = absolutePath gitDir commonDir';
refFile = lib.removePrefix "${commonDir}/" "${gitDir}/${file}";
in
readCommitFromFile refFile commonDir
else if
pathIsRegularFile fileName
# Sometimes git stores the commitId directly in the file but
# sometimes it stores something like: «ref: refs/heads/branch-name»
then
let
fileContent = lib.fileContents fileName;
matchRef = match "^ref: (.*)$" fileContent;
in
if matchRef == null then { value = fileContent; } else readCommitFromFile (lib.head matchRef) path
else if
pathIsRegularFile packedRefsName
# Sometimes, the file isn't there at all and has been packed away in the
# packed-refs file, so we have to grep through it:
then
let
fileContent = readFile packedRefsName;
matchRef = match "([a-z0-9]+) ${file}";
isRef = s: isString s && (matchRef s) != null;
# there is a bug in libstdc++ leading to stackoverflow for long strings:
# https://github.com/NixOS/nix/issues/2147#issuecomment-659868795
refs = filter isRef (split "\n" fileContent);
in
if refs == [ ] then
{ error = "Could not find " + file + " in " + packedRefsName; }
else
{ value = lib.head (matchRef (lib.head refs)); }
else
{ error = "Not a .git directory: " + toString path; };
in
readCommitFromFile "HEAD";
pathHasContext = builtins.hasContext or (lib.hasPrefix storeDir);
canCleanSource = src: src ? _isLibCleanSourceWith || !(pathHasContext (toString src));
# -------------------------------------------------------------------------- #
# Internal functions
#
# toSourceAttributes : sourceLike -> SourceAttrs
#
# Convert any source-like object into a simple, singular representation.
# We don't expose this representation in order to avoid having a fifth path-
# like class of objects in the wild.
# (Existing ones being: paths, strings, sources and x//{outPath})
# So instead of exposing internals, we build a library of combinator functions.
toSourceAttributes =
src:
let
isFiltered = src ? _isLibCleanSourceWith;
in
{
# The original path
origSrc = if isFiltered then src.origSrc else src;
filter = if isFiltered then src.filter else _: _: true;
name = if isFiltered then src.name else "source";
};
# fromSourceAttributes : SourceAttrs -> Source
#
# Inverse of toSourceAttributes for Source objects.
fromSourceAttributes =
{
origSrc,
filter,
name,
}:
{
_isLibCleanSourceWith = true;
inherit origSrc filter name;
outPath = builtins.path {
inherit filter name;
path = origSrc;
};
};
# urlToName : (URL | Path | String) -> String
#
# Transform a URL (or path, or string) into a clean package name.
urlToName =
url:
let
inherit (lib.strings) stringLength;
base = baseNameOf (lib.removeSuffix "/" (lib.last (lib.splitString ":" (toString url))));
# chop away one git or archive-related extension
removeExt =
name:
let
matchExt = match "(.*)\\.(git|tar|zip|gz|tgz|bz|tbz|bz2|tbz2|lzma|txz|xz|zstd)$" name;
in
if matchExt != null then lib.head matchExt else name;
# apply function f to string x while the result shrinks
shrink =
f: x:
let
v = f x;
in
if stringLength v < stringLength x then shrink f v else x;
in
shrink removeExt base;
# shortRev : (String | Integer) -> String
#
# Given a package revision (like "refs/tags/v12.0"), produce a short revision ("12.0").
shortRev =
rev:
let
baseRev = baseNameOf (toString rev);
matchHash = match "[a-f0-9]+" baseRev;
matchVer = match "([A-Za-z]+[-_. ]?)*(v)?([0-9.]+.*)" baseRev;
in
if matchHash != null then
builtins.substring 0 7 baseRev
else if matchVer != null then
lib.last matchVer
else
baseRev;
# revOrTag : String -> String -> String
#
# Turn git `rev` and `tag` pair into a revision usable in `repoRevToName*`.
revOrTag =
rev: tag:
if tag != null then
tag
else if rev != null then
rev
else
"HEAD";
# repoRevToNameFull : (URL | Path | String) -> (String | Integer | null) -> (String | null) -> String
#
# See `repoRevToName` below.
repoRevToNameFull =
repo_: rev_: suffix_:
let
repo = urlToName repo_;
rev = if rev_ != null then "-${shortRev rev_}" else "";
suffix = if suffix_ != null then "-${suffix_}" else "";
in
"${repo}${rev}${suffix}-source";
# repoRevToName : String -> (URL | Path | String) -> (String | Integer | null) -> String -> String
#
# Produce derivation.name attribute for a given repository URL/path/name and (optionally) its revision/version tag.
#
# This is used by fetch(zip|git|FromGitHub|hg|svn|etc) to generate discoverable
# /nix/store paths.
#
# This uses a different implementation depending on the `pretty` argument:
# "source" -> name everything as "source"
# "versioned" -> name everything as "${repo}-${rev}-source"
# "full" -> name everything as "${repo}-${rev}-${fetcher}-source"
repoRevToName =
kind:
# match on `kind` first to minimize the thunk
if kind == "source" then
(
repo: rev: suffix:
"source"
)
else if kind == "versioned" then
(
repo: rev: suffix:
repoRevToNameFull repo rev null
)
else if kind == "full" then
repoRevToNameFull
else
throw "repoRevToName: invalid kind";
in
{
pathType =
lib.warnIf (lib.oldestSupportedReleaseIsAtLeast 2305)
"lib.sources.pathType has been moved to lib.filesystem.pathType."
lib.filesystem.pathType;
pathIsDirectory =
lib.warnIf (lib.oldestSupportedReleaseIsAtLeast 2305)
"lib.sources.pathIsDirectory has been moved to lib.filesystem.pathIsDirectory."
lib.filesystem.pathIsDirectory;
pathIsRegularFile =
lib.warnIf (lib.oldestSupportedReleaseIsAtLeast 2305)
"lib.sources.pathIsRegularFile has been moved to lib.filesystem.pathIsRegularFile."
lib.filesystem.pathIsRegularFile;
inherit
pathIsGitRepo
commitIdFromGitRepo
cleanSource
cleanSourceWith
cleanSourceFilter
pathHasContext
canCleanSource
urlToName
shortRev
revOrTag
repoRevToName
sourceByRegex
sourceFilesBySuffices
trace
;
}

183
lib/strings-with-deps.nix Normal file
View File

@@ -0,0 +1,183 @@
{ lib }:
/**
Usage:
You define you custom builder script by adding all build steps to a list.
for example:
builder = writeScript "fsg-4.4-builder"
(textClosure [doUnpack addInputs preBuild doMake installPhase doForceShare]);
a step is defined by noDepEntry, fullDepEntry or packEntry.
To ensure that prerequisite are met those are added before the task itself by
textClosureDupList. Duplicated items are removed again.
See trace/nixpkgs/trunk/pkgs/top-level/builder-defs.nix for some predefined build steps
Attention:
let
pkgs = (import <nixpkgs>) {};
in let
inherit (pkgs.stringsWithDeps) fullDepEntry packEntry noDepEntry textClosureMap;
inherit (pkgs.lib) id;
nameA = noDepEntry "Text a";
nameB = fullDepEntry "Text b" ["nameA"];
nameC = fullDepEntry "Text c" ["nameA"];
stages = {
nameHeader = noDepEntry "#! /bin/sh \n";
inherit nameA nameB nameC;
};
in
textClosureMap id stages
[ "nameHeader" "nameA" "nameB" "nameC"
nameC # <- added twice. add a dep entry if you know that it will be added once only [1]
"nameB" # <- this will not be added again because the attr name (reference) is used
]
# result: Str("#! /bin/sh \n\nText a\nText b\nText c\nText c",[])
[1] maybe this behaviour should be removed to keep things simple (?)
*/
let
inherit (lib)
concatStringsSep
head
isAttrs
listToAttrs
tail
;
in
rec {
/**
Topologically sort a collection of dependent strings.
Only the values to keys listed in `arg` and their dependencies will be included in the result.
::: {.note}
This function doesn't formally fulfill the definition of topological sorting, but it's good enough for our purposes in Nixpkgs.
:::
# Inputs
`predefined` (attribute set)
: strings with annotated dependencies (strings or attribute set)
A value can be a simple string if it has no dependencies.
Otherwise, is can be an attribute set with the following attributes:
- `deps` (list of strings)
- `text` (Any
`arg` (list of strings)
: Keys for which the values in the dependency closure will be included in the result
# Type
```
textClosureList :: { ${phase} :: { deps :: [String]; text :: String; } | String; } -> [String] -> [String]
```
# Examples
:::{.example}
## `lib.stringsWithDeps.textClosureList` usage example
```nix
textClosureList {
a = {
deps = [ "b" "c" "e" ];
text = "a: depends on b, c and e";
};
b = {
deps = [ ];
text = "b: no dependencies";
};
c = {
deps = [ "b" ];
text = "c: depends on b";
};
d = {
deps = [ "c" ];
text = "d: not being depended on by anything in `arg`";
};
e = {
deps = [ "c" ];
text = "e: depends on c, depended on by a, not in `arg`";
};
} [
"a"
"b"
"c"
]
=> [
"b: no dependencies"
"c: depends on b"
"e: depends on c, depended on by a, not in `arg`"
"a: depends on b, c and e"
]
```
:::
Common real world usages are:
- Ordering the dependent phases of `system.activationScripts`
- Ordering the dependent phases of `system.userActivationScripts`
For further examples see: [NixOS activation script](https://nixos.org/manual/nixos/stable/#sec-activation-script)
*/
textClosureList =
predefined: arg:
let
f =
done: todo:
if todo == [ ] then
{
result = [ ];
inherit done;
}
else
let
entry = head todo;
in
if isAttrs entry then
let
x = f done entry.deps;
y = f x.done (tail todo);
in
{
result = x.result ++ [ entry.text ] ++ y.result;
done = y.done;
}
else if done ? ${entry} then
f done (tail todo)
else
f (
done
// listToAttrs [
{
name = entry;
value = 1;
}
]
) ([ predefined.${entry} ] ++ tail todo);
in
(f { } arg).result;
textClosureMap =
f: predefined: names:
concatStringsSep "\n" (map f (textClosureList predefined names));
noDepEntry = text: {
inherit text;
deps = [ ];
};
fullDepEntry = text: deps: { inherit text deps; };
packEntry = deps: {
inherit deps;
text = "";
};
stringAfter = deps: text: { inherit text deps; };
}

3221
lib/strings.nix Normal file
View File

File diff suppressed because it is too large Load Diff

626
lib/systems/architectures.nix Normal file
View File

@@ -0,0 +1,626 @@
{ lib }:
rec {
# gcc.arch to its features (as in /proc/cpuinfo)
features = {
# x86_64 Generic
# Spec: https://gitlab.com/x86-psABIs/x86-64-ABI/
default = [ ];
x86-64 = [ ];
x86-64-v2 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
];
x86-64-v3 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"avx"
"avx2"
"fma"
];
x86-64-v4 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"avx"
"avx2"
"avx512"
"fma"
];
# x86_64 Intel
nehalem = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
];
westmere = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
];
silvermont = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
];
sandybridge = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"avx"
];
ivybridge = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"avx"
];
haswell = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"avx"
"avx2"
"fma"
];
broadwell = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"avx"
"avx2"
"fma"
];
skylake = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"fma"
];
skylake-avx512 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
cannonlake = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
icelake-client = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
icelake-server = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
cascadelake = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
cooperlake = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
tigerlake = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
alderlake = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"fma"
];
sapphirerapids = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
emeraldrapids = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
sierraforest = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
"avx2"
"fma"
];
# x86_64 AMD
btver1 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
];
btver2 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"aes"
"avx"
];
bdver1 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"fma"
"fma4"
];
bdver2 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"fma"
"fma4"
];
bdver3 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"fma"
"fma4"
];
bdver4 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"avx2"
"fma"
"fma4"
];
znver1 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"avx2"
"fma"
];
znver2 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"avx2"
"fma"
];
znver3 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"avx2"
"fma"
];
znver4 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
znver5 = [
"sse3"
"ssse3"
"sse4_1"
"sse4_2"
"sse4a"
"aes"
"avx"
"avx2"
"avx512"
"fma"
];
# LoongArch64
# https://github.com/loongson/la-toolchain-conventions
loongarch64 = [
"fpu64"
];
la464 = [
"fpu64"
"lsx"
"lasx"
];
la664 = [
"fpu64"
"lsx"
"lasx"
"div32"
"frecipe"
"lam-bh"
"lamcas"
"ld-seq-sa"
];
"la64v1.0" = [
"fpu64"
"lsx"
];
"la64v1.1" = [
"fpu64"
"lsx"
"div32"
"frecipe"
"lam-bh"
"lamcas"
"ld-seq-sa"
];
# other
armv5te = [ ];
armv6 = [ ];
armv7-a = [ ];
armv8-a = [ ];
mips32 = [ ];
loongson2f = [ ];
};
# a superior CPU has all the features of an inferior and is able to build and test code for it
inferiors =
let
withInferiors = archs: lib.unique (archs ++ lib.flatten (lib.attrVals archs inferiors));
in
{
# x86_64 Generic
default = [ ];
x86-64 = [ ];
x86-64-v2 = [ "x86-64" ];
x86-64-v3 = [ "x86-64-v2" ] ++ inferiors.x86-64-v2;
x86-64-v4 = [ "x86-64-v3" ] ++ inferiors.x86-64-v3;
# x86_64 Intel
# https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
nehalem = [ "x86-64-v2" ] ++ inferiors.x86-64-v2;
westmere = [ "nehalem" ] ++ inferiors.nehalem;
sandybridge = [ "westmere" ] ++ inferiors.westmere;
ivybridge = [ "sandybridge" ] ++ inferiors.sandybridge;
haswell = lib.unique (
[
"ivybridge"
"x86-64-v3"
]
++ inferiors.ivybridge
++ inferiors.x86-64-v3
);
broadwell = [ "haswell" ] ++ inferiors.haswell;
skylake = [ "broadwell" ] ++ inferiors.broadwell;
skylake-avx512 = lib.unique (
[
"skylake"
"x86-64-v4"
]
++ inferiors.skylake
++ inferiors.x86-64-v4
);
cannonlake = [ "skylake-avx512" ] ++ inferiors.skylake-avx512;
icelake-client = [ "cannonlake" ] ++ inferiors.cannonlake;
icelake-server = [ "icelake-client" ] ++ inferiors.icelake-client;
cascadelake = [ "cannonlake" ] ++ inferiors.cannonlake;
cooperlake = [ "cascadelake" ] ++ inferiors.cascadelake;
tigerlake = [ "icelake-server" ] ++ inferiors.icelake-server;
sapphirerapids = [ "tigerlake" ] ++ inferiors.tigerlake;
emeraldrapids = [ "sapphirerapids" ] ++ inferiors.sapphirerapids;
alderlake = [ "skylake" ] ++ inferiors.skylake;
sierraforest = [ "alderlake" ] ++ inferiors.alderlake;
# x86_64 AMD
# TODO: fill in specific CPU architecture inferiors
btver1 = [ "x86-64" ];
btver2 = [ "x86-64-v2" ] ++ inferiors.x86-64-v2;
bdver1 = [ "x86-64-v2" ] ++ inferiors.x86-64-v2;
bdver2 = [ "x86-64-v2" ] ++ inferiors.x86-64-v2;
bdver3 = [ "x86-64-v2" ] ++ inferiors.x86-64-v2;
bdver4 = [ "x86-64-v3" ] ++ inferiors.x86-64-v3;
# Regarding `skylake` as inferior of `znver1`, there are reports of
# successful usage by Gentoo users and Phoronix benchmarking of different
# `-march` targets.
#
# The GCC documentation on extensions used and wikichip documentation
# regarding supperted extensions on znver1 and skylake was used to create
# this partial order.
#
# Note:
#
# - The successors of `skylake` (`cannonlake`, `icelake`, etc) use `avx512`
# which no current AMD Zen michroarch support.
# - `znver1` uses `ABM`, `CLZERO`, `CX16`, `MWAITX`, and `SSE4A` which no
# current Intel microarch support.
#
# https://www.phoronix.com/scan.php?page=article&item=amd-znver3-gcc11&num=1
# https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
# https://en.wikichip.org/wiki/amd/microarchitectures/zen
# https://en.wikichip.org/wiki/intel/microarchitectures/skylake
znver1 = [ "skylake" ] ++ inferiors.skylake; # Includes haswell and x86-64-v3
znver2 = [ "znver1" ] ++ inferiors.znver1;
znver3 = [ "znver2" ] ++ inferiors.znver2;
znver4 = lib.unique (
[
"znver3"
"x86-64-v4"
]
++ inferiors.znver3
++ inferiors.x86-64-v4
);
znver5 = [ "znver4" ] ++ inferiors.znver4;
# ARM64 (AArch64)
armv8-a = [ ];
"armv8.1-a" = [ "armv8-a" ];
"armv8.2-a" = [ "armv8.1-a" ] ++ inferiors."armv8.1-a";
"armv8.3-a" = [ "armv8.2-a" ] ++ inferiors."armv8.2-a";
"armv8.4-a" = [ "armv8.3-a" ] ++ inferiors."armv8.3-a";
"armv8.5-a" = [ "armv8.4-a" ] ++ inferiors."armv8.4-a";
"armv8.6-a" = [ "armv8.5-a" ] ++ inferiors."armv8.5-a";
"armv8.7-a" = [ "armv8.6-a" ] ++ inferiors."armv8.6-a";
"armv8.8-a" = [ "armv8.7-a" ] ++ inferiors."armv8.7-a";
"armv8.9-a" = [ "armv8.8-a" ] ++ inferiors."armv8.8-a";
armv9-a = [ "armv8.5-a" ] ++ inferiors."armv8.5-a";
"armv9.1-a" = [
"armv9-a"
"armv8.6-a"
]
++ inferiors."armv8.6-a";
"armv9.2-a" = lib.unique (
[
"armv9.1-a"
"armv8.7-a"
]
++ inferiors."armv9.1-a"
++ inferiors."armv8.7-a"
);
"armv9.3-a" = lib.unique (
[
"armv9.2-a"
"armv8.8-a"
]
++ inferiors."armv9.2-a"
++ inferiors."armv8.8-a"
);
"armv9.4-a" = [ "armv9.3-a" ] ++ inferiors."armv9.3-a";
# ARM
cortex-a53 = [ "armv8-a" ];
cortex-a72 = [ "armv8-a" ];
cortex-a55 = [
"armv8.2-a"
"cortex-a53"
"cortex-a72"
]
++ inferiors."armv8.2-a";
cortex-a76 = [
"armv8.2-a"
"cortex-a53"
"cortex-a72"
]
++ inferiors."armv8.2-a";
# Ampere
ampere1 = withInferiors [
"armv8.6-a"
"cortex-a55"
"cortex-a76"
];
ampere1a = [ "ampere1" ] ++ inferiors.ampere1;
ampere1b = [ "ampere1a" ] ++ inferiors.ampere1a;
# LoongArch64
loongarch64 = [ ];
"la64v1.0" = [ "loongarch64" ];
la464 = [ "la64v1.0" ] ++ inferiors."la64v1.0";
"la64v1.1" = [ "la64v1.0" ] ++ inferiors."la64v1.0";
la664 = withInferiors [
"la464"
"la64v1.1"
];
# other
armv5te = [ ];
armv6 = [ ];
armv7-a = [ ];
mips32 = [ ];
loongson2f = [ ];
};
/**
Check whether one GCC architecture has the the other inferior architecture.
# Inputs
`arch1`
: GCC architecture in string
`arch2`
: GCC architecture in string
# Type
```
hasInferior :: string -> string -> bool
```
# Examples
::: {.example}
## `lib.systems.architectures.hasInferior` usage example
```nix
hasInferior "x86-64-v3" "x86-64"
=> true
hasInferior "x86-64" "x86-64-v3"
=> false
hasInferior "x86-64" "x86-64"
=> false
```
*/
hasInferior = arch1: arch2: inferiors ? ${arch1} && lib.elem arch2 inferiors.${arch1};
/**
Check whether one GCC architecture can execute the other.
# Inputs
`arch1`
: GCC architecture in string
`arch2`
: GCC architecture in string
# Type
```
canExecute :: string -> string -> bool
```
# Examples
::: {.example}
## `lib.systems.architectures.canExecute` usage example
```nix
canExecute "x86-64" "x86-64-v3"
=> false
canExecute "x86-64-v3" "x86-64"
=> true
canExecute "x86-64" "x86-64"
=> true
```
*/
canExecute = arch1: arch2: arch1 == arch2 || hasInferior arch1 arch2;
predicates =
let
featureSupport = feature: x: builtins.elem feature features.${x} or [ ];
in
{
sse3Support = featureSupport "sse3";
ssse3Support = featureSupport "ssse3";
sse4_1Support = featureSupport "sse4_1";
sse4_2Support = featureSupport "sse4_2";
sse4_aSupport = featureSupport "sse4a";
avxSupport = featureSupport "avx";
avx2Support = featureSupport "avx2";
avx512Support = featureSupport "avx512";
aesSupport = featureSupport "aes";
fmaSupport = featureSupport "fma";
fma4Support = featureSupport "fma4";
lsxSupport = featureSupport "lsx";
lasxSupport = featureSupport "lasx";
};
}

638
lib/systems/default.nix Normal file
View File

@@ -0,0 +1,638 @@
{ lib }:
let
inherit (lib)
any
filterAttrs
foldl
hasInfix
isAttrs
isFunction
isList
mapAttrs
optional
optionalAttrs
optionalString
removeSuffix
replaceString
toUpper
;
inherit (lib.strings) toJSON;
doubles = import ./doubles.nix { inherit lib; };
parse = import ./parse.nix { inherit lib; };
inspect = import ./inspect.nix { inherit lib; };
platforms = import ./platforms.nix { inherit lib; };
examples = import ./examples.nix { inherit lib; };
architectures = import ./architectures.nix { inherit lib; };
/**
Elaborated systems contain functions, which means that they don't satisfy
`==` for a lack of reflexivity.
They might *appear* to satisfy `==` reflexivity when the same exact value is
compared to itself, because object identity is used as an "optimization";
compare the value with a reconstruction of itself, e.g. with `f == a: f a`,
or perhaps calling `elaborate` twice, and one will see reflexivity fail as described.
Hence a custom equality test.
Note that this does not canonicalize the systems, so you'll want to make sure
both arguments have been `elaborate`-d.
*/
equals =
let
removeFunctions = a: filterAttrs (_: v: !isFunction v) a;
in
a: b: removeFunctions a == removeFunctions b;
/**
List of all Nix system doubles the nixpkgs flake will expose the package set
for. All systems listed here must be supported by nixpkgs as `localSystem`.
:::{.warning}
This attribute is considered experimental and is subject to change.
:::
*/
flakeExposed = import ./flake-systems.nix { };
# Turn localSystem or crossSystem, which could be system-string or attrset, into
# attrset.
systemToAttrs =
systemOrArgs: if isAttrs systemOrArgs then systemOrArgs else { system = systemOrArgs; };
# Elaborate a `localSystem` or `crossSystem` so that it contains everything
# necessary.
#
# `parsed` is inferred from args, both because there are two options with one
# clearly preferred, and to prevent cycles. A simpler fixed point where the RHS
# always just used `final.*` would fail on both counts.
elaborate =
systemOrArgs:
let
allArgs = systemToAttrs systemOrArgs;
# Those two will always be derived from "config", if given, so they should NOT
# be overridden further down with "// args".
args = removeAttrs allArgs [
"parsed"
"system"
];
# TODO: deprecate args.rustc in favour of args.rust after 23.05 is EOL.
rust = args.rust or args.rustc or { };
final = {
# Prefer to parse `config` as it is strictly more informative.
parsed = parse.mkSystemFromString (args.config or allArgs.system);
# This can be losslessly-extracted from `parsed` iff parsing succeeds.
system = parse.doubleFromSystem final.parsed;
# TODO: This currently can't be losslessly-extracted from `parsed`, for example
# because of -mingw32.
config = parse.tripleFromSystem final.parsed;
# Determine whether we can execute binaries built for the provided platform.
canExecute =
platform:
final.isAndroid == platform.isAndroid
&& parse.isCompatible final.parsed.cpu platform.parsed.cpu
&& final.parsed.kernel == platform.parsed.kernel
&& (
# Only perform this check when cpus have the same type;
# assume compatible cpu have all the instructions included
final.parsed.cpu == platform.parsed.cpu
->
# if both have gcc.arch defined, check whether final can execute the given platform
(
(final ? gcc.arch && platform ? gcc.arch)
-> architectures.canExecute final.gcc.arch platform.gcc.arch
)
# if platform has gcc.arch defined but final doesn't, don't assume it can be executed
|| (platform ? gcc.arch -> !(final ? gcc.arch))
);
isCompatible =
_:
throw "2022-05-23: isCompatible has been removed in favor of canExecute, refer to the 22.11 changelog for details";
# Derived meta-data
useLLVM = final.isFreeBSD || final.isOpenBSD;
libc =
if final.isDarwin then
"libSystem"
else if final.isMsvc then
"ucrt"
else if final.isMinGW then
"msvcrt"
else if final.isCygwin then
"cygwin"
else if final.isWasi then
"wasilibc"
else if final.isWasm && !final.isWasi then
null
else if final.isRedox then
"relibc"
else if final.isMusl then
"musl"
else if final.isUClibc then
"uclibc"
else if final.isAndroid then
"bionic"
else if
final.isLinux # default
then
"glibc"
else if final.isFreeBSD then
"fblibc"
else if final.isOpenBSD then
"oblibc"
else if final.isNetBSD then
"nblibc"
else if final.isAvr then
"avrlibc"
else if final.isGhcjs then
null
else if final.isNone then
"newlib"
# TODO(@Ericson2314) think more about other operating systems
else
"native/impure";
# Choose what linker we wish to use by default. Someday we might also
# choose the C compiler, runtime library, C++ standard library, etc. in
# this way, nice and orthogonally, and deprecate `useLLVM`. But due to
# the monolithic GCC build we cannot actually make those choices
# independently, so we are just doing `linker` and keeping `useLLVM` for
# now.
linker =
if final.useLLVM or false then
"lld"
else if final.isDarwin then
"cctools"
# "bfd" and "gold" both come from GNU binutils. The existence of Gold
# is why we use the more obscure "bfd" and not "binutils" for this
# choice.
else
"bfd";
# The standard lib directory name that non-nixpkgs binaries distributed
# for this platform normally assume.
libDir =
if final.isLinux then
if final.isx86_64 || final.isMips64 || final.isPower64 then "lib64" else "lib"
else
null;
extensions =
optionalAttrs final.hasSharedLibraries {
sharedLibrary =
if final.isDarwin then
".dylib"
else if (final.isWindows || final.isCygwin) then
".dll"
else
".so";
}
// {
staticLibrary = if final.isWindows then ".lib" else ".a";
library = if final.isStatic then final.extensions.staticLibrary else final.extensions.sharedLibrary;
executable = if (final.isWindows || final.isCygwin) then ".exe" else "";
};
# Misc boolean options
useAndroidPrebuilt = false;
useiOSPrebuilt = false;
# Output from uname
uname = {
# uname -s
system =
{
linux = "Linux";
windows = "Windows";
cygwin = "CYGWIN_NT";
darwin = "Darwin";
netbsd = "NetBSD";
freebsd = "FreeBSD";
openbsd = "OpenBSD";
wasi = "Wasi";
redox = "Redox";
genode = "Genode";
}
.${final.parsed.kernel.name} or null;
# uname -m
processor =
if final.isPower64 then
"ppc64${optionalString final.isLittleEndian "le"}"
else if final.isPower then
"ppc${optionalString final.isLittleEndian "le"}"
else if final.isMips64 then
"mips64" # endianness is *not* included on mips64
else if final.isDarwin then
final.darwinArch
else
final.parsed.cpu.name;
# uname -r
release = null;
};
# It is important that hasSharedLibraries==false when the platform has no
# dynamic library loader. Various tools (including the gcc build system)
# have knowledge of which platforms are incapable of dynamic linking, and
# will still build on/for those platforms with --enable-shared, but simply
# omit any `.so` build products such as libgcc_s.so. When that happens,
# it causes hard-to-troubleshoot build failures.
hasSharedLibraries =
with final;
(
isAndroid
|| isGnu
|| isMusl # Linux (allows multiple libcs)
|| isDarwin
|| isSunOS
|| isOpenBSD
|| isFreeBSD
|| isNetBSD # BSDs
|| isCygwin
|| isMinGW
|| isWindows # Windows
|| isWasm # WASM
)
&& !isStatic;
# The difference between `isStatic` and `hasSharedLibraries` is mainly the
# addition of the `staticMarker` (see make-derivation.nix). Some
# platforms, like embedded machines without a libc (e.g. arm-none-eabi)
# don't support dynamic linking, but don't get the `staticMarker`.
# `pkgsStatic` sets `isStatic=true`, so `pkgsStatic.hostPlatform` always
# has the `staticMarker`.
isStatic = final.isWasi || final.isRedox;
# Just a guess, based on `system`
inherit
(
{
linux-kernel = args.linux-kernel or { };
gcc = args.gcc or { };
}
// platforms.select final
)
linux-kernel
gcc
;
# TODO: remove after 23.05 is EOL, with an error pointing to the rust.* attrs.
rustc = args.rustc or { };
linuxArch =
if final.isAarch32 then
"arm"
else if final.isAarch64 then
"arm64"
else if final.isx86_32 then
"i386"
else if final.isx86_64 then
"x86_64"
# linux kernel does not distinguish microblaze/microblazeel
else if final.isMicroBlaze then
"microblaze"
else if final.isMips32 then
"mips"
else if final.isMips64 then
"mips" # linux kernel does not distinguish mips32/mips64
else if final.isPower then
"powerpc"
else if final.isRiscV then
"riscv"
else if final.isS390 then
"s390"
else if final.isLoongArch64 then
"loongarch"
else
final.parsed.cpu.name;
# https://source.denx.de/u-boot/u-boot/-/blob/9bfb567e5f1bfe7de8eb41f8c6d00f49d2b9a426/common/image.c#L81-106
ubootArch =
if final.isx86_32 then
"x86" # not i386
else if final.isMips64 then
"mips64" # uboot *does* distinguish between mips32/mips64
else
final.linuxArch; # other cases appear to agree with linuxArch
qemuArch =
if final.isAarch32 then
"arm"
else if final.isAarch64 then
"aarch64"
else if final.isS390 && !final.isS390x then
null
else if final.isx86_64 then
"x86_64"
else if final.isx86 then
"i386"
else if final.isMips64n32 then
"mipsn32${optionalString final.isLittleEndian "el"}"
else if final.isMips64 then
"mips64${optionalString final.isLittleEndian "el"}"
else
final.uname.processor;
# Name used by UEFI for architectures.
efiArch =
if final.isx86_32 then
"ia32"
else if final.isx86_64 then
"x64"
else if final.isAarch32 then
"arm"
else if final.isAarch64 then
"aa64"
else
final.parsed.cpu.name;
darwinArch = parse.darwinArch final.parsed.cpu;
darwinPlatform =
if final.isMacOS then
"macos"
else if final.isiOS then
"ios"
else
null;
# The canonical name for this attribute is darwinSdkVersion, but some
# platforms define the old name "sdkVer".
darwinSdkVersion = final.sdkVer or "11.3";
darwinMinVersion = final.darwinSdkVersion;
darwinMinVersionVariable =
if final.isMacOS then
"MACOSX_DEPLOYMENT_TARGET"
else if final.isiOS then
"IPHONEOS_DEPLOYMENT_TARGET"
else
null;
# Handle Android SDK and NDK versions.
androidSdkVersion = args.androidSdkVersion or null;
androidNdkVersion = args.androidNdkVersion or null;
}
// (
let
selectEmulator =
pkgs:
let
wine = (pkgs.winePackagesFor "wine${toString final.parsed.cpu.bits}").minimal;
in
# Note: we guarantee that the return value is either `null` or a path
# to an emulator program. That is, if an emulator requires additional
# arguments, a wrapper should be used.
if pkgs.stdenv.hostPlatform.canExecute final then
lib.getExe (pkgs.writeShellScriptBin "exec" ''exec "$@"'')
else if final.isWindows then
"${wine}/bin/wine${optionalString (final.parsed.cpu.bits == 64) "64"}"
else if final.isLinux && pkgs.stdenv.hostPlatform.isLinux && final.qemuArch != null then
"${pkgs.qemu-user}/bin/qemu-${final.qemuArch}"
else if final.isWasi then
"${pkgs.wasmtime}/bin/wasmtime"
else if final.isMmix then
"${pkgs.mmixware}/bin/mmix"
else
null;
in
{
emulatorAvailable = pkgs: (selectEmulator pkgs) != null;
# whether final.emulator pkgs.pkgsStatic works
staticEmulatorAvailable =
pkgs: final.emulatorAvailable pkgs && (final.isLinux || final.isWasi || final.isMmix);
emulator =
pkgs:
if (final.emulatorAvailable pkgs) then
selectEmulator pkgs
else
throw "Don't know how to run ${final.config} executables.";
}
)
// mapAttrs (n: v: v final.parsed) inspect.predicates
// mapAttrs (n: v: v final.gcc.arch or "default") architectures.predicates
// args
// {
rust = rust // {
# Once args.rustc.platform.target-family is deprecated and
# removed, there will no longer be any need to modify any
# values from args.rust.platform, so we can drop all the
# "args ? rust" etc. checks, and merge args.rust.platform in
# /after/.
platform = rust.platform or { } // {
# https://doc.rust-lang.org/reference/conditional-compilation.html#target_arch
arch =
if rust ? platform then
rust.platform.arch
else if final.isAarch32 then
"arm"
else if final.isMips64 then
"mips64" # never add "el" suffix
else if final.isPower64 then
"powerpc64" # never add "le" suffix
else
final.parsed.cpu.name;
# https://doc.rust-lang.org/reference/conditional-compilation.html#target_os
os =
if rust ? platform then
rust.platform.os or "none"
else if final.isDarwin then
"macos"
else if final.isWasm && !final.isWasi then
"unknown" # Needed for {wasm32,wasm64}-unknown-unknown.
else
final.parsed.kernel.name;
# https://doc.rust-lang.org/reference/conditional-compilation.html#target_family
target-family =
if args ? rust.platform.target-family then
args.rust.platform.target-family
else if args ? rustc.platform.target-family then
(
# Since https://github.com/rust-lang/rust/pull/84072
# `target-family` is a list instead of single value.
let
f = args.rustc.platform.target-family;
in
if isList f then f else [ f ]
)
else
optional final.isUnix "unix" ++ optional final.isWindows "windows" ++ optional final.isWasm "wasm";
# https://doc.rust-lang.org/reference/conditional-compilation.html#target_vendor
vendor =
let
inherit (final.parsed) vendor;
in
rust.platform.vendor or {
"w64" = "pc";
}
.${vendor.name} or vendor.name;
};
# The name of the rust target, even if it is custom. Adjustments are
# because rust has slightly different naming conventions than we do.
rustcTarget =
let
inherit (final.parsed) cpu kernel abi;
cpu_ =
rust.platform.arch or {
"armv7a" = "armv7";
"armv7l" = "armv7";
"armv6l" = "arm";
"armv5tel" = "armv5te";
"riscv32" = "riscv32gc";
"riscv64" = "riscv64gc";
}
.${cpu.name} or cpu.name;
vendor_ = final.rust.platform.vendor;
in
# TODO: deprecate args.rustc in favour of args.rust after 23.05 is EOL.
args.rust.rustcTarget or args.rustc.config or (
# Rust uses `wasm32-wasip?` rather than `wasm32-unknown-wasi`.
# We cannot know which subversion does the user want, and
# currently use WASI 0.1 as default for compatibility. Custom
# users can set `rust.rustcTarget` to override it.
if final.isWasi then
"${cpu_}-wasip1"
else
"${cpu_}-${vendor_}-${kernel.name}${optionalString (abi.name != "unknown") "-${abi.name}"}"
);
# The name of the rust target if it is standard, or the json file
# containing the custom target spec.
rustcTargetSpec =
rust.rustcTargetSpec or (
if rust ? platform then
builtins.toFile (final.rust.rustcTarget + ".json") (toJSON rust.platform)
else
final.rust.rustcTarget
);
# The name of the rust target if it is standard, or the
# basename of the file containing the custom target spec,
# without the .json extension.
#
# This is the name used by Cargo for target subdirectories.
cargoShortTarget = removeSuffix ".json" (baseNameOf "${final.rust.rustcTargetSpec}");
# When used as part of an environment variable name, triples are
# uppercased and have all hyphens replaced by underscores:
#
# https://github.com/rust-lang/cargo/pull/9169
# https://github.com/rust-lang/cargo/issues/8285#issuecomment-634202431
cargoEnvVarTarget = replaceString "-" "_" (toUpper final.rust.cargoShortTarget);
# True if the target is no_std
# https://github.com/rust-lang/rust/blob/2e44c17c12cec45b6a682b1e53a04ac5b5fcc9d2/src/bootstrap/config.rs#L415-L421
isNoStdTarget = any (t: hasInfix t final.rust.rustcTarget) [
"-none"
"nvptx"
"switch"
"-uefi"
];
};
}
// {
go = {
# See https://pkg.go.dev/internal/platform for a list of known platforms
GOARCH =
{
"aarch64" = "arm64";
"arm" = "arm";
"armv5tel" = "arm";
"armv6l" = "arm";
"armv7l" = "arm";
"i686" = "386";
"loongarch64" = "loong64";
"mips" = "mips";
"mips64el" = "mips64le";
"mipsel" = "mipsle";
"powerpc64" = "ppc64";
"powerpc64le" = "ppc64le";
"riscv64" = "riscv64";
"s390x" = "s390x";
"x86_64" = "amd64";
"wasm32" = "wasm";
}
.${final.parsed.cpu.name} or null;
GOOS = if final.isWasi then "wasip1" else final.parsed.kernel.name;
# See https://go.dev/wiki/GoArm
GOARM = toString (lib.intersectLists [ (final.parsed.cpu.version or "") ] [ "5" "6" "7" ]);
};
node = {
# See these locations for a list of known architectures/platforms:
# - https://nodejs.org/api/os.html#osarch
# - https://nodejs.org/api/os.html#osplatform
arch =
if final.isAarch then
"arm" + lib.optionalString final.is64bit "64"
else if final.isMips32 then
"mips" + lib.optionalString final.isLittleEndian "el"
else if final.isMips64 && final.isLittleEndian then
"mips64el"
else if final.isPower then
"ppc" + lib.optionalString final.is64bit "64"
else if final.isx86_64 then
"x64"
else if final.isx86_32 then
"ia32"
else if final.isS390x then
"s390x"
else if final.isRiscV64 then
"riscv64"
else if final.isLoongArch64 then
"loong64"
else
null;
platform =
if final.isAndroid then
"android"
else if final.isDarwin then
"darwin"
else if final.isFreeBSD then
"freebsd"
else if final.isLinux then
"linux"
else if final.isOpenBSD then
"openbsd"
else if final.isSunOS then
"sunos"
else if (final.isWindows || final.isCygwin) then
"win32"
else
null;
};
};
in
assert final.useAndroidPrebuilt -> final.isAndroid;
assert foldl (pass: { assertion, message }: if assertion final then pass else throw message) true (
final.parsed.abi.assertions or [ ]
);
final;
in
# Everything in this attrset is the public interface of the file.
{
inherit
architectures
doubles
elaborate
equals
examples
flakeExposed
inspect
parse
platforms
systemToAttrs
;
}

194
lib/systems/doubles.nix Normal file
View File

@@ -0,0 +1,194 @@
{ lib }:
let
inherit (lib) lists;
inherit (lib.systems) parse;
inherit (lib.systems.inspect) predicates;
inherit (lib.attrsets) matchAttrs;
all = [
# Cygwin
"i686-cygwin"
"x86_64-cygwin"
# Darwin
"x86_64-darwin"
"aarch64-darwin"
# FreeBSD
"i686-freebsd"
"x86_64-freebsd"
"aarch64-freebsd"
# Genode
"aarch64-genode"
"i686-genode"
"x86_64-genode"
# illumos
"x86_64-solaris"
# JS
"javascript-ghcjs"
# Linux
"aarch64-linux"
"armv5tel-linux"
"armv6l-linux"
"armv7a-linux"
"armv7l-linux"
"i686-linux"
"loongarch64-linux"
"m68k-linux"
"microblaze-linux"
"microblazeel-linux"
"mips-linux"
"mips64-linux"
"mips64el-linux"
"mipsel-linux"
"powerpc-linux"
"powerpc64-linux"
"powerpc64le-linux"
"riscv32-linux"
"riscv64-linux"
"s390-linux"
"s390x-linux"
"x86_64-linux"
# MMIXware
"mmix-mmixware"
# NetBSD
"aarch64-netbsd"
"armv6l-netbsd"
"armv7a-netbsd"
"armv7l-netbsd"
"i686-netbsd"
"m68k-netbsd"
"mipsel-netbsd"
"powerpc-netbsd"
"riscv32-netbsd"
"riscv64-netbsd"
"x86_64-netbsd"
# none
"aarch64_be-none"
"aarch64-none"
"arm-none"
"armv6l-none"
"avr-none"
"i686-none"
"microblaze-none"
"microblazeel-none"
"mips-none"
"mips64-none"
"msp430-none"
"or1k-none"
"m68k-none"
"powerpc-none"
"powerpcle-none"
"riscv32-none"
"riscv64-none"
"rx-none"
"s390-none"
"s390x-none"
"vc4-none"
"x86_64-none"
# OpenBSD
"i686-openbsd"
"x86_64-openbsd"
# Redox
"x86_64-redox"
# WASI
"wasm64-wasi"
"wasm32-wasi"
# Windows
"aarch64-windows"
"x86_64-windows"
"i686-windows"
];
allParsed = map parse.mkSystemFromString all;
filterDoubles = f: map parse.doubleFromSystem (lists.filter f allParsed);
in
{
inherit all;
none = [ ];
arm = filterDoubles predicates.isAarch32;
armv7 = filterDoubles predicates.isArmv7;
aarch = filterDoubles predicates.isAarch;
aarch64 = filterDoubles predicates.isAarch64;
x86 = filterDoubles predicates.isx86;
i686 = filterDoubles predicates.isi686;
x86_64 = filterDoubles predicates.isx86_64;
microblaze = filterDoubles predicates.isMicroBlaze;
mips = filterDoubles predicates.isMips;
mmix = filterDoubles predicates.isMmix;
power = filterDoubles predicates.isPower;
riscv = filterDoubles predicates.isRiscV;
riscv32 = filterDoubles predicates.isRiscV32;
riscv64 = filterDoubles predicates.isRiscV64;
rx = filterDoubles predicates.isRx;
vc4 = filterDoubles predicates.isVc4;
or1k = filterDoubles predicates.isOr1k;
m68k = filterDoubles predicates.isM68k;
s390 = filterDoubles predicates.isS390;
s390x = filterDoubles predicates.isS390x;
loongarch64 = filterDoubles predicates.isLoongArch64;
js = filterDoubles predicates.isJavaScript;
bigEndian = filterDoubles predicates.isBigEndian;
littleEndian = filterDoubles predicates.isLittleEndian;
cygwin = filterDoubles predicates.isCygwin;
darwin = filterDoubles predicates.isDarwin;
freebsd = filterDoubles predicates.isFreeBSD;
# Should be better, but MinGW is unclear.
gnu =
filterDoubles (matchAttrs {
kernel = parse.kernels.linux;
abi = parse.abis.gnu;
})
++ filterDoubles (matchAttrs {
kernel = parse.kernels.linux;
abi = parse.abis.gnueabi;
})
++ filterDoubles (matchAttrs {
kernel = parse.kernels.linux;
abi = parse.abis.gnueabihf;
})
++ filterDoubles (matchAttrs {
kernel = parse.kernels.linux;
abi = parse.abis.gnuabin32;
})
++ filterDoubles (matchAttrs {
kernel = parse.kernels.linux;
abi = parse.abis.gnuabi64;
})
++ filterDoubles (matchAttrs {
kernel = parse.kernels.linux;
abi = parse.abis.gnuabielfv1;
})
++ filterDoubles (matchAttrs {
kernel = parse.kernels.linux;
abi = parse.abis.gnuabielfv2;
});
illumos = filterDoubles predicates.isSunOS;
linux = filterDoubles predicates.isLinux;
netbsd = filterDoubles predicates.isNetBSD;
openbsd = filterDoubles predicates.isOpenBSD;
unix = filterDoubles predicates.isUnix;
wasi = filterDoubles predicates.isWasi;
redox = filterDoubles predicates.isRedox;
windows = filterDoubles predicates.isWindows;
genode = filterDoubles predicates.isGenode;
embedded = filterDoubles predicates.isNone;
}

444
lib/systems/examples.nix Normal file
View File

@@ -0,0 +1,444 @@
# These can be passed to nixpkgs as either the `localSystem` or
# `crossSystem`. They are put here for user convenience, but also used by cross
# tests and linux cross stdenv building, so handle with care!
{ lib }:
let
platforms = import ./platforms.nix { inherit lib; };
riscv = bits: {
config = "riscv${bits}-unknown-linux-gnu";
};
in
rec {
#
# Linux
#
powernv = {
config = "powerpc64le-unknown-linux-gnu";
};
musl-power = {
config = "powerpc64le-unknown-linux-musl";
};
ppc64-elfv1 = {
config = "powerpc64-unknown-linux-gnuabielfv1";
rust.rustcTarget = "powerpc64-unknown-linux-gnu";
};
ppc64-elfv2 = {
config = "powerpc64-unknown-linux-gnuabielfv2";
};
ppc64 = ppc64-elfv2;
ppc64-musl = {
config = "powerpc64-unknown-linux-musl";
gcc = {
abi = "elfv2";
};
};
ppc32 = {
config = "powerpc-unknown-linux-gnu";
rust.rustcTarget = "powerpc-unknown-linux-gnu";
};
sheevaplug = {
config = "armv5tel-unknown-linux-gnueabi";
}
// platforms.sheevaplug;
raspberryPi = {
config = "armv6l-unknown-linux-gnueabihf";
}
// platforms.raspberrypi;
bluefield2 = {
config = "aarch64-unknown-linux-gnu";
}
// platforms.bluefield2;
remarkable1 = {
config = "armv7l-unknown-linux-gnueabihf";
}
// platforms.zero-gravitas;
remarkable2 = {
config = "armv7l-unknown-linux-gnueabihf";
}
// platforms.zero-sugar;
armv7l-hf-multiplatform = {
config = "armv7l-unknown-linux-gnueabihf";
};
aarch64-multiplatform = {
config = "aarch64-unknown-linux-gnu";
};
armv7a-android-prebuilt = {
config = "armv7a-unknown-linux-androideabi";
rust.rustcTarget = "armv7-linux-androideabi";
androidSdkVersion = "35";
androidNdkVersion = "27";
useAndroidPrebuilt = true;
}
// platforms.armv7a-android;
aarch64-android-prebuilt = {
config = "aarch64-unknown-linux-android";
rust.rustcTarget = "aarch64-linux-android";
androidSdkVersion = "35";
androidNdkVersion = "27";
useAndroidPrebuilt = true;
};
aarch64-android = {
config = "aarch64-unknown-linux-android";
androidSdkVersion = "35";
androidNdkVersion = "27";
libc = "bionic";
useAndroidPrebuilt = false;
useLLVM = true;
};
pogoplug4 = {
config = "armv5tel-unknown-linux-gnueabi";
}
// platforms.pogoplug4;
ben-nanonote = {
config = "mipsel-unknown-linux-uclibc";
}
// platforms.ben_nanonote;
fuloongminipc = {
config = "mipsel-unknown-linux-gnu";
}
// platforms.fuloong2f_n32;
# can execute on 32bit chip
mips-linux-gnu = {
config = "mips-unknown-linux-gnu";
}
// platforms.gcc_mips32r2_o32;
mipsel-linux-gnu = {
config = "mipsel-unknown-linux-gnu";
}
// platforms.gcc_mips32r2_o32;
# require 64bit chip (for more registers, 64-bit floating point, 64-bit "long long") but use 32bit pointers
mips64-linux-gnuabin32 = {
config = "mips64-unknown-linux-gnuabin32";
}
// platforms.gcc_mips64r2_n32;
mips64el-linux-gnuabin32 = {
config = "mips64el-unknown-linux-gnuabin32";
}
// platforms.gcc_mips64r2_n32;
# 64bit pointers
mips64-linux-gnuabi64 = {
config = "mips64-unknown-linux-gnuabi64";
}
// platforms.gcc_mips64r2_64;
mips64el-linux-gnuabi64 = {
config = "mips64el-unknown-linux-gnuabi64";
}
// platforms.gcc_mips64r2_64;
muslpi = raspberryPi // {
config = "armv6l-unknown-linux-musleabihf";
};
aarch64-multiplatform-musl = {
config = "aarch64-unknown-linux-musl";
};
gnu64 = {
config = "x86_64-unknown-linux-gnu";
};
gnu64_simplekernel = gnu64 // platforms.pc_simplekernel; # see test/cross/default.nix
gnu32 = {
config = "i686-unknown-linux-gnu";
};
musl64 = {
config = "x86_64-unknown-linux-musl";
};
musl32 = {
config = "i686-unknown-linux-musl";
};
riscv64 = riscv "64";
riscv32 = riscv "32";
riscv64-musl = {
config = "riscv64-unknown-linux-musl";
};
riscv64-embedded = {
config = "riscv64-none-elf";
libc = "newlib";
};
riscv32-embedded = {
config = "riscv32-none-elf";
libc = "newlib";
};
mips64-embedded = {
config = "mips64-none-elf";
libc = "newlib";
};
mips-embedded = {
config = "mips-none-elf";
libc = "newlib";
};
# https://github.com/loongson/la-softdev-convention/blob/master/la-softdev-convention.adoc#10-operating-system-package-build-requirements
loongarch64-linux = lib.recursiveUpdate platforms.loongarch64-multiplatform {
config = "loongarch64-unknown-linux-gnu";
};
loongarch64-linux-embedded = lib.recursiveUpdate platforms.loongarch64-multiplatform {
config = "loongarch64-unknown-linux-gnu";
gcc = {
arch = "loongarch64";
strict-align = true;
};
};
mmix = {
config = "mmix-unknown-mmixware";
libc = "newlib";
};
rx-embedded = {
config = "rx-none-elf";
libc = "newlib";
};
msp430 = {
config = "msp430-elf";
libc = "newlib";
};
avr = {
config = "avr";
};
vc4 = {
config = "vc4-elf";
libc = "newlib";
};
or1k = {
config = "or1k-elf";
libc = "newlib";
};
m68k = {
config = "m68k-unknown-linux-gnu";
};
s390 = {
config = "s390-unknown-linux-gnu";
};
s390x = {
config = "s390x-unknown-linux-gnu";
};
arm-embedded = {
config = "arm-none-eabi";
libc = "newlib";
};
arm-embedded-nano = {
config = "arm-none-eabi";
libc = "newlib-nano";
};
armhf-embedded = {
config = "arm-none-eabihf";
libc = "newlib";
# GCC8+ does not build without this
# (https://www.mail-archive.com/gcc-bugs@gcc.gnu.org/msg552339.html):
gcc = {
arch = "armv5t";
fpu = "vfp";
};
};
aarch64-embedded = {
config = "aarch64-none-elf";
libc = "newlib";
rust.rustcTarget = "aarch64-unknown-none";
};
aarch64be-embedded = {
config = "aarch64_be-none-elf";
libc = "newlib";
};
ppc-embedded = {
config = "powerpc-none-eabi";
libc = "newlib";
};
ppcle-embedded = {
config = "powerpcle-none-eabi";
libc = "newlib";
};
i686-embedded = {
config = "i686-elf";
libc = "newlib";
};
x86_64-embedded = {
config = "x86_64-elf";
libc = "newlib";
};
microblaze-embedded = {
config = "microblazeel-none-elf";
libc = "newlib";
};
#
# Redox
#
x86_64-unknown-redox = {
config = "x86_64-unknown-redox";
libc = "relibc";
};
#
# Darwin
#
iphone64 = {
config = "arm64-apple-ios";
# config = "aarch64-apple-darwin14";
darwinSdkVersion = "14.3";
xcodeVer = "12.3";
xcodePlatform = "iPhoneOS";
useiOSPrebuilt = true;
};
iphone64-simulator = {
config = "x86_64-apple-ios";
# config = "x86_64-apple-darwin14";
darwinSdkVersion = "14.3";
xcodeVer = "12.3";
xcodePlatform = "iPhoneSimulator";
darwinPlatform = "ios-simulator";
useiOSPrebuilt = true;
};
aarch64-darwin = {
config = "arm64-apple-darwin";
xcodePlatform = "MacOSX";
platform = { };
};
x86_64-darwin = {
config = "x86_64-apple-darwin";
xcodePlatform = "MacOSX";
platform = { };
};
#
# Windows
#
# 32 bit mingw-w64
mingw32 = {
config = "i686-w64-mingw32";
libc = "msvcrt"; # This distinguishes the mingw (non posix) toolchain
};
# 64 bit mingw-w64
mingwW64 = {
# That's the triplet they use in the mingw-w64 docs.
config = "x86_64-w64-mingw32";
libc = "msvcrt"; # This distinguishes the mingw (non posix) toolchain
};
ucrt64 = {
config = "x86_64-w64-mingw32";
libc = "ucrt"; # This distinguishes the mingw (non posix) toolchain
};
# LLVM-based mingw-w64 for ARM
ucrtAarch64 = {
config = "aarch64-w64-mingw32";
libc = "ucrt";
rust.rustcTarget = "aarch64-pc-windows-gnullvm";
useLLVM = true;
};
# Target the MSVC ABI
x86_64-windows = {
config = "x86_64-pc-windows-msvc";
useLLVM = true;
};
aarch64-windows = {
config = "aarch64-pc-windows-msvc";
useLLVM = true;
};
x86_64-cygwin = {
config = "x86_64-pc-cygwin";
};
# BSDs
aarch64-freebsd = {
config = "aarch64-unknown-freebsd";
useLLVM = true;
};
x86_64-freebsd = {
config = "x86_64-unknown-freebsd";
useLLVM = true;
};
x86_64-netbsd = {
config = "x86_64-unknown-netbsd";
};
# this is broken and never worked fully
x86_64-netbsd-llvm = {
config = "x86_64-unknown-netbsd";
useLLVM = true;
};
x86_64-openbsd = {
config = "x86_64-unknown-openbsd";
useLLVM = true;
};
#
# WASM
#
wasi32 = {
config = "wasm32-unknown-wasi";
useLLVM = true;
};
wasm32-unknown-none = {
config = "wasm32-unknown-none";
rust.rustcTarget = "wasm32-unknown-unknown";
useLLVM = true;
};
# Ghcjs
ghcjs = {
# This triple is special to GHC/Cabal/GHCJS and not recognized by autotools
# See: https://gitlab.haskell.org/ghc/ghc/-/commit/6636b670233522f01d002c9b97827d00289dbf5c
# https://github.com/ghcjs/ghcjs/issues/53
config = "javascript-unknown-ghcjs";
};
}

28
lib/systems/flake-systems.nix Normal file
View File

@@ -0,0 +1,28 @@
# See [RFC 46] for mandated platform support and ../../pkgs/stdenv for
# implemented platform support. This list is mainly descriptive, i.e. all
# system doubles for platforms where nixpkgs can do native compilation
# reasonably well are included.
#
# [RFC 46]: https://github.com/NixOS/rfcs/blob/master/rfcs/0046-platform-support-tiers.md
{ }:
[
# Tier 1
"x86_64-linux"
# Tier 2
"aarch64-linux"
"x86_64-darwin"
# Tier 3
"armv6l-linux"
"armv7l-linux"
"i686-linux"
# "mipsel-linux" is excluded because it is not bootstrapped
# Other platforms with sufficient support in stdenv which is not formally
# mandated by their platform tier.
"aarch64-darwin"
# "armv5tel-linux" is excluded because it is not bootstrapped
"powerpc64le-linux"
"riscv64-linux"
"x86_64-freebsd"
]

489
lib/systems/inspect.nix Normal file
View File

@@ -0,0 +1,489 @@
{ lib }:
let
inherit (lib)
any
attrValues
concatMap
filter
hasPrefix
isList
mapAttrs
matchAttrs
recursiveUpdateUntil
toList
;
inherit (lib.strings) toJSON;
inherit (lib.systems.parse)
kernels
kernelFamilies
significantBytes
cpuTypes
execFormats
;
abis = mapAttrs (_: abi: removeAttrs abi [ "assertions" ]) lib.systems.parse.abis;
in
rec {
# these patterns are to be matched against {host,build,target}Platform.parsed
patterns = rec {
# The patterns below are lists in sum-of-products form.
#
# Each attribute is list of product conditions; non-list values are treated
# as a singleton list. If *any* product condition in the list matches then
# the predicate matches. Each product condition is tested by
# `lib.attrsets.matchAttrs`, which requires a match on *all* attributes of
# the product.
isi686 = {
cpu = cpuTypes.i686;
};
isx86_32 = {
cpu = {
family = "x86";
bits = 32;
};
};
isx86_64 = {
cpu = {
family = "x86";
bits = 64;
};
};
isPower = {
cpu = {
family = "power";
};
};
isPower64 = {
cpu = {
family = "power";
bits = 64;
};
};
isAbiElfv1 = {
abi = {
abi = "elfv1";
};
};
# This ABI is the default in NixOS PowerPC64 BE, but not on mainline GCC,
# so it sometimes causes issues in certain packages that makes the wrong
# assumption on the used ABI.
isAbiElfv2 = [
{
abi = {
abi = "elfv2";
};
}
{
abi = {
name = "musl";
};
cpu = {
family = "power";
bits = 64;
};
}
];
isx86 = {
cpu = {
family = "x86";
};
};
isAarch32 = {
cpu = {
family = "arm";
bits = 32;
};
};
isArmv7 = map (
{ arch, ... }:
{
cpu = { inherit arch; };
}
) (filter (cpu: hasPrefix "armv7" cpu.arch or "") (attrValues cpuTypes));
isAarch64 = {
cpu = {
family = "arm";
bits = 64;
};
};
isAarch = {
cpu = {
family = "arm";
};
};
isMicroBlaze = {
cpu = {
family = "microblaze";
};
};
isMips = {
cpu = {
family = "mips";
};
};
isMips32 = {
cpu = {
family = "mips";
bits = 32;
};
};
isMips64 = {
cpu = {
family = "mips";
bits = 64;
};
};
isMips64n32 = {
cpu = {
family = "mips";
bits = 64;
};
abi = {
abi = "n32";
};
};
isMips64n64 = {
cpu = {
family = "mips";
bits = 64;
};
abi = {
abi = "64";
};
};
isMmix = {
cpu = {
family = "mmix";
};
};
isRiscV = {
cpu = {
family = "riscv";
};
};
isRiscV32 = {
cpu = {
family = "riscv";
bits = 32;
};
};
isRiscV64 = {
cpu = {
family = "riscv";
bits = 64;
};
};
isRx = {
cpu = {
family = "rx";
};
};
isSparc = {
cpu = {
family = "sparc";
};
};
isSparc64 = {
cpu = {
family = "sparc";
bits = 64;
};
};
isWasm = {
cpu = {
family = "wasm";
};
};
isMsp430 = {
cpu = {
family = "msp430";
};
};
isVc4 = {
cpu = {
family = "vc4";
};
};
isAvr = {
cpu = {
family = "avr";
};
};
isAlpha = {
cpu = {
family = "alpha";
};
};
isOr1k = {
cpu = {
family = "or1k";
};
};
isM68k = {
cpu = {
family = "m68k";
};
};
isS390 = {
cpu = {
family = "s390";
};
};
isS390x = {
cpu = {
family = "s390";
bits = 64;
};
};
isLoongArch64 = {
cpu = {
family = "loongarch";
bits = 64;
};
};
isJavaScript = {
cpu = cpuTypes.javascript;
};
is32bit = {
cpu = {
bits = 32;
};
};
is64bit = {
cpu = {
bits = 64;
};
};
isILP32 = [
{
cpu = {
family = "wasm";
bits = 32;
};
}
]
++
map
(a: {
abi = {
abi = a;
};
})
[
"n32"
"ilp32"
"x32"
];
isBigEndian = {
cpu = {
significantByte = significantBytes.bigEndian;
};
};
isLittleEndian = {
cpu = {
significantByte = significantBytes.littleEndian;
};
};
isBSD = {
kernel = {
families = { inherit (kernelFamilies) bsd; };
};
};
isDarwin = {
kernel = {
families = { inherit (kernelFamilies) darwin; };
};
};
isUnix = [
isBSD
isDarwin
isLinux
isSunOS
isCygwin
isRedox
];
isMacOS = {
kernel = kernels.macos;
};
isiOS = {
kernel = kernels.ios;
};
isLinux = {
kernel = kernels.linux;
};
isSunOS = {
kernel = kernels.solaris;
};
isFreeBSD = {
kernel = {
name = "freebsd";
};
};
isNetBSD = {
kernel = kernels.netbsd;
};
isOpenBSD = {
kernel = kernels.openbsd;
};
isWindows = {
kernel = kernels.windows;
};
isCygwin = {
kernel = kernels.cygwin;
};
isMinGW = {
kernel = kernels.windows;
abi = abis.gnu;
};
isMsvc = {
kernel = kernels.windows;
abi = abis.msvc;
};
isWasi = {
kernel = kernels.wasi;
};
isRedox = {
kernel = kernels.redox;
};
isGhcjs = {
kernel = kernels.ghcjs;
};
isGenode = {
kernel = kernels.genode;
};
isNone = {
kernel = kernels.none;
};
isAndroid = [
{ abi = abis.android; }
{ abi = abis.androideabi; }
];
isGnu =
with abis;
map (a: { abi = a; }) [
gnuabi64
gnuabin32
gnu
gnueabi
gnueabihf
gnuabielfv1
gnuabielfv2
];
isMusl =
with abis;
map (a: { abi = a; }) [
musl
musleabi
musleabihf
muslabin32
muslabi64
];
isUClibc =
with abis;
map (a: { abi = a; }) [
uclibc
uclibceabi
uclibceabihf
];
isEfi = [
{
cpu = {
family = "arm";
version = "6";
};
}
{
cpu = {
family = "arm";
version = "7";
};
}
{
cpu = {
family = "arm";
version = "8";
};
}
{
cpu = {
family = "riscv";
};
}
{
cpu = {
family = "x86";
};
}
{
cpu = {
family = "loongarch";
};
}
];
isElf = {
kernel.execFormat = execFormats.elf;
};
isMacho = {
kernel.execFormat = execFormats.macho;
};
};
# given two patterns, return a pattern which is their logical AND.
# Since a pattern is a list-of-disjuncts, this needs to
patternLogicalAnd =
pat1_: pat2_:
let
# patterns can be either a list or a (bare) singleton; turn
# them into singletons for uniform handling
pat1 = toList pat1_;
pat2 = toList pat2_;
in
concatMap (
attr1:
map (
attr2:
recursiveUpdateUntil (
path: subattr1: subattr2:
if (builtins.intersectAttrs subattr1 subattr2) == { } || subattr1 == subattr2 then
true
else
throw ''
pattern conflict at path ${toString path}:
${toJSON subattr1}
${toJSON subattr2}
''
) attr1 attr2
) pat2
) pat1;
matchAnyAttrs =
patterns:
if isList patterns then
attrs: any (pattern: matchAttrs pattern attrs) patterns
else
matchAttrs patterns;
predicates = mapAttrs (_: matchAnyAttrs) patterns;
# these patterns are to be matched against the entire
# {host,build,target}Platform structure; they include a `parsed={}` marker so
# that `lib.meta.availableOn` can distinguish them from the patterns which
# apply only to the `parsed` field.
platformPatterns = mapAttrs (_: p: { parsed = { }; } // p) {
isStatic = {
isStatic = true;
};
};
}

969
lib/systems/parse.nix Normal file
View File

@@ -0,0 +1,969 @@
# Define the list of system with their properties.
#
# See https://clang.llvm.org/docs/CrossCompilation.html and
# http://llvm.org/docs/doxygen/html/Triple_8cpp_source.html especially
# Triple::normalize. Parsing should essentially act as a more conservative
# version of that last function.
#
# Most of the types below come in "open" and "closed" pairs. The open ones
# specify what information we need to know about systems in general, and the
# closed ones are sub-types representing the whitelist of systems we support in
# practice.
#
# Code in the remainder of nixpkgs shouldn't rely on the closed ones in
# e.g. exhaustive cases. Its more a sanity check to make sure nobody defines
# systems that overlap with existing ones and won't notice something amiss.
#
{ lib }:
let
inherit (lib)
all
any
attrValues
elem
elemAt
hasPrefix
id
length
mapAttrs
mergeOneOption
optionalString
splitString
versionAtLeast
;
inherit (lib.strings) match;
inherit (lib.systems.inspect.predicates)
isAarch32
isBigEndian
isDarwin
isLinux
isPower64
isWindows
isCygwin
;
inherit (lib.types)
enum
float
isType
mkOptionType
number
setType
string
types
;
setTypes =
type:
mapAttrs (
name: value:
assert type.check value;
setType type.name ({ inherit name; } // value)
);
# gnu-config will ignore the portion of a triple matching the
# regex `e?abi.*$` when determining the validity of a triple. In
# other words, `i386-linuxabichickenlips` is a valid triple.
removeAbiSuffix =
x:
let
found = match "(.*)e?abi.*" x;
in
if found == null then x else elemAt found 0;
in
rec {
################################################################################
types.openSignificantByte = mkOptionType {
name = "significant-byte";
description = "Endianness";
merge = mergeOneOption;
};
types.significantByte = enum (attrValues significantBytes);
significantBytes = setTypes types.openSignificantByte {
bigEndian = { };
littleEndian = { };
};
################################################################################
# Reasonable power of 2
types.bitWidth = enum [
8
16
32
64
128
];
################################################################################
types.openCpuType = mkOptionType {
name = "cpu-type";
description = "instruction set architecture name and information";
merge = mergeOneOption;
check =
x:
types.bitWidth.check x.bits
&& (if 8 < x.bits then types.significantByte.check x.significantByte else !(x ? significantByte));
};
types.cpuType = enum (attrValues cpuTypes);
cpuTypes =
let
inherit (significantBytes) bigEndian littleEndian;
in
setTypes types.openCpuType {
arm = {
bits = 32;
significantByte = littleEndian;
family = "arm";
};
armv5tel = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "5";
arch = "armv5t";
};
armv6m = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "6";
arch = "armv6-m";
};
armv6l = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "6";
arch = "armv6";
};
armv7a = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "7";
arch = "armv7-a";
};
armv7r = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "7";
arch = "armv7-r";
};
armv7m = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "7";
arch = "armv7-m";
};
armv7l = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "7";
arch = "armv7";
};
armv8a = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "8";
arch = "armv8-a";
};
armv8r = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "8";
arch = "armv8-a";
};
armv8m = {
bits = 32;
significantByte = littleEndian;
family = "arm";
version = "8";
arch = "armv8-m";
};
aarch64 = {
bits = 64;
significantByte = littleEndian;
family = "arm";
version = "8";
arch = "armv8-a";
};
aarch64_be = {
bits = 64;
significantByte = bigEndian;
family = "arm";
version = "8";
arch = "armv8-a";
};
i386 = {
bits = 32;
significantByte = littleEndian;
family = "x86";
arch = "i386";
};
i486 = {
bits = 32;
significantByte = littleEndian;
family = "x86";
arch = "i486";
};
i586 = {
bits = 32;
significantByte = littleEndian;
family = "x86";
arch = "i586";
};
i686 = {
bits = 32;
significantByte = littleEndian;
family = "x86";
arch = "i686";
};
x86_64 = {
bits = 64;
significantByte = littleEndian;
family = "x86";
arch = "x86-64";
};
microblaze = {
bits = 32;
significantByte = bigEndian;
family = "microblaze";
};
microblazeel = {
bits = 32;
significantByte = littleEndian;
family = "microblaze";
};
mips = {
bits = 32;
significantByte = bigEndian;
family = "mips";
};
mipsel = {
bits = 32;
significantByte = littleEndian;
family = "mips";
};
mips64 = {
bits = 64;
significantByte = bigEndian;
family = "mips";
};
mips64el = {
bits = 64;
significantByte = littleEndian;
family = "mips";
};
mmix = {
bits = 64;
significantByte = bigEndian;
family = "mmix";
};
m68k = {
bits = 32;
significantByte = bigEndian;
family = "m68k";
};
powerpc = {
bits = 32;
significantByte = bigEndian;
family = "power";
};
powerpc64 = {
bits = 64;
significantByte = bigEndian;
family = "power";
};
powerpc64le = {
bits = 64;
significantByte = littleEndian;
family = "power";
};
powerpcle = {
bits = 32;
significantByte = littleEndian;
family = "power";
};
riscv32 = {
bits = 32;
significantByte = littleEndian;
family = "riscv";
};
riscv64 = {
bits = 64;
significantByte = littleEndian;
family = "riscv";
};
s390 = {
bits = 32;
significantByte = bigEndian;
family = "s390";
};
s390x = {
bits = 64;
significantByte = bigEndian;
family = "s390";
};
sparc = {
bits = 32;
significantByte = bigEndian;
family = "sparc";
};
sparc64 = {
bits = 64;
significantByte = bigEndian;
family = "sparc";
};
wasm32 = {
bits = 32;
significantByte = littleEndian;
family = "wasm";
};
wasm64 = {
bits = 64;
significantByte = littleEndian;
family = "wasm";
};
alpha = {
bits = 64;
significantByte = littleEndian;
family = "alpha";
};
rx = {
bits = 32;
significantByte = littleEndian;
family = "rx";
};
msp430 = {
bits = 16;
significantByte = littleEndian;
family = "msp430";
};
avr = {
bits = 8;
family = "avr";
};
vc4 = {
bits = 32;
significantByte = littleEndian;
family = "vc4";
};
or1k = {
bits = 32;
significantByte = bigEndian;
family = "or1k";
};
loongarch64 = {
bits = 64;
significantByte = littleEndian;
family = "loongarch";
};
javascript = {
bits = 32;
significantByte = littleEndian;
family = "javascript";
};
}
// {
# aliases
# Apple architecture name, as used by `darwinArch`; required by
# LLVM ≥ 20.
arm64 = cpuTypes.aarch64;
};
# GNU build systems assume that older NetBSD architectures are using a.out.
gnuNetBSDDefaultExecFormat =
cpu:
if
(cpu.family == "arm" && cpu.bits == 32)
|| (cpu.family == "sparc" && cpu.bits == 32)
|| (cpu.family == "m68k" && cpu.bits == 32)
|| (cpu.family == "x86" && cpu.bits == 32)
then
execFormats.aout
else
execFormats.elf;
# Determine when two CPUs are compatible with each other. That is,
# can code built for system B run on system A? For that to happen,
# the programs that system B accepts must be a subset of the
# programs that system A accepts.
#
# We have the following properties of the compatibility relation,
# which must be preserved when adding compatibility information for
# additional CPUs.
# - (reflexivity)
# Every CPU is compatible with itself.
# - (transitivity)
# If A is compatible with B and B is compatible with C then A is compatible with C.
#
# Note: Since 22.11 the archs of a mode switching CPU are no longer considered
# pairwise compatible. Mode switching implies that binaries built for A
# and B respectively can't be executed at the same time.
isCompatible =
with cpuTypes;
a: b:
any id [
# x86
(b == i386 && isCompatible a i486)
(b == i486 && isCompatible a i586)
(b == i586 && isCompatible a i686)
# XXX: Not true in some cases. Like in WSL mode.
(b == i686 && isCompatible a x86_64)
# ARMv4
(b == arm && isCompatible a armv5tel)
# ARMv5
(b == armv5tel && isCompatible a armv6l)
# ARMv6
(b == armv6l && isCompatible a armv6m)
(b == armv6m && isCompatible a armv7l)
# ARMv7
(b == armv7l && isCompatible a armv7a)
(b == armv7l && isCompatible a armv7r)
(b == armv7l && isCompatible a armv7m)
# ARMv8
(b == aarch64 && a == armv8a)
(b == armv8a && isCompatible a aarch64)
(b == armv8r && isCompatible a armv8a)
(b == armv8m && isCompatible a armv8a)
# PowerPC
(b == powerpc && isCompatible a powerpc64)
(b == powerpcle && isCompatible a powerpc64le)
# MIPS
(b == mips && isCompatible a mips64)
(b == mipsel && isCompatible a mips64el)
# RISCV
(b == riscv32 && isCompatible a riscv64)
# SPARC
(b == sparc && isCompatible a sparc64)
# WASM
(b == wasm32 && isCompatible a wasm64)
# identity
(b == a)
];
################################################################################
types.openVendor = mkOptionType {
name = "vendor";
description = "vendor for the platform";
merge = mergeOneOption;
};
types.vendor = enum (attrValues vendors);
vendors = setTypes types.openVendor {
apple = { };
pc = { };
knuth = { };
# Actually matters, unlocking some MinGW-w64-specific options in GCC. See
# bottom of https://sourceforge.net/p/mingw-w64/wiki2/Unicode%20apps/
w64 = { };
none = { };
unknown = { };
};
################################################################################
types.openExecFormat = mkOptionType {
name = "exec-format";
description = "executable container used by the kernel";
merge = mergeOneOption;
};
types.execFormat = enum (attrValues execFormats);
execFormats = setTypes types.openExecFormat {
aout = { }; # a.out
elf = { };
macho = { };
pe = { };
wasm = { };
unknown = { };
};
################################################################################
types.openKernelFamily = mkOptionType {
name = "exec-format";
description = "executable container used by the kernel";
merge = mergeOneOption;
};
types.kernelFamily = enum (attrValues kernelFamilies);
kernelFamilies = setTypes types.openKernelFamily {
bsd = { };
darwin = { };
};
################################################################################
types.openKernel = mkOptionType {
name = "kernel";
description = "kernel name and information";
merge = mergeOneOption;
check =
x: types.execFormat.check x.execFormat && all types.kernelFamily.check (attrValues x.families);
};
types.kernel = enum (attrValues kernels);
kernels =
let
inherit (execFormats)
elf
pe
wasm
unknown
macho
;
inherit (kernelFamilies) bsd darwin;
in
setTypes types.openKernel {
# TODO(@Ericson2314): Don't want to mass-rebuild yet to keeping 'darwin' as
# the normalized name for macOS.
macos = {
execFormat = macho;
families = { inherit darwin; };
name = "darwin";
};
ios = {
execFormat = macho;
families = { inherit darwin; };
};
freebsd = {
execFormat = elf;
families = { inherit bsd; };
name = "freebsd";
};
linux = {
execFormat = elf;
families = { };
};
netbsd = {
execFormat = elf;
families = { inherit bsd; };
};
none = {
execFormat = unknown;
families = { };
};
openbsd = {
execFormat = elf;
families = { inherit bsd; };
};
solaris = {
execFormat = elf;
families = { };
};
wasi = {
execFormat = wasm;
families = { };
};
redox = {
execFormat = elf;
families = { };
};
windows = {
execFormat = pe;
families = { };
};
cygwin = {
execFormat = pe;
families = { };
};
ghcjs = {
execFormat = unknown;
families = { };
};
genode = {
execFormat = elf;
families = { };
};
mmixware = {
execFormat = unknown;
families = { };
};
}
// {
# aliases
# 'darwin' is the kernel for all of them. We choose macOS by default.
darwin = kernels.macos;
watchos = kernels.ios;
tvos = kernels.ios;
win32 = kernels.windows;
};
################################################################################
types.openAbi = mkOptionType {
name = "abi";
description = "binary interface for compiled code and syscalls";
merge = mergeOneOption;
};
types.abi = enum (attrValues abis);
abis = setTypes types.openAbi {
msvc = { };
# Note: eabi is specific to ARM and PowerPC.
# On PowerPC, this corresponds to PPCEABI.
# On ARM, this corresponds to ARMEABI.
eabi = {
float = "soft";
};
eabihf = {
float = "hard";
};
# Other architectures should use ELF in embedded situations.
elf = { };
androideabi = { };
android = {
assertions = [
{
assertion = platform: !platform.isAarch32;
message = ''
The "android" ABI is not for 32-bit ARM. Use "androideabi" instead.
'';
}
];
};
gnueabi = {
float = "soft";
};
gnueabihf = {
float = "hard";
};
gnu = {
assertions = [
{
assertion = platform: !platform.isAarch32;
message = ''
The "gnu" ABI is ambiguous on 32-bit ARM. Use "gnueabi" or "gnueabihf" instead.
'';
}
{
assertion = platform: !(platform.isPower64 && platform.isBigEndian);
message = ''
The "gnu" ABI is ambiguous on big-endian 64-bit PowerPC. Use "gnuabielfv2" or "gnuabielfv1" instead.
'';
}
];
};
gnuabi64 = {
abi = "64";
};
muslabi64 = {
abi = "64";
};
# NOTE: abi=n32 requires a 64-bit MIPS chip! That is not a typo.
# It is basically the 64-bit abi with 32-bit pointers. Details:
# https://www.linux-mips.org/pub/linux/mips/doc/ABI/MIPS-N32-ABI-Handbook.pdf
gnuabin32 = {
abi = "n32";
};
muslabin32 = {
abi = "n32";
};
gnuabielfv2 = {
abi = "elfv2";
};
gnuabielfv1 = {
abi = "elfv1";
};
musleabi = {
float = "soft";
};
musleabihf = {
float = "hard";
};
musl = { };
uclibceabi = {
float = "soft";
};
uclibceabihf = {
float = "hard";
};
uclibc = { };
unknown = { };
};
################################################################################
types.parsedPlatform = mkOptionType {
name = "system";
description = "fully parsed representation of llvm- or nix-style platform tuple";
merge = mergeOneOption;
check =
{
cpu,
vendor,
kernel,
abi,
}:
types.cpuType.check cpu
&& types.vendor.check vendor
&& types.kernel.check kernel
&& types.abi.check abi;
};
isSystem = isType "system";
mkSystem =
components:
assert types.parsedPlatform.check components;
setType "system" components;
mkSkeletonFromList =
l:
{
"1" =
if elemAt l 0 == "avr" then
{
cpu = elemAt l 0;
kernel = "none";
abi = "unknown";
}
else
throw "system string '${lib.concatStringsSep "-" l}' with 1 component is ambiguous";
"2" = # We only do 2-part hacks for things Nix already supports
if elemAt l 1 == "cygwin" then
mkSkeletonFromList [
(elemAt l 0)
"pc"
"cygwin"
]
# MSVC ought to be the default ABI so this case isn't needed. But then it
# becomes difficult to handle the gnu* variants for Aarch32 correctly for
# minGW. So it's easier to make gnu* the default for the MinGW, but
# hack-in MSVC for the non-MinGW case right here.
else if elemAt l 1 == "windows" then
{
cpu = elemAt l 0;
kernel = "windows";
abi = "msvc";
}
else if (elemAt l 1) == "elf" then
{
cpu = elemAt l 0;
vendor = "unknown";
kernel = "none";
abi = elemAt l 1;
}
else
{
cpu = elemAt l 0;
kernel = elemAt l 1;
};
"3" =
# cpu-kernel-environment
if
elemAt l 1 == "linux"
|| elem (elemAt l 2) [
"eabi"
"eabihf"
"elf"
"gnu"
]
then
{
cpu = elemAt l 0;
kernel = elemAt l 1;
abi = elemAt l 2;
vendor = "unknown";
}
# cpu-vendor-os
else if
elemAt l 1 == "apple"
|| elem (elemAt l 2) [
"redox"
"mmixware"
"ghcjs"
"mingw32"
]
|| hasPrefix "freebsd" (elemAt l 2)
|| hasPrefix "netbsd" (elemAt l 2)
|| hasPrefix "openbsd" (elemAt l 2)
|| hasPrefix "genode" (elemAt l 2)
|| hasPrefix "wasm32" (elemAt l 0)
then
{
cpu = elemAt l 0;
vendor = elemAt l 1;
kernel =
if elemAt l 2 == "mingw32" then
"windows" # autotools breaks on -gnu for window
else
elemAt l 2;
}
# lots of tools expect a triplet for Cygwin, even though the vendor is just "pc"
else if elemAt l 2 == "cygwin" then
{
cpu = elemAt l 0;
vendor = elemAt l 1;
kernel = "cygwin";
}
else
throw "system string '${lib.concatStringsSep "-" l}' with 3 components is ambiguous";
"4" = {
cpu = elemAt l 0;
vendor = elemAt l 1;
kernel = elemAt l 2;
abi = elemAt l 3;
};
}
.${toString (length l)}
or (throw "system string '${lib.concatStringsSep "-" l}' has invalid number of hyphen-separated components");
# This should revert the job done by config.guess from the gcc compiler.
mkSystemFromSkeleton =
{
cpu,
# Optional, but fallback too complex for here.
# Inferred below instead.
vendor ?
assert false;
null,
kernel,
# Also inferred below
abi ?
assert false;
null,
}@args:
let
getCpu = name: cpuTypes.${name} or (throw "Unknown CPU type: ${name}");
getVendor = name: vendors.${name} or (throw "Unknown vendor: ${name}");
getKernel = name: kernels.${name} or (throw "Unknown kernel: ${name}");
getAbi = name: abis.${name} or (throw "Unknown ABI: ${name}");
parsed = {
cpu = getCpu args.cpu;
vendor =
if args ? vendor then
getVendor args.vendor
else if isDarwin parsed then
vendors.apple
else if (isWindows parsed || isCygwin parsed) then
vendors.pc
else
vendors.unknown;
kernel =
if hasPrefix "darwin" args.kernel then
getKernel "darwin"
else if hasPrefix "netbsd" args.kernel then
getKernel "netbsd"
else
getKernel (removeAbiSuffix args.kernel);
abi =
if args ? abi then
getAbi args.abi
else if isLinux parsed || isWindows parsed then
if isAarch32 parsed then
if versionAtLeast (parsed.cpu.version or "0") "6" then abis.gnueabihf else abis.gnueabi
# Default ppc64 BE to ELFv2
else if isPower64 parsed && isBigEndian parsed then
abis.gnuabielfv2
else
abis.gnu
else
abis.unknown;
};
in
mkSystem parsed;
mkSystemFromString = s: mkSystemFromSkeleton (mkSkeletonFromList (splitString "-" s));
kernelName = kernel: kernel.name + toString (kernel.version or "");
darwinArch = cpu: if cpu.name == "aarch64" then "arm64" else cpu.name;
doubleFromSystem =
{
cpu,
kernel,
abi,
...
}:
if kernel.families ? darwin then "${cpu.name}-darwin" else "${cpu.name}-${kernelName kernel}";
tripleFromSystem =
{
cpu,
vendor,
kernel,
abi,
...
}@sys:
assert isSystem sys;
let
optExecFormat = optionalString (
kernel.name == "netbsd" && gnuNetBSDDefaultExecFormat cpu != kernel.execFormat
) kernel.execFormat.name;
optAbi = optionalString (abi != abis.unknown) "-${abi.name}";
cpuName = if kernel.families ? darwin then darwinArch cpu else cpu.name;
in
"${cpuName}-${vendor.name}-${kernelName kernel}${optExecFormat}${optAbi}";
################################################################################
}

638
lib/systems/platforms.nix Normal file
View File

@@ -0,0 +1,638 @@
# Note: lib/systems/default.nix takes care of producing valid,
# fully-formed "platform" values (e.g. hostPlatform, buildPlatform,
# targetPlatform, etc) containing at least the minimal set of attrs
# required (see types.parsedPlatform in lib/systems/parse.nix). This
# file takes an already-valid platform and further elaborates it with
# optional fields; currently these are: linux-kernel, gcc, and rustc.
{ lib }:
rec {
pc = {
linux-kernel = {
name = "pc";
baseConfig = "defconfig";
# Build whatever possible as a module, if not stated in the extra config.
autoModules = true;
target = "bzImage";
};
};
pc_simplekernel = lib.recursiveUpdate pc {
linux-kernel.autoModules = false;
};
powernv = {
linux-kernel = {
name = "PowerNV";
baseConfig = "powernv_defconfig";
target = "vmlinux";
autoModules = true;
# avoid driver/FS trouble arising from unusual page size
extraConfig = ''
PPC_64K_PAGES n
PPC_4K_PAGES y
IPV6 y
ATA_BMDMA y
ATA_SFF y
VIRTIO_MENU y
'';
};
};
##
## ARM
##
pogoplug4 = {
linux-kernel = {
name = "pogoplug4";
baseConfig = "multi_v5_defconfig";
autoModules = false;
extraConfig = ''
# Ubi for the mtd
MTD_UBI y
UBIFS_FS y
UBIFS_FS_XATTR y
UBIFS_FS_ADVANCED_COMPR y
UBIFS_FS_LZO y
UBIFS_FS_ZLIB y
UBIFS_FS_DEBUG n
'';
makeFlags = [ "LOADADDR=0x8000" ];
target = "uImage";
# TODO reenable once manual-config's config actually builds a .dtb and this is checked to be working
#DTB = true;
};
gcc = {
arch = "armv5te";
};
};
sheevaplug = {
linux-kernel = {
name = "sheevaplug";
baseConfig = "multi_v5_defconfig";
autoModules = false;
extraConfig = ''
BLK_DEV_RAM y
BLK_DEV_INITRD y
BLK_DEV_CRYPTOLOOP m
BLK_DEV_DM m
DM_CRYPT m
MD y
REISERFS_FS m
BTRFS_FS m
XFS_FS m
JFS_FS m
EXT4_FS m
USB_STORAGE_CYPRESS_ATACB m
# mv cesa requires this sw fallback, for mv-sha1
CRYPTO_SHA1 y
# Fast crypto
CRYPTO_TWOFISH y
CRYPTO_TWOFISH_COMMON y
CRYPTO_BLOWFISH y
CRYPTO_BLOWFISH_COMMON y
IP_PNP y
IP_PNP_DHCP y
NFS_FS y
ROOT_NFS y
TUN m
NFS_V4 y
NFS_V4_1 y
NFS_FSCACHE y
NFSD m
NFSD_V2_ACL y
NFSD_V3 y
NFSD_V3_ACL y
NFSD_V4 y
NETFILTER y
IP_NF_IPTABLES y
IP_NF_FILTER y
IP_NF_MATCH_ADDRTYPE y
IP_NF_TARGET_LOG y
IP_NF_MANGLE y
IPV6 m
VLAN_8021Q m
CIFS y
CIFS_XATTR y
CIFS_POSIX y
CIFS_FSCACHE y
CIFS_ACL y
WATCHDOG y
WATCHDOG_CORE y
ORION_WATCHDOG m
ZRAM m
NETCONSOLE m
# Disable OABI to have seccomp_filter (required for systemd)
# https://github.com/raspberrypi/firmware/issues/651
OABI_COMPAT n
# Fail to build
DRM n
SCSI_ADVANSYS n
USB_ISP1362_HCD n
SND_SOC n
SND_ALI5451 n
FB_SAVAGE n
SCSI_NSP32 n
ATA_SFF n
SUNGEM n
IRDA n
ATM_HE n
SCSI_ACARD n
BLK_DEV_CMD640_ENHANCED n
FUSE_FS m
# systemd uses cgroups
CGROUPS y
# Latencytop
LATENCYTOP y
# Ubi for the mtd
MTD_UBI y
UBIFS_FS y
UBIFS_FS_XATTR y
UBIFS_FS_ADVANCED_COMPR y
UBIFS_FS_LZO y
UBIFS_FS_ZLIB y
UBIFS_FS_DEBUG n
# Kdb, for kernel troubles
KGDB y
KGDB_SERIAL_CONSOLE y
KGDB_KDB y
'';
makeFlags = [ "LOADADDR=0x0200000" ];
target = "uImage";
DTB = true; # Beyond 3.10
};
gcc = {
arch = "armv5te";
};
};
raspberrypi = {
linux-kernel = {
name = "raspberrypi";
baseConfig = "bcm2835_defconfig";
DTB = true;
autoModules = true;
preferBuiltin = true;
extraConfig = ''
# Disable OABI to have seccomp_filter (required for systemd)
# https://github.com/raspberrypi/firmware/issues/651
OABI_COMPAT n
'';
target = "zImage";
};
gcc = {
# https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications
arch = "armv6kz";
fpu = "vfpv2";
};
};
# Legacy attribute, for compatibility with existing configs only.
raspberrypi2 = armv7l-hf-multiplatform;
# Nvidia Bluefield 2 (w. crypto support)
bluefield2 = {
gcc = {
arch = "armv8-a+fp+simd+crc+crypto";
};
};
zero-gravitas = {
linux-kernel = {
name = "zero-gravitas";
baseConfig = "zero-gravitas_defconfig";
# Target verified by checking /boot on reMarkable 1 device
target = "zImage";
autoModules = false;
DTB = true;
};
gcc = {
fpu = "neon";
cpu = "cortex-a9";
};
};
zero-sugar = {
linux-kernel = {
name = "zero-sugar";
baseConfig = "zero-sugar_defconfig";
DTB = true;
autoModules = false;
preferBuiltin = true;
target = "zImage";
};
gcc = {
cpu = "cortex-a7";
fpu = "neon-vfpv4";
float-abi = "hard";
};
};
utilite = {
linux-kernel = {
name = "utilite";
maseConfig = "multi_v7_defconfig";
autoModules = false;
extraConfig = ''
# Ubi for the mtd
MTD_UBI y
UBIFS_FS y
UBIFS_FS_XATTR y
UBIFS_FS_ADVANCED_COMPR y
UBIFS_FS_LZO y
UBIFS_FS_ZLIB y
UBIFS_FS_DEBUG n
'';
makeFlags = [ "LOADADDR=0x10800000" ];
target = "uImage";
DTB = true;
};
gcc = {
cpu = "cortex-a9";
fpu = "neon";
};
};
guruplug = lib.recursiveUpdate sheevaplug {
# Define `CONFIG_MACH_GURUPLUG' (see
# <http://kerneltrap.org/mailarchive/git-commits-head/2010/5/19/33618>)
# and other GuruPlug-specific things. Requires the `guruplug-defconfig'
# patch.
linux-kernel.baseConfig = "guruplug_defconfig";
};
beaglebone = lib.recursiveUpdate armv7l-hf-multiplatform {
linux-kernel = {
name = "beaglebone";
baseConfig = "bb.org_defconfig";
autoModules = false;
extraConfig = ""; # TBD kernel config
target = "zImage";
};
};
# https://developer.android.com/ndk/guides/abis#v7a
armv7a-android = {
linux-kernel.name = "armeabi-v7a";
gcc = {
arch = "armv7-a";
float-abi = "softfp";
fpu = "vfpv3-d16";
};
};
armv7l-hf-multiplatform = {
linux-kernel = {
name = "armv7l-hf-multiplatform";
Major = "2.6"; # Using "2.6" enables 2.6 kernel syscalls in glibc.
baseConfig = "multi_v7_defconfig";
DTB = true;
autoModules = true;
preferBuiltin = true;
target = "zImage";
extraConfig = ''
# Serial port for Raspberry Pi 3. Wasn't included in ARMv7 defconfig
# until 4.17.
SERIAL_8250_BCM2835AUX y
SERIAL_8250_EXTENDED y
SERIAL_8250_SHARE_IRQ y
# Hangs ODROID-XU4
ARM_BIG_LITTLE_CPUIDLE n
# Disable OABI to have seccomp_filter (required for systemd)
# https://github.com/raspberrypi/firmware/issues/651
OABI_COMPAT n
# >=5.12 fails with:
# drivers/net/ethernet/micrel/ks8851_common.o: in function `ks8851_probe_common':
# ks8851_common.c:(.text+0x179c): undefined reference to `__this_module'
# See: https://lore.kernel.org/netdev/20210116164828.40545-1-marex@denx.de/T/
KS8851_MLL y
'';
};
gcc = {
# Some table about fpu flags:
# http://community.arm.com/servlet/JiveServlet/showImage/38-1981-3827/blogentry-103749-004812900+1365712953_thumb.png
# Cortex-A5: -mfpu=neon-fp16
# Cortex-A7 (rpi2): -mfpu=neon-vfpv4
# Cortex-A8 (beaglebone): -mfpu=neon
# Cortex-A9: -mfpu=neon-fp16
# Cortex-A15: -mfpu=neon-vfpv4
# More about FPU:
# https://wiki.debian.org/ArmHardFloatPort/VfpComparison
# vfpv3-d16 is what Debian uses and seems to be the best compromise: NEON is not supported in e.g. Scaleway or Tegra 2,
# and the above page suggests NEON is only an improvement with hand-written assembly.
arch = "armv7-a";
fpu = "vfpv3-d16";
# For Raspberry Pi the 2 the best would be:
# cpu = "cortex-a7";
# fpu = "neon-vfpv4";
};
};
aarch64-multiplatform = {
linux-kernel = {
name = "aarch64-multiplatform";
baseConfig = "defconfig";
DTB = true;
autoModules = true;
preferBuiltin = true;
extraConfig = ''
# Raspberry Pi 3 stuff. Not needed for s >= 4.10.
ARCH_BCM2835 y
BCM2835_MBOX y
BCM2835_WDT y
RASPBERRYPI_FIRMWARE y
RASPBERRYPI_POWER y
SERIAL_8250_BCM2835AUX y
SERIAL_8250_EXTENDED y
SERIAL_8250_SHARE_IRQ y
# Cavium ThunderX stuff.
PCI_HOST_THUNDER_ECAM y
# Nvidia Tegra stuff.
PCI_TEGRA y
# The default (=y) forces us to have the XHCI firmware available in initrd,
# which our initrd builder can't currently do easily.
USB_XHCI_TEGRA m
'';
target = "Image";
};
gcc = {
arch = "armv8-a";
};
};
apple-m1 = {
gcc = {
arch = "armv8.3-a+crypto+sha2+aes+crc+fp16+lse+simd+ras+rdm+rcpc";
cpu = "apple-a13";
};
};
##
## MIPS
##
ben_nanonote = {
linux-kernel = {
name = "ben_nanonote";
};
gcc = {
arch = "mips32";
float = "soft";
};
};
fuloong2f_n32 = {
linux-kernel = {
name = "fuloong2f_n32";
baseConfig = "lemote2f_defconfig";
autoModules = false;
extraConfig = ''
MIGRATION n
COMPACTION n
# nixos mounts some cgroup
CGROUPS y
BLK_DEV_RAM y
BLK_DEV_INITRD y
BLK_DEV_CRYPTOLOOP m
BLK_DEV_DM m
DM_CRYPT m
MD y
REISERFS_FS m
EXT4_FS m
USB_STORAGE_CYPRESS_ATACB m
IP_PNP y
IP_PNP_DHCP y
IP_PNP_BOOTP y
NFS_FS y
ROOT_NFS y
TUN m
NFS_V4 y
NFS_V4_1 y
NFS_FSCACHE y
NFSD m
NFSD_V2_ACL y
NFSD_V3 y
NFSD_V3_ACL y
NFSD_V4 y
# Fail to build
DRM n
SCSI_ADVANSYS n
USB_ISP1362_HCD n
SND_SOC n
SND_ALI5451 n
FB_SAVAGE n
SCSI_NSP32 n
ATA_SFF n
SUNGEM n
IRDA n
ATM_HE n
SCSI_ACARD n
BLK_DEV_CMD640_ENHANCED n
FUSE_FS m
# Needed for udev >= 150
SYSFS_DEPRECATED_V2 n
VGA_CONSOLE n
VT_HW_CONSOLE_BINDING y
SERIAL_8250_CONSOLE y
FRAMEBUFFER_CONSOLE y
EXT2_FS y
EXT3_FS y
REISERFS_FS y
MAGIC_SYSRQ y
# The kernel doesn't boot at all, with FTRACE
FTRACE n
'';
target = "vmlinux";
};
gcc = {
arch = "loongson2f";
float = "hard";
abi = "n32";
};
};
# can execute on 32bit chip
gcc_mips32r2_o32 = {
gcc = {
arch = "mips32r2";
abi = "32";
};
};
gcc_mips32r6_o32 = {
gcc = {
arch = "mips32r6";
abi = "32";
};
};
gcc_mips64r2_n32 = {
gcc = {
arch = "mips64r2";
abi = "n32";
};
};
gcc_mips64r6_n32 = {
gcc = {
arch = "mips64r6";
abi = "n32";
};
};
gcc_mips64r2_64 = {
gcc = {
arch = "mips64r2";
abi = "64";
};
};
gcc_mips64r6_64 = {
gcc = {
arch = "mips64r6";
abi = "64";
};
};
# based on:
# https://www.mail-archive.com/qemu-discuss@nongnu.org/msg05179.html
# https://gmplib.org/~tege/qemu.html#mips64-debian
mips64el-qemu-linux-gnuabi64 = {
linux-kernel = {
name = "mips64el";
baseConfig = "64r2el_defconfig";
target = "vmlinuz";
autoModules = false;
DTB = true;
# for qemu 9p passthrough filesystem
extraConfig = ''
MIPS_MALTA y
PAGE_SIZE_4KB y
CPU_LITTLE_ENDIAN y
CPU_MIPS64_R2 y
64BIT y
CPU_MIPS64_R2 y
NET_9P y
NET_9P_VIRTIO y
9P_FS y
9P_FS_POSIX_ACL y
PCI y
VIRTIO_PCI y
'';
};
};
##
## Other
##
riscv-multiplatform = {
linux-kernel = {
name = "riscv-multiplatform";
target = "Image";
autoModules = true;
preferBuiltin = true;
baseConfig = "defconfig";
DTB = true;
};
};
loongarch64-multiplatform = {
gcc = {
# https://github.com/loongson/la-softdev-convention/blob/master/la-softdev-convention.adoc#10-operating-system-package-build-requirements
arch = "la64v1.0";
strict-align = false;
# Avoid text sections of large apps exceeding default code model
# Will be default behavior in LLVM 21 and hopefully GCC16
# https://github.com/loongson-community/discussions/issues/43
# https://github.com/llvm/llvm-project/pull/132173
cmodel = "medium";
};
linux-kernel = {
name = "loongarch-multiplatform";
target = "vmlinuz.efi";
autoModules = true;
preferBuiltin = true;
baseConfig = "defconfig";
DTB = true;
};
};
# This function takes a minimally-valid "platform" and returns an
# attrset containing zero or more additional attrs which should be
# included in the platform in order to further elaborate it.
select =
platform:
# x86
if platform.isx86 then
pc
# ARM
else if platform.isAarch32 then
let
version = platform.parsed.cpu.version or null;
in
if version == null then
pc
else if lib.versionOlder version "6" then
sheevaplug
else if lib.versionOlder version "7" then
raspberrypi
else
armv7l-hf-multiplatform
else if platform.isAarch64 then
if platform.isDarwin then apple-m1 else aarch64-multiplatform
else if platform.isLoongArch64 then
loongarch64-multiplatform
else if platform.isRiscV then
riscv-multiplatform
else if platform.parsed.cpu == lib.systems.parse.cpuTypes.mipsel then
(import ./examples.nix { inherit lib; }).mipsel-linux-gnu
else if platform.parsed.cpu == lib.systems.parse.cpuTypes.powerpc64le then
powernv
else if platform.isLoongArch64 then
loongarch64-multiplatform
else
{ };
}

3
lib/tests/.editorconfig Normal file
View File

@@ -0,0 +1,3 @@
[*.plist]
indent_style = tab
insert_final_newline = unset

379
lib/tests/checkAndMergeCompat.nix Normal file
View File

@@ -0,0 +1,379 @@
{
pkgs ? import ../.. { },
currLibPath ? ../.,
prevLibPath ? "${
pkgs.fetchFromGitHub {
owner = "nixos";
repo = "nixpkgs";
# Parent commit of [#391544](https://github.com/NixOS/nixpkgs/pull/391544)
# Which was before the type.merge.v2 introduction
rev = "bcf94dd3f07189b7475d823c8d67d08b58289905";
hash = "sha256-MuMiIY3MX5pFSOCvutmmRhV6RD0R3CG0Hmazkg8cMFI=";
}
}/lib",
}:
let
lib = import currLibPath;
lib_with_merge_v2 = lib;
lib_with_merge_v1 = import prevLibPath;
getMatrix =
{
getType ? null,
# If getType is set this is only used as test prefix
# And the type from getType is used
outerTypeName,
innerTypeName,
value,
testAttrs,
}:
let
evalModules.call_v1 = lib_with_merge_v1.evalModules;
evalModules.call_v2 = lib_with_merge_v2.evalModules;
outerTypes.outer_v1 = lib_with_merge_v1.types;
outerTypes.outer_v2 = lib_with_merge_v2.types;
innerTypes.inner_v1 = lib_with_merge_v1.types;
innerTypes.inner_v2 = lib_with_merge_v2.types;
in
lib.mapAttrs (
_: evalModules:
lib.mapAttrs (
_: outerTypes:
lib.mapAttrs (_: innerTypes: {
"test_${outerTypeName}_${innerTypeName}" = testAttrs // {
expr =
(evalModules {
modules = [
(m: {
options.foo = m.lib.mkOption {
type =
if getType != null then
getType outerTypes innerTypes
else
outerTypes.${outerTypeName} innerTypes.${innerTypeName};
default = value;
};
})
];
}).config.foo;
};
}) innerTypes
) outerTypes
) evalModules;
in
{
# AttrsOf string
attrsOf_str_ok = getMatrix {
outerTypeName = "attrsOf";
innerTypeName = "str";
value = {
bar = "test";
};
testAttrs = {
expected = {
bar = "test";
};
};
};
attrsOf_str_err_inner = getMatrix {
outerTypeName = "attrsOf";
innerTypeName = "str";
value = {
bar = 1; # not a string
};
testAttrs = {
expectedError = {
type = "ThrownError";
msg = "A definition for option `foo.bar' is not of type `string'.*";
};
};
};
attrsOf_str_err_outer = getMatrix {
outerTypeName = "attrsOf";
innerTypeName = "str";
value = [ "foo" ]; # not an attrset
testAttrs = {
expectedError = {
type = "ThrownError";
msg = "A definition for option `foo' is not of type `attribute set of string'.*";
};
};
};
# listOf string
listOf_str_ok = getMatrix {
outerTypeName = "listOf";
innerTypeName = "str";
value = [
"foo"
"bar"
];
testAttrs = {
expected = [
"foo"
"bar"
];
};
};
listOf_str_err_inner = getMatrix {
outerTypeName = "listOf";
innerTypeName = "str";
value = [
"foo"
1
]; # not a string
testAttrs = {
expectedError = {
type = "ThrownError";
msg = ''A definition for option `foo."\[definition 1-entry 2\]"' is not of type `string'.'';
};
};
};
listOf_str_err_outer = getMatrix {
outerTypeName = "listOf";
innerTypeName = "str";
value = {
foo = 42;
}; # not a list
testAttrs = {
expectedError = {
type = "ThrownError";
msg = "A definition for option `foo' is not of type `list of string'.*";
};
};
};
attrsOf_submodule_ok = getMatrix {
getType =
a: b:
a.attrsOf (
b.submodule (m: {
options.nested = m.lib.mkOption {
type = m.lib.types.str;
};
})
);
outerTypeName = "attrsOf";
innerTypeName = "submodule";
value = {
foo = {
nested = "test1";
};
bar = {
nested = "test2";
};
};
testAttrs = {
expected = {
foo = {
nested = "test1";
};
bar = {
nested = "test2";
};
};
};
};
attrsOf_submodule_err_inner = getMatrix {
outerTypeName = "attrsOf";
innerTypeName = "submodule";
getType =
a: b:
a.attrsOf (
b.submodule (m: {
options.nested = m.lib.mkOption {
type = m.lib.types.str;
};
})
);
value = {
foo = [ 1 ]; # not a submodule
bar = {
nested = "test2";
};
};
testAttrs = {
expectedError = {
type = "ThrownError";
msg = "A definition for option `foo.foo' is not of type `submodule'.*";
};
};
};
attrsOf_submodule_err_outer = getMatrix {
outerTypeName = "attrsOf";
innerTypeName = "submodule";
getType =
a: b:
a.attrsOf (
b.submodule (m: {
options.nested = m.lib.mkOption {
type = m.lib.types.str;
};
})
);
value = [ 123 ]; # not an attrsOf
testAttrs = {
expectedError = {
type = "ThrownError";
msg = ''A definition for option `foo' is not of type `attribute set of \(submodule\).*'';
};
};
};
# either
either_str_attrsOf_ok = getMatrix {
outerTypeName = "either";
innerTypeName = "str_or_attrsOf_str";
getType = a: b: a.either b.str (b.attrsOf a.str);
value = "string value";
testAttrs = {
expected = "string value";
};
};
either_str_attrsOf_err_1 = getMatrix {
outerTypeName = "either";
innerTypeName = "str_or_attrsOf_str";
getType = a: b: a.either b.str (b.attrsOf a.str);
value = 1;
testAttrs = {
expectedError = {
type = "ThrownError";
msg = "A definition for option `foo' is not of type `string or attribute set of string'.*";
};
};
};
either_str_attrsOf_err_2 = getMatrix {
outerTypeName = "either";
innerTypeName = "str_or_attrsOf_str";
getType = a: b: a.either b.str (b.attrsOf a.str);
value = {
bar = 1; # not a string
};
testAttrs = {
expectedError = {
type = "ThrownError";
msg = "A definition for option `foo.bar' is not of type `string'.*";
};
};
};
# Coereced to
coerce_attrsOf_str_to_listOf_str_run = getMatrix {
outerTypeName = "coercedTo";
innerTypeName = "attrsOf_str->listOf_str";
getType = a: b: a.coercedTo (b.attrsOf b.str) builtins.attrValues (b.listOf b.str);
value = {
bar = "test1"; # coerced to listOf string
foo = "test2"; # coerced to listOf string
};
testAttrs = {
expected = [
"test1"
"test2"
];
};
};
coerce_attrsOf_str_to_listOf_str_final = getMatrix {
outerTypeName = "coercedTo";
innerTypeName = "attrsOf_str->listOf_str";
getType = a: b: a.coercedTo (b.attrsOf b.str) (abort "This shouldnt run") (b.listOf b.str);
value = [
"test1"
"test2"
]; # already a listOf string
testAttrs = {
expected = [
"test1"
"test2"
]; # Order should be kept
};
};
coerce_attrsOf_str_to_listOf_err_coercer_input = getMatrix {
outerTypeName = "coercedTo";
innerTypeName = "attrsOf_str->listOf_str";
getType = a: b: a.coercedTo (b.attrsOf b.str) builtins.attrValues (b.listOf b.str);
value = [
{ }
{ }
]; # not coercible to listOf string, with the given coercer
testAttrs = {
expectedError = {
type = "ThrownError";
msg = ''A definition for option `foo."\[definition 1-entry 1\]"' is not of type `string'.*'';
};
};
};
coerce_attrsOf_str_to_listOf_err_coercer_ouput = getMatrix {
outerTypeName = "coercedTo";
innerTypeName = "attrsOf_str->listOf_str";
getType = a: b: a.coercedTo (b.attrsOf b.str) builtins.attrValues (b.listOf b.str);
value = {
foo = {
bar = 1;
}; # coercer produces wrong type -> [ { bar = 1; } ]
};
testAttrs = {
expectedError = {
type = "ThrownError";
msg = ''A definition for option `foo."\[definition 1-entry 1\]"' is not of type `string'.*'';
};
};
};
coerce_str_to_int_coercer_ouput = getMatrix {
outerTypeName = "coercedTo";
innerTypeName = "int->str";
getType = a: b: a.coercedTo b.int toString a.str;
value = [ ];
testAttrs = {
expectedError = {
type = "ThrownError";
msg = ''A definition for option `foo' is not of type `string or signed integer convertible to it.*'';
};
};
};
# Submodule
submodule_with_ok = getMatrix {
outerTypeName = "submoduleWith";
innerTypeName = "mixed_types";
getType =
a: b:
a.submodule (m: {
options.attrs = m.lib.mkOption {
type = b.attrsOf b.str;
};
options.list = m.lib.mkOption {
type = b.listOf b.str;
};
options.either = m.lib.mkOption {
type = b.either a.str a.int;
};
});
value = {
attrs = {
foo = "bar";
};
list = [
"foo"
"bar"
];
either = 123; # int
};
testAttrs = {
expected = {
attrs = {
foo = "bar";
};
list = [
"foo"
"bar"
];
either = 123;
};
};
};
}

165
lib/tests/fetchers.nix Normal file
View File

@@ -0,0 +1,165 @@
let
lib = import ./..;
inherit (lib)
fakeHash
fakeSha256
fakeSha512
flip
functionArgs
runTests
;
inherit (lib.fetchers) normalizeHash withNormalizedHash;
testingThrow = expr: {
expr = with builtins; tryEval (seq expr "didn't throw");
expected = {
success = false;
value = false;
};
};
# hashes of empty
sri256 = "sha256-d6xi4mKdjkX2JFicDIv5niSzpyI0m/Hnm8GGAIU04kY=";
sri512 = "sha512-AXFyVo7jiZ5we10fxZ5E9qfPjSfqkizY2apCzORKFVYZaNhCIVbooY+J4cYST00ztLf0EjivIBPPdtIYFUMfzQ==";
unionOfDisjoints = lib.foldl lib.attrsets.unionOfDisjoint { };
genTests = n: f: {
"test${n}AlreadyNormalized" = {
expr = f { } {
outputHash = "";
outputHashAlgo = "md42";
};
expected = {
outputHash = "";
outputHashAlgo = "md42";
};
};
"test${n}EmptySha256" = {
expr = f { } { sha256 = ""; };
expected = {
outputHash = fakeSha256;
outputHashAlgo = "sha256";
};
};
"test${n}EmptySha512" = {
expr = f { hashTypes = [ "sha512" ]; } { sha512 = ""; };
expected = {
outputHash = fakeSha512;
outputHashAlgo = "sha512";
};
};
"test${n}EmptyHash" = {
expr = f { } { hash = ""; };
expected = {
outputHash = fakeHash;
outputHashAlgo = null;
};
};
"test${n}Sri256" = {
expr = f { } { hash = sri256; };
expected = {
outputHash = sri256;
outputHashAlgo = null;
};
};
"test${n}Sri512" = {
expr = f { } { hash = sri512; };
expected = {
outputHash = sri512;
outputHashAlgo = null;
};
};
"test${n}PreservesAttrs" = {
expr = f { } {
hash = "aaaa";
destination = "Earth";
};
expected = {
outputHash = "aaaa";
outputHashAlgo = null;
destination = "Earth";
};
};
"test${n}RejectsSha1ByDefault" = testingThrow (f { } { sha1 = ""; });
"test${n}RejectsSha512ByDefault" = testingThrow (f { } { sha512 = ""; });
"test${n}ThrowsOnMissing" = testingThrow (f { } { gibi = false; });
};
in
runTests (unionOfDisjoints [
(genTests "NormalizeHash" normalizeHash)
(genTests "WithNormalized" (
flip withNormalizedHash ({ outputHash, outputHashAlgo, ... }@args: args)
))
{
testNormalizeNotRequiredEquivalent = {
expr = normalizeHash { required = false; } {
hash = "";
prof = "shadoko";
};
expected = normalizeHash { } {
hash = "";
prof = "shadoko";
};
};
testNormalizeNotRequiredPassthru = {
expr = normalizeHash { required = false; } { "ga bu" = "zo meu"; };
expected."ga bu" = "zo meu";
};
testOptionalArg = {
expr = withNormalizedHash { } (
{
outputHash ? "",
outputHashAlgo ? null,
...
}@args:
args
) { author = "Jacques Rouxel"; };
expected.author = "Jacques Rouxel";
};
testOptionalArgMetadata = {
expr = functionArgs (
withNormalizedHash { } (
{
outputHash ? "",
outputHashAlgo ? null,
}:
{ }
)
);
expected.hash = true;
};
testPreservesArgsMetadata = {
expr = functionArgs (
withNormalizedHash { } (
{
outputHash,
outputHashAlgo,
pumping ? true,
}:
{ }
)
);
expected = {
hash = false;
pumping = true;
};
};
testRejectsMissingHashArg = testingThrow (withNormalizedHash { } ({ outputHashAlgo }: { }));
testRejectsMissingAlgoArg = testingThrow (withNormalizedHash { } ({ outputHash }: { }));
}
])

90
lib/tests/filesystem.sh Executable file
View File

@@ -0,0 +1,90 @@
#!/usr/bin/env bash
# Tests lib/filesystem.nix
# Run:
# [nixpkgs]$ lib/tests/filesystem.sh
# or:
# [nixpkgs]$ nix-build lib/tests/release.nix
set -euo pipefail
shopt -s inherit_errexit
# Use
# || die
die() {
echo >&2 "test case failed: " "$@"
exit 1
}
if test -n "${TEST_LIB:-}"; then
NIX_PATH=nixpkgs="$(dirname "$TEST_LIB")"
else
NIX_PATH=nixpkgs="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.."; pwd)"
fi
export NIX_PATH
work="$(mktemp -d)"
clean_up() {
rm -rf "$work"
}
trap clean_up EXIT
cd "$work"
mkdir directory
touch regular
ln -s target symlink
mkfifo fifo
expectSuccess() {
local expr=$1
local expectedResultRegex=$2
if ! result=$(nix-instantiate --eval --strict --json \
--expr "with (import <nixpkgs/lib>).filesystem; $expr"); then
die "$expr failed to evaluate, but it was expected to succeed"
fi
if [[ ! "$result" =~ $expectedResultRegex ]]; then
die "$expr == $result, but $expectedResultRegex was expected"
fi
}
expectFailure() {
local expr=$1
local expectedErrorRegex=$2
if result=$(nix-instantiate --eval --strict --json 2>"$work/stderr" \
--expr "with (import <nixpkgs/lib>).filesystem; $expr"); then
die "$expr evaluated successfully to $result, but it was expected to fail"
fi
if [[ ! "$(<"$work/stderr")" =~ $expectedErrorRegex ]]; then
die "Error was $(<"$work/stderr"), but $expectedErrorRegex was expected"
fi
}
expectSuccess "pathType /." '"directory"'
expectSuccess "pathType $PWD/directory" '"directory"'
expectSuccess "pathType $PWD/regular" '"regular"'
expectSuccess "pathType $PWD/symlink" '"symlink"'
expectSuccess "pathType $PWD/fifo" '"unknown"'
# Only check error message when a Nixpkgs-specified error is thrown,
# which is only the case when `readFileType` is not available
# and the fallback implementation needs to be used.
if [[ "$(nix-instantiate --eval --expr 'builtins ? readFileType')" == false ]]; then
expectFailure "pathType $PWD/non-existent" \
"error: evaluation aborted with the following error message: 'lib.filesystem.pathType: Path $PWD/non-existent does not exist.'"
fi
expectSuccess "pathIsDirectory /." "true"
expectSuccess "pathIsDirectory $PWD/directory" "true"
expectSuccess "pathIsDirectory $PWD/regular" "false"
expectSuccess "pathIsDirectory $PWD/symlink" "false"
expectSuccess "pathIsDirectory $PWD/fifo" "false"
expectSuccess "pathIsDirectory $PWD/non-existent" "false"
expectSuccess "pathIsRegularFile /." "false"
expectSuccess "pathIsRegularFile $PWD/directory" "false"
expectSuccess "pathIsRegularFile $PWD/regular" "true"
expectSuccess "pathIsRegularFile $PWD/symlink" "false"
expectSuccess "pathIsRegularFile $PWD/fifo" "false"
expectSuccess "pathIsRegularFile $PWD/non-existent" "false"
echo >&2 tests ok

33
lib/tests/maintainer-module.nix Normal file
View File

@@ -0,0 +1,33 @@
{ lib, ... }:
let
inherit (lib) types;
in
{
options = {
name = lib.mkOption {
type = types.str;
};
github = lib.mkOption {
type = types.str;
};
githubId = lib.mkOption {
type = types.ints.unsigned;
};
email = lib.mkOption {
type = types.nullOr types.str;
default = null;
};
matrix = lib.mkOption {
type = types.nullOr types.str;
default = null;
};
keys = lib.mkOption {
type = types.listOf (
types.submodule {
options.fingerprint = lib.mkOption { type = types.str; };
}
);
default = [ ];
};
};
}

76
lib/tests/maintainers.nix Normal file
View File

@@ -0,0 +1,76 @@
# to run these tests (and the others)
# nix-build nixpkgs/lib/tests/release.nix
# These tests should stay in sync with the comment in maintainers/maintainer-list.nix
{
# The pkgs used for dependencies for the testing itself
pkgs ? import ../.. { },
lib ? pkgs.lib,
}:
let
checkMaintainer =
handle: uncheckedAttrs:
let
prefix = [
"lib"
"maintainers"
handle
];
checkedAttrs =
(lib.modules.evalModules {
inherit prefix;
modules = [
./maintainer-module.nix
{
_file = toString ../../maintainers/maintainer-list.nix;
config = uncheckedAttrs;
}
];
}).config;
checks =
lib.optional (checkedAttrs.github != null && checkedAttrs.githubId == null) ''
echo ${lib.escapeShellArg (lib.showOption prefix)}': If `github` is specified, `githubId` must be too.'
# Calling this too often would hit non-authenticated API limits, but this
# shouldn't happen since such errors will get fixed rather quickly
info=$(curl -sS https://api.github.com/users/${checkedAttrs.github})
id=$(jq -r '.id' <<< "$info")
echo "The GitHub ID for GitHub user ${checkedAttrs.github} is $id:"
echo -e " githubId = $id;\n"
''
++
lib.optional
(checkedAttrs.email == null && checkedAttrs.github == null && checkedAttrs.matrix == null)
''
echo ${lib.escapeShellArg (lib.showOption prefix)}': At least one of `email`, `github` or `matrix` must be specified, so that users know how to reach you.'
''
++
lib.optional (checkedAttrs.email != null && lib.hasSuffix "noreply.github.com" checkedAttrs.email)
''
echo ${lib.escapeShellArg (lib.showOption prefix)}': If an email address is given, it should allow people to reach you. If you do not want that, you can just provide `github` or `matrix` instead.'
'';
in
lib.deepSeq checkedAttrs checks;
missingGithubIds = lib.concatLists (lib.mapAttrsToList checkMaintainer lib.maintainers);
success = pkgs.runCommand "checked-maintainers-success" { } "mkdir $out";
failure =
pkgs.runCommand "checked-maintainers-failure"
{
nativeBuildInputs = [
pkgs.curl
pkgs.jq
];
outputHash = "sha256:${lib.fakeSha256}";
outputHAlgo = "sha256";
outputHashMode = "flat";
SSL_CERT_FILE = "${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt";
}
''
${lib.concatStringsSep "\n" missingGithubIds}
exit 1
'';
in
if missingGithubIds == [ ] then success else failure

4664
lib/tests/misc.nix Normal file
View File

File diff suppressed because it is too large Load Diff

819
lib/tests/modules.sh Executable file
View File

@@ -0,0 +1,819 @@
#!/usr/bin/env bash
# This script is used to test that the module system is working as expected.
# Executing it runs tests for `lib.modules`, `lib.options` and `lib.types`.
# By default it test the version of nixpkgs which is defined in the NIX_PATH.
#
# Run:
# [nixpkgs]$ lib/tests/modules.sh
# or:
# [nixpkgs]$ nix-build lib/tests/release.nix
set -o errexit -o noclobber -o nounset -o pipefail
shopt -s failglob inherit_errexit
# https://stackoverflow.com/a/246128/6605742
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
cd "$DIR"/modules
pass=0
fail=0
local-nix-instantiate() {
nix-instantiate --timeout 1 --eval-only --show-trace --read-write-mode --json "$@"
}
# loc
# prints the location of the call of to the function that calls it
# loc n
# prints the location n levels up the call stack
loc() {
local caller depth
depth=1
if [[ $# -gt 0 ]]; then
depth=$1
fi
# ( lineno fnname file ) of the caller
caller=( $(caller $depth) )
echo "${caller[2]}:${caller[0]}"
}
line() {
echo "----------------------------------------"
}
logStartFailure() {
line
}
logEndFailure() {
line
echo
}
logFailure() {
# bold red
printf '\033[1;31mTEST FAILED\033[0m at %s\n' "$(loc 2)"
}
evalConfig() {
local attr=$1
shift
local script="import ./default.nix { modules = [ $* ];}"
local-nix-instantiate -E "$script" -A "$attr"
}
reportFailure() {
local attr=$1
shift
local script="import ./default.nix { modules = [ $* ];}"
echo "$ nix-instantiate -E '$script' -A '$attr' --eval-only --json"
evalConfig "$attr" "$@" || true
((++fail))
}
checkConfigOutput() {
local outputContains=$1
shift
if evalConfig "$@" 2>/dev/null | grep -E --silent "$outputContains" ; then
((++pass))
else
logStartFailure
echo "ACTUAL:"
reportFailure "$@"
echo "EXPECTED: result matching '$outputContains'"
logFailure
logEndFailure
fi
}
invertIfUnset() {
gate="$1"
shift
if [[ -n "${!gate:-}" ]]; then
"$@"
else
! "$@"
fi
}
globalErrorLogCheck() {
invertIfUnset "REQUIRE_INFINITE_RECURSION_HINT" \
grep -i 'if you get an infinite recursion here' \
<<<"$err" >/dev/null \
|| {
if [[ -n "${REQUIRE_INFINITE_RECURSION_HINT:-}" ]]; then
echo "Unexpected infinite recursion hint"
else
echo "Expected infinite recursion hint, but none found"
fi
return 1
}
}
checkExpression() {
local path=$1
local output
{
output="$(local-nix-instantiate --strict "$path" 2>&1)" && ((++pass))
} || {
logStartFailure
echo "$output"
((++fail))
logFailure
logEndFailure
}
}
checkConfigError() {
local errorContains=$1
local err=""
shift
if err="$(evalConfig "$@" 2>&1 >/dev/null)"; then
logStartFailure
echo "ACTUAL: exit code 0, output:"
reportFailure "$@"
echo "EXPECTED: non-zero exit code"
logFailure
logEndFailure
else
if ! globalErrorLogCheck "$err"; then
logStartFailure
echo "LOG:"
reportFailure "$@"
echo "GLOBAL ERROR LOG CHECK FAILED"
logFailure
logEndFailure
fi
if echo "$err" | grep -zP --silent "$errorContains" ; then
((++pass))
else
logStartFailure
echo "ACTUAL:"
reportFailure "$@"
echo "EXPECTED: error matching '$errorContains'"
logFailure
logEndFailure
fi
fi
}
# Shorthand meta attribute does not duplicate the config
checkConfigOutput '^"one two"$' config.result ./shorthand-meta.nix
checkConfigOutput '^true$' config.result ./test-mergeAttrDefinitionsWithPrio.nix
# Check that a module argument is passed, also when a default is available
# (but not needed)
#
# When the default is needed, we currently fail to do what the users expect, as
# we pass our own argument anyway, even if it *turns out* not to exist.
#
# The reason for this is that we don't know at invocation time what is in the
# _module.args option. That value is only available *after* all modules have been
# invoked.
#
# Hypothetically, Nix could help support this by giving access to the default
# values, through a new built-in function.
# However the default values are allowed to depend on other arguments, so those
# would have to be passed in somehow, making this not just a getter but
# something more complicated.
#
# At that point we have to wonder whether the extra complexity is worth the cost.
# Another - subjective - reason not to support it is that default values
# contradict the notion that an option has a single value, where _module.args
# is the option.
checkConfigOutput '^true$' config.result ./module-argument-default.nix
# gvariant
checkConfigOutput '^true$' config.assertion ./gvariant.nix
checkConfigOutput '"ok"' config.result ./specialArgs-lib.nix
# https://github.com/NixOS/nixpkgs/pull/131205
# We currently throw this error already in `config`, but throwing in `config.wrong1` would be acceptable.
checkConfigError 'It seems as if you.re trying to declare an option by placing it into .config. rather than .options.' config.wrong1 ./error-mkOption-in-config.nix
# We currently throw this error already in `config`, but throwing in `config.nest.wrong2` would be acceptable.
checkConfigError 'It seems as if you.re trying to declare an option by placing it into .config. rather than .options.' config.nest.wrong2 ./error-mkOption-in-config.nix
checkConfigError 'The option .sub.wrong2. does not exist. Definition values:' config.sub ./error-mkOption-in-submodule-config.nix
checkConfigError '.*This can happen if you e.g. declared your options in .types.submodule.' config.sub ./error-mkOption-in-submodule-config.nix
checkConfigError '.*A definition for option .bad. is not of type .non-empty .list of .submodule...\.' config.bad ./error-nonEmptyListOf-submodule.nix
# types.attrTag
checkConfigOutput '^true$' config.okChecks ./types-attrTag.nix
checkConfigError 'A definition for option .intStrings\.syntaxError. is not of type .attribute-tagged union' config.intStrings.syntaxError ./types-attrTag.nix
checkConfigError 'A definition for option .intStrings\.syntaxError2. is not of type .attribute-tagged union' config.intStrings.syntaxError2 ./types-attrTag.nix
checkConfigError 'A definition for option .intStrings\.syntaxError3. is not of type .attribute-tagged union' config.intStrings.syntaxError3 ./types-attrTag.nix
checkConfigError 'A definition for option .intStrings\.syntaxError4. is not of type .attribute-tagged union' config.intStrings.syntaxError4 ./types-attrTag.nix
checkConfigError 'A definition for option .intStrings\.mergeError. is not of type .attribute-tagged union' config.intStrings.mergeError ./types-attrTag.nix
checkConfigError 'A definition for option .intStrings\.badTagError. is not of type .attribute-tagged union' config.intStrings.badTagError ./types-attrTag.nix
checkConfigError 'A definition for option .intStrings\.badTagTypeError\.left. is not of type .signed integer.' config.intStrings.badTagTypeError.left ./types-attrTag.nix
checkConfigError 'A definition for option .nested\.right\.left. is not of type .signed integer.' config.nested.right.left ./types-attrTag.nix
checkConfigError 'In attrTag, each tag value must be an option, but tag int was a bare type, not wrapped in mkOption.' config.opt.int ./types-attrTag-wrong-decl.nix
# types
checkConfigOutput '"ok"' config.assertions ./types.nix
# types.pathInStore
checkConfigOutput '".*/store/0lz9p8xhf89kb1c1kk6jxrzskaiygnlh-bash-5.2-p15.drv"' config.pathInStore.ok1 ./types.nix
checkConfigOutput '".*/store/0fb3ykw9r5hpayd05sr0cizwadzq1d8q-bash-5.2-p15"' config.pathInStore.ok2 ./types.nix
checkConfigOutput '".*/store/0fb3ykw9r5hpayd05sr0cizwadzq1d8q-bash-5.2-p15/bin/bash"' config.pathInStore.ok3 ./types.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ""' config.pathInStore.bad1 ./types.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ".*/store"' config.pathInStore.bad2 ./types.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ".*/store/"' config.pathInStore.bad3 ./types.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ".*/store/.links"' config.pathInStore.bad4 ./types.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: "/foo/bar"' config.pathInStore.bad5 ./types.nix
# Check boolean option.
checkConfigOutput '^false$' config.enable ./declare-enable.nix
checkConfigError 'The option .* does not exist. Definition values:\n\s*- In .*: true' config.enable ./define-enable.nix
checkConfigError 'The option .* does not exist. Definition values:\n\s*- In .*' config.enable ./define-enable-throw.nix
checkConfigError 'while evaluating a definition from `.*/define-enable-abort.nix' config.enable ./define-enable-abort.nix
checkConfigError 'while evaluating the error message for definitions for .enable., which is an option that does not exist' config.enable ./define-enable-abort.nix
# Check boolByOr type.
checkConfigOutput '^false$' config.value.falseFalse ./boolByOr.nix
checkConfigOutput '^true$' config.value.trueFalse ./boolByOr.nix
checkConfigOutput '^true$' config.value.falseTrue ./boolByOr.nix
checkConfigOutput '^true$' config.value.trueTrue ./boolByOr.nix
checkConfigOutput '^1$' config.bare-submodule.nested ./declare-bare-submodule.nix ./declare-bare-submodule-nested-option.nix
checkConfigOutput '^2$' config.bare-submodule.deep ./declare-bare-submodule.nix ./declare-bare-submodule-deep-option.nix
checkConfigOutput '^42$' config.bare-submodule.nested ./declare-bare-submodule.nix ./declare-bare-submodule-nested-option.nix ./declare-bare-submodule-deep-option.nix ./define-bare-submodule-values.nix
checkConfigOutput '^420$' config.bare-submodule.deep ./declare-bare-submodule.nix ./declare-bare-submodule-nested-option.nix ./declare-bare-submodule-deep-option.nix ./define-bare-submodule-values.nix
checkConfigOutput '^2$' config.bare-submodule.deep ./declare-bare-submodule.nix ./declare-bare-submodule-deep-option.nix ./define-shorthandOnlyDefinesConfig-true.nix
checkConfigError 'The option .bare-submodule.deep. in .*/declare-bare-submodule-deep-option.nix. is already declared in .*/declare-bare-submodule-deep-option-duplicate.nix' config.bare-submodule.deep ./declare-bare-submodule.nix ./declare-bare-submodule-deep-option.nix ./declare-bare-submodule-deep-option-duplicate.nix
# Check that strMatching can be merged
checkConfigOutput '^"strMatching.*"$' options.sm.type.name ./strMatching-merge.nix
# Check integer types.
# unsigned
checkConfigOutput '^42$' config.value ./declare-int-unsigned-value.nix ./define-value-int-positive.nix
checkConfigError 'A definition for option .* is not of type.*unsigned integer.*. Definition values:\n\s*- In .*: -23' config.value ./declare-int-unsigned-value.nix ./define-value-int-negative.nix
# positive
checkConfigError 'A definition for option .* is not of type.*positive integer.*. Definition values:\n\s*- In .*: 0' config.value ./declare-int-positive-value.nix ./define-value-int-zero.nix
# between
checkConfigOutput '^42$' config.value ./declare-int-between-value.nix ./define-value-int-positive.nix
checkConfigError 'A definition for option .* is not of type.*between.*-21 and 43.*inclusive.*. Definition values:\n\s*- In .*: -23' config.value ./declare-int-between-value.nix ./define-value-int-negative.nix
# Check either types
# types.either
checkConfigOutput '^42$' config.value ./declare-either.nix ./define-value-int-positive.nix
checkConfigOutput '^"24"$' config.value ./declare-either.nix ./define-value-string.nix
# types.oneOf
checkConfigOutput '^42$' config.value ./declare-oneOf.nix ./define-value-int-positive.nix
checkConfigOutput '^\[\]$' config.value ./declare-oneOf.nix ./define-value-list.nix
checkConfigOutput '^"24"$' config.value ./declare-oneOf.nix ./define-value-string.nix
# Check mkForce without submodules.
set -- config.enable ./declare-enable.nix ./define-enable.nix
checkConfigOutput '^true$' "$@"
checkConfigOutput '^false$' "$@" ./define-force-enable.nix
checkConfigOutput '^false$' "$@" ./define-enable-force.nix
# Check mkForce with option and submodules.
checkConfigError 'attribute .*foo.* .* not found' config.attrsOfSub.foo.enable ./declare-attrsOfSub-any-enable.nix
checkConfigOutput '^false$' config.attrsOfSub.foo.enable ./declare-attrsOfSub-any-enable.nix ./define-attrsOfSub-foo.nix
set -- config.attrsOfSub.foo.enable ./declare-attrsOfSub-any-enable.nix ./define-attrsOfSub-foo-enable.nix
checkConfigOutput '^true$' "$@"
checkConfigOutput '^false$' "$@" ./define-force-attrsOfSub-foo-enable.nix
checkConfigOutput '^false$' "$@" ./define-attrsOfSub-force-foo-enable.nix
checkConfigOutput '^false$' "$@" ./define-attrsOfSub-foo-force-enable.nix
checkConfigOutput '^false$' "$@" ./define-attrsOfSub-foo-enable-force.nix
# Check overriding effect of mkForce on submodule definitions.
checkConfigError 'attribute .*bar.* .* not found' config.attrsOfSub.bar.enable ./declare-attrsOfSub-any-enable.nix ./define-attrsOfSub-foo.nix
checkConfigOutput '^false$' config.attrsOfSub.bar.enable ./declare-attrsOfSub-any-enable.nix ./define-attrsOfSub-foo.nix ./define-attrsOfSub-bar.nix
set -- config.attrsOfSub.bar.enable ./declare-attrsOfSub-any-enable.nix ./define-attrsOfSub-foo.nix ./define-attrsOfSub-bar-enable.nix
checkConfigOutput '^true$' "$@"
checkConfigError 'attribute .*bar.* .* not found' "$@" ./define-force-attrsOfSub-foo-enable.nix
checkConfigError 'attribute .*bar.* .* not found' "$@" ./define-attrsOfSub-force-foo-enable.nix
checkConfigOutput '^true$' "$@" ./define-attrsOfSub-foo-force-enable.nix
checkConfigOutput '^true$' "$@" ./define-attrsOfSub-foo-enable-force.nix
# Check mkIf with submodules.
checkConfigError 'attribute .*foo.* .* not found' config.attrsOfSub.foo.enable ./declare-enable.nix ./declare-attrsOfSub-any-enable.nix
set -- config.attrsOfSub.foo.enable ./declare-enable.nix ./declare-attrsOfSub-any-enable.nix
checkConfigError 'attribute .*foo.* .* not found' "$@" ./define-if-attrsOfSub-foo-enable.nix
checkConfigError 'attribute .*foo.* .* not found' "$@" ./define-attrsOfSub-if-foo-enable.nix
checkConfigError 'attribute .*foo.* .* not found' "$@" ./define-attrsOfSub-foo-if-enable.nix
checkConfigOutput '^false$' "$@" ./define-attrsOfSub-foo-enable-if.nix
checkConfigOutput '^true$' "$@" ./define-enable.nix ./define-if-attrsOfSub-foo-enable.nix
checkConfigOutput '^true$' "$@" ./define-enable.nix ./define-attrsOfSub-if-foo-enable.nix
checkConfigOutput '^true$' "$@" ./define-enable.nix ./define-attrsOfSub-foo-if-enable.nix
checkConfigOutput '^true$' "$@" ./define-enable.nix ./define-attrsOfSub-foo-enable-if.nix
# Check importApply
checkConfigOutput '"abc"' config.value ./importApply.nix
# importApply does not set a key.
# Disabling the function file is not sufficient, because importApply can't reasonably assume that the key is unique.
# e.g. user may call it multiple times with different arguments and expect each of the module to apply.
# While this is excusable for the disabledModules aspect, it is not for the deduplication of modules.
checkConfigOutput '"abc"' config.value ./importApply-disabling.nix
# Check disabledModules with config definitions and option declarations.
set -- config.enable ./define-enable.nix ./declare-enable.nix
checkConfigOutput '^true$' "$@"
checkConfigOutput '^false$' "$@" ./disable-define-enable.nix
checkConfigOutput '^false$' "$@" ./disable-define-enable-string-path.nix
checkConfigError "The option .*enable.* does not exist. Definition values:\n\s*- In .*: true" "$@" ./disable-declare-enable.nix
checkConfigError "attribute .*enable.* in selection path .*config.enable.* not found" "$@" ./disable-define-enable.nix ./disable-declare-enable.nix
checkConfigError "attribute .*enable.* in selection path .*config.enable.* not found" "$@" ./disable-enable-modules.nix
checkConfigOutput '^true$' 'config.positive.enable' ./disable-module-with-key.nix
checkConfigOutput '^false$' 'config.negative.enable' ./disable-module-with-key.nix
checkConfigError 'Module ..*disable-module-bad-key.nix. contains a disabledModules item that is an attribute set, presumably a module, that does not have a .key. attribute. .*' 'config.enable' ./disable-module-bad-key.nix
# Not sure if we want to keep supporting module keys that aren't strings, paths or v?key, but we shouldn't remove support accidentally.
checkConfigOutput '^true$' 'config.positive.enable' ./disable-module-with-toString-key.nix
checkConfigOutput '^false$' 'config.negative.enable' ./disable-module-with-toString-key.nix
# Check _module.args.
set -- config.enable ./declare-enable.nix ./define-enable-with-custom-arg.nix
checkConfigError 'while evaluating the module argument .*custom.* in .*define-enable-with-custom-arg.nix.*:' "$@"
checkConfigOutput '^true$' "$@" ./define-_module-args-custom.nix
# Check that using _module.args on imports cause infinite recursions, with
# the proper error context.
set -- "$@" ./define-_module-args-custom.nix ./import-custom-arg.nix
REQUIRE_INFINITE_RECURSION_HINT=1 checkConfigError 'while evaluating the module argument .*custom.* in .*import-custom-arg.nix.*:' "$@"
REQUIRE_INFINITE_RECURSION_HINT=1 checkConfigError 'infinite recursion encountered' "$@"
# Check _module.check.
set -- config.enable ./declare-enable.nix ./define-enable.nix ./define-attrsOfSub-foo.nix
checkConfigError 'The option .* does not exist. Definition values:\n\s*- In .*' "$@"
checkConfigOutput '^true$' "$@" ./define-module-check.nix
# Check coerced value.
set --
checkConfigOutput '^"42"$' config.value ./declare-coerced-value.nix
checkConfigOutput '^"24"$' config.value ./declare-coerced-value.nix ./define-value-string.nix
checkConfigError 'A definition for option .*. is not of type .*.\n\s*- In .*: \[ \]' config.value ./declare-coerced-value.nix ./define-value-list.nix
# Check coerced option merging.
checkConfigError 'The option .value. in .*/declare-coerced-value.nix. is already declared in .*/declare-coerced-value-no-default.nix.' config.value ./declare-coerced-value.nix ./declare-coerced-value-no-default.nix
# Check coerced value with unsound coercion
checkConfigOutput '^12$' config.value ./declare-coerced-value-unsound.nix
checkConfigError 'A definition for option .* is not of type .*.\n\s*- In .*: "1000"' config.value ./declare-coerced-value-unsound.nix ./define-value-string-bigint.nix
checkConfigError 'toInt: Could not convert .* to int' config.value ./declare-coerced-value-unsound.nix ./define-value-string-arbitrary.nix
# Check `graph` attribute
checkExpression './graph/test.nix'
# Check mkAliasOptionModule.
checkConfigOutput '^true$' config.enable ./alias-with-priority.nix
checkConfigOutput '^true$' config.enableAlias ./alias-with-priority.nix
checkConfigOutput '^false$' config.enable ./alias-with-priority-can-override.nix
checkConfigOutput '^false$' config.enableAlias ./alias-with-priority-can-override.nix
# Check mkPackageOption
checkConfigOutput '^"hello"$' config.package.pname ./declare-mkPackageOption.nix
checkConfigOutput '^"hello"$' config.namedPackage.pname ./declare-mkPackageOption.nix
checkConfigOutput '^".*Hello.*"$' options.namedPackage.description ./declare-mkPackageOption.nix
checkConfigOutput '^"literalExpression"$' options.namedPackage.defaultText._type ./declare-mkPackageOption.nix
checkConfigOutput '^"pkgs\.hello"$' options.namedPackage.defaultText.text ./declare-mkPackageOption.nix
checkConfigOutput '^"hello"$' config.namedPackageSingletonDefault.pname ./declare-mkPackageOption.nix
checkConfigOutput '^".*Hello.*"$' options.namedPackageSingletonDefault.description ./declare-mkPackageOption.nix
checkConfigOutput '^"pkgs\.hello"$' options.namedPackageSingletonDefault.defaultText.text ./declare-mkPackageOption.nix
checkConfigOutput '^"hello"$' config.pathPackage.pname ./declare-mkPackageOption.nix
checkConfigOutput '^"literalExpression"$' options.packageWithExample.example._type ./declare-mkPackageOption.nix
checkConfigOutput '^"pkgs\.hello\.override \{ stdenv = pkgs\.clangStdenv; \}"$' options.packageWithExample.example.text ./declare-mkPackageOption.nix
checkConfigOutput '^"literalExpression"$' options.packageWithPathExample.example._type ./declare-mkPackageOption.nix
checkConfigOutput '^"pkgs\.hello"$' options.packageWithPathExample.example.text ./declare-mkPackageOption.nix
checkConfigOutput '^".*Example extra description\..*"$' options.packageWithExtraDescription.description ./declare-mkPackageOption.nix
checkConfigError 'The option .undefinedPackage. was accessed but has no value defined. Try setting the option.' config.undefinedPackage ./declare-mkPackageOption.nix
checkConfigOutput '^null$' config.nullablePackage ./declare-mkPackageOption.nix
checkConfigOutput '^"null or package"$' options.nullablePackage.type.description ./declare-mkPackageOption.nix
checkConfigOutput '^"hello"$' config.nullablePackageWithDefault.pname ./declare-mkPackageOption.nix
checkConfigOutput '^"myPkgs\.hello"$' options.packageWithPkgsText.defaultText.text ./declare-mkPackageOption.nix
checkConfigOutput '^"hello-other"$' options.packageFromOtherSet.default.pname ./declare-mkPackageOption.nix
checkConfigOutput '^"hello"$' config.packageInvalidIdentifier.pname ./declare-mkPackageOption.nix
checkConfigOutput '^"pkgs\.\\"123\\"\.\\"with\\\\\\"quote\\"\.hello"$' options.packageInvalidIdentifier.defaultText.text ./declare-mkPackageOption.nix
checkConfigOutput '^"pkgs\.\\"123\\"\.\\"with\\\\\\"quote\\"\.hello"$' options.packageInvalidIdentifierExample.example.text ./declare-mkPackageOption.nix
# submoduleWith
## specialArgs should work
checkConfigOutput '^"foo"$' config.submodule.foo ./declare-submoduleWith-special.nix
## shorthandOnlyDefines config behaves as expected
checkConfigOutput '^true$' config.submodule.config ./declare-submoduleWith-shorthand.nix ./define-submoduleWith-shorthand.nix
checkConfigError 'is not of type `boolean' config.submodule.config ./declare-submoduleWith-shorthand.nix ./define-submoduleWith-noshorthand.nix
checkConfigError "In module ..*define-submoduleWith-shorthand.nix., you're trying to define a value of type \`bool'\n\s*rather than an attribute set for the option" config.submodule.config ./declare-submoduleWith-noshorthand.nix ./define-submoduleWith-shorthand.nix
checkConfigOutput '^true$' config.submodule.config ./declare-submoduleWith-noshorthand.nix ./define-submoduleWith-noshorthand.nix
## submoduleWith should merge all modules in one swoop
checkConfigOutput '^true$' config.submodule.inner ./declare-submoduleWith-modules.nix
checkConfigOutput '^true$' config.submodule.outer ./declare-submoduleWith-modules.nix
# Should also be able to evaluate the type name (which evaluates freeformType,
# which evaluates all the modules defined by the type)
checkConfigOutput '^"submodule"$' options.submodule.type.description ./declare-submoduleWith-modules.nix
## submodules can be declared using (evalModules {...}).type
checkConfigOutput '^true$' config.submodule.inner ./declare-submodule-via-evalModules.nix
checkConfigOutput '^true$' config.submodule.outer ./declare-submodule-via-evalModules.nix
# Should also be able to evaluate the type name (which evaluates freeformType,
# which evaluates all the modules defined by the type)
checkConfigOutput '^"submodule"$' options.submodule.type.description ./declare-submodule-via-evalModules.nix
## Paths should be allowed as values and work as expected
checkConfigOutput '^true$' config.submodule.enable ./declare-submoduleWith-path.nix
## _prefix module argument is available at import time and contains the prefix
checkConfigOutput '^true$' config.foo.ok ./prefix-module-argument.nix
## deferredModule
# default module is merged into nodes.foo
checkConfigOutput '"beta"' config.nodes.foo.settingsDict.c ./deferred-module.nix
# errors from the default module are reported with accurate location
checkConfigError 'In `the-file-that-contains-the-bad-config.nix, via option default'\'': "bogus"' config.nodes.foo.bottom ./deferred-module.nix
checkConfigError '.*lib/tests/modules/deferred-module-error.nix, via option deferred [(]:anon-1:anon-1:anon-1[)] does not look like a module.' config.result ./deferred-module-error.nix
# Check the file location information is propagated into submodules
checkConfigOutput the-file.nix config.submodule.internalFiles.0 ./submoduleFiles.nix
# Check that disabledModules works recursively and correctly
checkConfigOutput '^true$' config.enable ./disable-recursive/main.nix
checkConfigOutput '^true$' config.enable ./disable-recursive/{main.nix,disable-foo.nix}
checkConfigOutput '^true$' config.enable ./disable-recursive/{main.nix,disable-bar.nix}
checkConfigError 'The option .* does not exist. Definition values:\n\s*- In .*: true' config.enable ./disable-recursive/{main.nix,disable-foo.nix,disable-bar.nix}
# Check that imports can depend on derivations
checkConfigOutput '^true$' config.enable ./import-from-store.nix
# Check that configs can be conditional on option existence
checkConfigOutput '^true$' config.enable ./define-option-dependently.nix ./declare-enable.nix ./declare-int-positive-value.nix
checkConfigOutput '^360$' config.value ./define-option-dependently.nix ./declare-enable.nix ./declare-int-positive-value.nix
checkConfigOutput '^7$' config.value ./define-option-dependently.nix ./declare-int-positive-value.nix
checkConfigOutput '^true$' config.set.enable ./define-option-dependently-nested.nix ./declare-enable-nested.nix ./declare-int-positive-value-nested.nix
checkConfigOutput '^360$' config.set.value ./define-option-dependently-nested.nix ./declare-enable-nested.nix ./declare-int-positive-value-nested.nix
checkConfigOutput '^7$' config.set.value ./define-option-dependently-nested.nix ./declare-int-positive-value-nested.nix
# Check attrsOf and lazyAttrsOf. Only lazyAttrsOf should be lazy, and only
# attrsOf should work with conditional definitions
# In addition, lazyAttrsOf should honor an options emptyValue
checkConfigError "is not lazy" config.isLazy ./declare-attrsOf.nix ./attrsOf-lazy-check.nix
checkConfigOutput '^true$' config.isLazy ./declare-lazyAttrsOf.nix ./attrsOf-lazy-check.nix
checkConfigOutput '^true$' config.conditionalWorks ./declare-attrsOf.nix ./attrsOf-conditional-check.nix
checkConfigOutput '^false$' config.conditionalWorks ./declare-lazyAttrsOf.nix ./attrsOf-conditional-check.nix
checkConfigOutput '^"empty"$' config.value.foo ./declare-lazyAttrsOf.nix ./attrsOf-conditional-check.nix
# Check attrsWith type merging
checkConfigError 'The option `mergedLazyNonLazy'\'' in `.*'\'' is already declared in `.*'\''\.' options.mergedLazyNonLazy ./lazy-attrsWith.nix
checkConfigOutput '^11$' config.lazyResult ./lazy-attrsWith.nix
checkConfigError 'infinite recursion encountered' config.nonLazyResult ./lazy-attrsWith.nix
# AttrsWith placeholder tests
checkConfigOutput '^"mergedName.<id>.nested"$' config.result ./name-merge-attrsWith-1.nix
checkConfigError 'The option .mergedName. in .*\.nix. is already declared in .*\.nix' config.mergedName ./name-merge-attrsWith-2.nix
# Test type.functor.wrapped deprecation warning
# should emit the warning on:
# - merged types
# - non-merged types
# - nestedTypes elemType
# attrsWith
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.attrsWith.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedAttrsWith.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.attrsWith.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedAttrsWith.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
# listOf
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.listOf.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedListOf.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.listOf.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedListOf.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
# unique / uniq
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.unique.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedUnique.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.unique.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedUnique.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
# nullOr
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.nullOr.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedNullOr.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.nullOr.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedNullOr.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
# functionTo
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.functionTo.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedFunctionTo.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.functionTo.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedFunctionTo.type.nestedTypes.elemType.functor.wrapped ./deprecated-wrapped.nix
# coercedTo
# Note: test 'nestedTypes.finalType' and 'nestedTypes.coercedType'
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.coercedTo.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.coercedTo.type.nestedTypes.finalType.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.coercedTo.type.nestedTypes.coercedType.functor.wrapped ./deprecated-wrapped.nix
# either
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.either.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedEither.type.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.either.type.nestedTypes.left.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.either.type.nestedTypes.right.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedEither.type.nestedTypes.left.functor.wrapped ./deprecated-wrapped.nix
NIX_ABORT_ON_WARN=1 checkConfigError 'The deprecated `.*functor.wrapped` attribute .*is accessed, use `.*nestedTypes.elemType` instead.' options.mergedEither.type.nestedTypes.right.functor.wrapped ./deprecated-wrapped.nix
# Even with multiple assignments, a type error should be thrown if any of them aren't valid
checkConfigError 'A definition for option .* is not of type .*' \
config.value ./declare-int-unsigned-value.nix ./define-value-list.nix ./define-value-int-positive.nix
## Freeform modules
# Assigning without a declared option should work
checkConfigOutput '^"24"$' config.value ./freeform-attrsOf.nix ./define-value-string.nix
# Shorthand modules interpret `meta` and `class` as config items
checkConfigOutput '^true$' options._module.args.value.result ./freeform-attrsOf.nix ./define-freeform-keywords-shorthand.nix
# No freeform assignments shouldn't make it error
checkConfigOutput '^{}$' config ./freeform-attrsOf.nix
# but only if the type matches
checkConfigError 'A definition for option .* is not of type .*' config.value ./freeform-attrsOf.nix ./define-value-list.nix
# and properties should be applied
checkConfigOutput '^"yes"$' config.value ./freeform-attrsOf.nix ./define-value-string-properties.nix
# Options should still be declarable, and be able to have a type that doesn't match the freeform type
checkConfigOutput '^false$' config.enable ./freeform-attrsOf.nix ./define-value-string.nix ./declare-enable.nix
checkConfigOutput '^"24"$' config.value ./freeform-attrsOf.nix ./define-value-string.nix ./declare-enable.nix
# and this should work too with nested values
checkConfigOutput '^false$' config.nest.foo ./freeform-attrsOf.nix ./freeform-nested.nix
checkConfigOutput '^"bar"$' config.nest.bar ./freeform-attrsOf.nix ./freeform-nested.nix
# Check whether a declared option can depend on an freeform-typed one
checkConfigOutput '^null$' config.foo ./freeform-attrsOf.nix ./freeform-str-dep-unstr.nix
checkConfigOutput '^"24"$' config.foo ./freeform-attrsOf.nix ./freeform-str-dep-unstr.nix ./define-value-string.nix
# Check whether an freeform-typed value can depend on a declared option, this can only work with lazyAttrsOf
REQUIRE_INFINITE_RECURSION_HINT=1 checkConfigError 'infinite recursion encountered' config.foo ./freeform-attrsOf.nix ./freeform-unstr-dep-str.nix
checkConfigError 'The option .* was accessed but has no value defined. Try setting the option.' config.foo ./freeform-lazyAttrsOf.nix ./freeform-unstr-dep-str.nix
checkConfigOutput '^"24"$' config.foo ./freeform-lazyAttrsOf.nix ./freeform-unstr-dep-str.nix ./define-value-string.nix
# submodules in freeformTypes should have their locations annotated
checkConfigOutput '/freeform-submodules.nix"$' config.fooDeclarations.0 ./freeform-submodules.nix
# freeformTypes can get merged using `types.type`, including submodules
checkConfigOutput '^10$' config.free.xxx.foo ./freeform-submodules.nix
checkConfigOutput '^10$' config.free.yyy.bar ./freeform-submodules.nix
# Regression of either, due to freeform not beeing checked previously
checkConfigOutput '^"foo"$' config.either.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.either.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
checkConfigOutput '^"foo"$' config.eitherBehindNullor.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.eitherBehindNullor.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
checkConfigOutput '^"foo"$' config.oneOf.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.oneOf.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
checkConfigOutput '^"foo"$' config.number.str ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.number.str ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong.nix
checkConfigOutput '^42$' config.either.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.either.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
checkConfigOutput '^42$' config.eitherBehindNullor.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.eitherBehindNullor.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
checkConfigOutput '^42$' config.oneOf.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.oneOf.int ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
checkConfigOutput '^42$' config.number.str ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
NIX_ABORT_ON_WARN=1 checkConfigError "One or more definitions did not pass the type-check of the \'either\' type" config.number.str ./freeform-deprecated-malicous.nix ./freeform-deprecated-malicous-wrong2.nix
# Value OK: Fail if a warning is emitted
NIX_ABORT_ON_WARN=1 checkConfigOutput "^42$" config.number.int ./freeform-attrsof-either.nix
## types.anything
# Check that attribute sets are merged recursively
checkConfigOutput '^null$' config.value.foo ./types-anything/nested-attrs.nix
checkConfigOutput '^null$' config.value.l1.foo ./types-anything/nested-attrs.nix
checkConfigOutput '^null$' config.value.l1.l2.foo ./types-anything/nested-attrs.nix
checkConfigOutput '^null$' config.value.l1.l2.l3.foo ./types-anything/nested-attrs.nix
# Attribute sets that are coercible to strings shouldn't be recursed into
checkConfigOutput '^"foo"$' config.value.outPath ./types-anything/attrs-coercible.nix
# Multiple lists aren't concatenated together if their definitions are not equal
checkConfigError 'The option .* has conflicting definition values' config.value ./types-anything/lists.nix
# Check that all equalizable atoms can be used as long as all definitions are equal
checkConfigOutput '^0$' config.value.int ./types-anything/equal-atoms.nix
checkConfigOutput '^false$' config.value.bool ./types-anything/equal-atoms.nix
checkConfigOutput '^""$' config.value.string ./types-anything/equal-atoms.nix
checkConfigOutput '^"/[^"]+"$' config.value.path ./types-anything/equal-atoms.nix
checkConfigOutput '^null$' config.value.null ./types-anything/equal-atoms.nix
checkConfigOutput '^0.1$' config.value.float ./types-anything/equal-atoms.nix
checkConfigOutput '^\[1,"a",{"x":null}\]$' config.value.list ./types-anything/equal-atoms.nix
# Functions can't be merged together
checkConfigError "The option .value.multiple-lambdas.<function body>. has conflicting option types" config.applied.multiple-lambdas ./types-anything/functions.nix
checkConfigOutput '^true$' config.valueIsFunction.single-lambda ./types-anything/functions.nix
checkConfigOutput '^null$' config.applied.merging-lambdas.x ./types-anything/functions.nix
checkConfigOutput '^null$' config.applied.merging-lambdas.y ./types-anything/functions.nix
# Check that all mk* modifiers are applied
checkConfigError 'attribute .* not found' config.value.mkiffalse ./types-anything/mk-mods.nix
checkConfigOutput '^{}$' config.value.mkiftrue ./types-anything/mk-mods.nix
checkConfigOutput '^1$' config.value.mkdefault ./types-anything/mk-mods.nix
checkConfigOutput '^{}$' config.value.mkmerge ./types-anything/mk-mods.nix
checkConfigOutput '^true$' config.value.mkbefore ./types-anything/mk-mods.nix
checkConfigOutput '^1$' config.value.nested.foo ./types-anything/mk-mods.nix
checkConfigOutput '^"baz"$' config.value.nested.bar.baz ./types-anything/mk-mods.nix
## types.functionTo
checkConfigOutput '^"input is input"$' config.result ./functionTo/trivial.nix
checkConfigOutput '^"a b"$' config.result ./functionTo/merging-list.nix
checkConfigError 'A definition for option .fun.<function body>. is not of type .string.. Definition values:\n\s*- In .*wrong-type.nix' config.result ./functionTo/wrong-type.nix
checkConfigOutput '^"b a"$' config.result ./functionTo/list-order.nix
checkConfigOutput '^"a c"$' config.result ./functionTo/merging-attrs.nix
checkConfigOutput '^"a bee"$' config.result ./functionTo/submodule-options.nix
checkConfigOutput '^"fun.<function body>.a fun.<function body>.b"$' config.optionsResult ./functionTo/submodule-options.nix
# moduleType
checkConfigOutput '^"a b"$' config.resultFoo ./declare-variants.nix ./define-variant.nix
checkConfigOutput '^"a b y z"$' config.resultFooBar ./declare-variants.nix ./define-variant.nix
checkConfigOutput '^"a b c"$' config.resultFooFoo ./declare-variants.nix ./define-variant.nix
## emptyValue's
checkConfigOutput "\[\]" config.list.a ./emptyValues.nix
checkConfigOutput "{}" config.attrs.a ./emptyValues.nix
checkConfigOutput "null" config.null.a ./emptyValues.nix
checkConfigOutput "{}" config.submodule.a ./emptyValues.nix
# These types don't have empty values
checkConfigError 'The option .int.a. was accessed but has no value defined. Try setting the option.' config.int.a ./emptyValues.nix
checkConfigError 'The option .nonEmptyList.a. was accessed but has no value defined. Try setting the option.' config.nonEmptyList.a ./emptyValues.nix
# types.unique
# requires a single definition
checkConfigError 'The option .examples\.merged. is defined multiple times while it.s expected to be unique' config.examples.merged.a ./types-unique.nix
# user message is printed
checkConfigError 'We require a single definition, because seeing the whole value at once helps us maintain critical invariants of our system.' config.examples.merged.a ./types-unique.nix
# let the inner merge function check the values (on demand)
checkConfigError 'A definition for option .examples\.badLazyType\.a. is not of type .string.' config.examples.badLazyType.a ./types-unique.nix
# overriding still works (unlike option uniqueness)
checkConfigOutput '^"bee"$' config.examples.override.b ./types-unique.nix
## types.raw
checkConfigOutput '^true$' config.unprocessedNestingEvaluates.success ./raw.nix
checkConfigOutput "10" config.processedToplevel ./raw.nix
checkConfigError "The option .multiple. is defined multiple times" config.multiple ./raw.nix
checkConfigOutput "bar" config.priorities ./raw.nix
## Option collision
checkConfigError \
'The option .set. in module .*/declare-set.nix. would be a parent of the following options, but its type .attribute set of signed integer. does not support nested options.\n\s*- option[(]s[)] with prefix .set.enable. in module .*/declare-enable-nested.nix.' \
config.set \
./declare-set.nix ./declare-enable-nested.nix
# Options: accidental use of an option-type instead of option (or other tagged type; unlikely)
checkConfigError 'In module .*/options-type-error-typical.nix: expected an option declaration at option path .result. but got an attribute set with type option-type' config.result ./options-type-error-typical.nix
checkConfigError 'In module .*/options-type-error-typical-nested.nix: expected an option declaration at option path .result.here. but got an attribute set with type option-type' config.result.here ./options-type-error-typical-nested.nix
checkConfigError 'In module .*/options-type-error-configuration.nix: expected an option declaration at option path .result. but got an attribute set with type configuration' config.result ./options-type-error-configuration.nix
# Check that that merging of option collisions doesn't depend on type being set
checkConfigError 'The option .group..*would be a parent of the following options, but its type .<no description>. does not support nested options.\n\s*- option.s. with prefix .group.enable..*' config.group.enable ./merge-typeless-option.nix
# Test that types.optionType merges types correctly
checkConfigOutput '^10$' config.theOption.int ./optionTypeMerging.nix
checkConfigOutput '^"hello"$' config.theOption.str ./optionTypeMerging.nix
# Test that types.optionType correctly annotates option locations
checkConfigError 'The option .theOption.nested. in .other.nix. is already declared in .optionTypeFile.nix.' config.theOption.nested ./optionTypeFile.nix
# Test that types.optionType leaves types untouched as long as they don't need to be merged
checkConfigOutput 'ok' config.freeformItems.foo.bar ./adhoc-freeformType-survives-type-merge.nix
# Test that specifying both functor.wrapped and functor.payload isn't allowed
checkConfigError 'Type foo defines both `functor.payload` and `functor.wrapped` at the same time, which is not supported.' config.result ./default-type-merge-both.nix
# Anonymous submodules don't get nixed by import resolution/deduplication
# because of an `extendModules` bug, issue 168767.
checkConfigOutput '^1$' config.sub.specialisation.value ./extendModules-168767-imports.nix
# Class checks, evalModules
checkConfigOutput '^{}$' config.ok.config ./class-check.nix
checkConfigOutput '"nixos"' config.ok.class ./class-check.nix
checkConfigError 'The module `.*/module-class-is-darwin.nix`.*?expects class "nixos".' config.fail.config ./class-check.nix
checkConfigError 'The module `foo.nix#darwinModules.default`.*?expects class "nixos".' config.fail-anon.config ./class-check.nix
# Class checks, submoduleWith
checkConfigOutput '^{}$' config.sub.nixosOk ./class-check.nix
checkConfigError 'The module `.*/module-class-is-darwin.nix`.*?expects class "nixos".' config.sub.nixosFail.config ./class-check.nix
# submoduleWith type merge with different class
checkConfigError 'A submoduleWith option is declared multiple times with conflicting class values "darwin" and "nixos".' config.sub.mergeFail.config ./class-check.nix
# _type check
checkConfigError 'Expected a module, but found a value of type .*"flake".*, while trying to load a module into .*/module-imports-_type-check.nix' config.ok.config ./module-imports-_type-check.nix
checkConfigOutput '^true$' config.enable ./declare-enable.nix ./define-enable-with-top-level-mkIf.nix
checkConfigError 'Expected a module, but found a value of type .*"configuration".*, while trying to load a module into .*/import-configuration.nix.' config ./import-configuration.nix
checkConfigError 'please only import the modules that make up the configuration' config ./import-configuration.nix
checkConfigError 'Expected a module, but found a value of type "configuration", while trying to load a module into .*/import-error-submodule.nix, while trying to load a module into .*foo.*\.' config.foo ./import-error-submodule.nix
# doRename works when `warnings` does not exist.
checkConfigOutput '^1234$' config.c.d.e ./doRename-basic.nix
# doRename adds a warning.
checkConfigOutput '^"The option `a\.b. defined in `.*/doRename-warnings\.nix. has been renamed to `c\.d\.e.\."$' \
config.result \
./doRename-warnings.nix
checkConfigOutput "^true$" config.result ./doRename-condition.nix ./doRename-condition-enable.nix
checkConfigOutput "^true$" config.result ./doRename-condition.nix ./doRename-condition-no-enable.nix
checkConfigOutput "^true$" config.result ./doRename-condition.nix ./doRename-condition-migrated.nix
# Anonymous modules get deduplicated by key
checkConfigOutput '^"pear"$' config.once.raw ./merge-module-with-key.nix
checkConfigOutput '^"pear\\npear"$' config.twice.raw ./merge-module-with-key.nix
# Declaration positions
# Line should be present for direct options
checkConfigOutput '^14$' options.imported.line14.declarationPositions.0.line ./declaration-positions.nix
checkConfigOutput '/declaration-positions.nix"$' options.imported.line14.declarationPositions.0.file ./declaration-positions.nix
# Generated options may not have line numbers but they will at least get the
# right file
checkConfigOutput '/declaration-positions.nix"$' options.generated.line22.declarationPositions.0.file ./declaration-positions.nix
checkConfigOutput '^null$' options.generated.line22.declarationPositions.0.line ./declaration-positions.nix
# Submodules don't break it
checkConfigOutput '^45$' config.submoduleLine38.submodDeclLine45.0.line ./declaration-positions.nix
checkConfigOutput '/declaration-positions.nix"$' config.submoduleLine38.submodDeclLine45.0.file ./declaration-positions.nix
# New options under freeform submodules get collected into the parent submodule
# (consistent with .declarations behaviour, but weird; notably appears in system.build)
checkConfigOutput '^38|27$' options.submoduleLine38.declarationPositions.0.line ./declaration-positions.nix
checkConfigOutput '^38|27$' options.submoduleLine38.declarationPositions.1.line ./declaration-positions.nix
# nested options work
checkConfigOutput '^34$' options.nested.nestedLine34.declarationPositions.0.line ./declaration-positions.nix
# types.pathWith { inStore = true; }
checkConfigOutput '".*/store/0lz9p8xhf89kb1c1kk6jxrzskaiygnlh-bash-5.2-p15.drv"' config.pathInStore.ok1 ./pathWith.nix
checkConfigOutput '".*/store/0fb3ykw9r5hpayd05sr0cizwadzq1d8q-bash-5.2-p15"' config.pathInStore.ok2 ./pathWith.nix
checkConfigOutput '".*/store/0fb3ykw9r5hpayd05sr0cizwadzq1d8q-bash-5.2-p15/bin/bash"' config.pathInStore.ok3 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ""' config.pathInStore.bad1 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ".*/store"' config.pathInStore.bad2 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ".*/store/"' config.pathInStore.bad3 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: ".*/store/.links"' config.pathInStore.bad4 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path in the Nix store.. Definition values:\n\s*- In .*: "/foo/bar"' config.pathInStore.bad5 ./pathWith.nix
# types.pathWith { inStore = false; }
checkConfigOutput '"/foo/bar"' config.pathNotInStore.ok1 ./pathWith.nix
checkConfigOutput '".*/store"' config.pathNotInStore.ok2 ./pathWith.nix
checkConfigOutput '".*/store/"' config.pathNotInStore.ok3 ./pathWith.nix
checkConfigOutput '""' config.pathNotInStore.ok4 ./pathWith.nix
checkConfigOutput '".*/store/.links"' config.pathNotInStore.ok5 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path not in the Nix store.. Definition values:\n\s*- In .*: ".*/0lz9p8xhf89kb1c1kk6jxrzskaiygnlh-bash-5.2-p15.drv"' config.pathNotInStore.bad1 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path not in the Nix store.. Definition values:\n\s*- In .*: ".*/0fb3ykw9r5hpayd05sr0cizwadzq1d8q-bash-5.2-p15"' config.pathNotInStore.bad2 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path not in the Nix store.. Definition values:\n\s*- In .*: ".*/0fb3ykw9r5hpayd05sr0cizwadzq1d8q-bash-5.2-p15/bin/bash"' config.pathNotInStore.bad3 ./pathWith.nix
checkConfigError 'A definition for option .* is not of type .path not in the Nix store.. Definition values:\n\s*- In .*: .*/pathWith.nix' config.pathNotInStore.bad4 ./pathWith.nix
# types.pathWith { }
checkConfigOutput '"/this/is/absolute"' config.anyPath.ok1 ./pathWith.nix
checkConfigOutput '"./this/is/relative"' config.anyPath.ok2 ./pathWith.nix
checkConfigError 'A definition for option .anyPath.bad1. is not of type .path.' config.anyPath.bad1 ./pathWith.nix
# types.pathWith { absolute = true; }
checkConfigOutput '"/this/is/absolute"' config.absolutePathNotInStore.ok1 ./pathWith.nix
checkConfigError 'A definition for option .absolutePathNotInStore.bad1. is not of type .absolute path not in the Nix store.' config.absolutePathNotInStore.bad1 ./pathWith.nix
checkConfigError 'A definition for option .absolutePathNotInStore.bad2. is not of type .absolute path not in the Nix store.' config.absolutePathNotInStore.bad2 ./pathWith.nix
# types.pathWith failed type merge
checkConfigError 'The option .conflictingPathOptionType. in .*/pathWith.nix. is already declared in .*/pathWith.nix' config.conflictingPathOptionType ./pathWith.nix
# types.pathWith { inStore = true; absolute = false; }
checkConfigError 'In pathWith, inStore means the path must be absolute' config.impossiblePathOptionType ./pathWith.nix
# mkDefinition
# check that mkDefinition 'file' is printed in the error message
checkConfigError 'Cannot merge definitions.*\n\s*- In .file.*\n\s*- In .other.*' config.conflict ./mkDefinition.nix
checkConfigError 'A definition for option .viaOptionDefault. is not of type .boolean.*' config.viaOptionDefault ./mkDefinition.nix
checkConfigOutput '^true$' config.viaConfig ./mkDefinition.nix
checkConfigOutput '^true$' config.mkMerge ./mkDefinition.nix
checkConfigOutput '^true$' config.mkForce ./mkDefinition.nix
# specialArgs._class
checkConfigOutput '"nixos"' config.nixos.config.foo ./specialArgs-class.nix
checkConfigOutput '"bar"' config.conditionalImportAsNixos.config.foo ./specialArgs-class.nix
checkConfigError 'attribute .*bar.* not found' config.conditionalImportAsNixos.config.bar ./specialArgs-class.nix
checkConfigError 'attribute .*foo.* not found' config.conditionalImportAsDarwin.config.foo ./specialArgs-class.nix
checkConfigOutput '"foo"' config.conditionalImportAsDarwin.config.bar ./specialArgs-class.nix
checkConfigOutput '"nixos"' config.sub.nixos.foo ./specialArgs-class.nix
checkConfigOutput '"bar"' config.sub.conditionalImportAsNixos.foo ./specialArgs-class.nix
checkConfigError 'attribute .*bar.* not found' config.sub.conditionalImportAsNixos.bar ./specialArgs-class.nix
checkConfigError 'attribute .*foo.* not found' config.sub.conditionalImportAsDarwin.foo ./specialArgs-class.nix
checkConfigOutput '"foo"' config.sub.conditionalImportAsDarwin.bar ./specialArgs-class.nix
# Check that some types expose the 'valueMeta'
checkConfigOutput '\{\}' options.str.valueMeta ./types-valueMeta.nix
checkConfigOutput '["foo", "bar"]' config.attrsOfResult ./types-valueMeta.nix
checkConfigOutput '2' config.listOfResult ./types-valueMeta.nix
# Check that composed types expose the 'valueMeta'
# attrsOf submodule (also on merged options,types)
checkConfigOutput '42' options.attrsOfModule.valueMeta.attrs.foo.configuration.options.bar.value ./composed-types-valueMeta.nix
checkConfigOutput '42' options.mergedAttrsOfModule.valueMeta.attrs.foo.configuration.options.bar.value ./composed-types-valueMeta.nix
# listOf submodule (also on merged options,types)
checkConfigOutput '42' config.listResult ./composed-types-valueMeta.nix
checkConfigOutput '42' config.mergedListResult ./composed-types-valueMeta.nix
# Add check
checkConfigOutput '^0$' config.v1CheckedPass ./add-check.nix
checkConfigError 'A definition for option .* is not of type .signed integer.*' config.v1CheckedFail ./add-check.nix
checkConfigOutput '^true$' config.v2checkedPass ./add-check.nix
checkConfigError 'A definition for option .* is not of type .attribute set of signed integer.*' config.v2checkedFail ./add-check.nix
cat <<EOF
====== module tests ======
$pass Pass
$fail Fail
EOF
if [ "$fail" -ne 0 ]; then
exit 1
fi
exit 0

36
lib/tests/modules/add-check.nix Normal file
View File

@@ -0,0 +1,36 @@
(
{ lib, ... }:
let
inherit (lib) types mkOption;
inherit (types) addCheck int attrsOf;
# type with a v1 merge
v1Type = addCheck int (v: v == 0);
# type with a v2 merge
v2Type = addCheck (attrsOf int) (v: v ? foo);
in
{
options.v1CheckedPass = mkOption {
type = v1Type;
default = 0;
};
options.v1CheckedFail = mkOption {
type = v1Type;
default = 1;
};
options.v2checkedPass = mkOption {
type = v2Type;
default = {
foo = 1;
};
# plug the value to make test script regex simple
apply = v: v.foo == 1;
};
options.v2checkedFail = mkOption {
type = v2Type;
default = { };
apply = v: lib.deepSeq v v;
};
}
)

19
lib/tests/modules/adhoc-freeformType-survives-type-merge.nix Normal file
View File

@@ -0,0 +1,19 @@
{ lib, ... }:
{
options.dummy = lib.mkOption {
type = lib.types.anything;
default = { };
};
freeformType =
let
a = lib.types.attrsOf (lib.types.submodule { options.bar = lib.mkOption { }; });
in
# modifying types like this breaks type merging.
# This test makes sure that type merging is not performed when only a single declaration exists.
# Don't modify types in practice!
a
// {
merge = loc: defs: { freeformItems = a.merge loc defs; };
};
config.foo.bar = "ok";
}

64
lib/tests/modules/alias-with-priority-can-override.nix Normal file
View File

@@ -0,0 +1,64 @@
# This is a test to show that mkAliasOptionModule sets the priority correctly
# for aliased options.
#
# This test shows that an alias with a high priority is able to override
# a non-aliased option.
{ config, lib, ... }:
let
inherit (lib)
mkAliasOptionModule
mkForce
mkOption
types
;
in
{
options = {
# A simple boolean option that can be enabled or disabled.
enable = mkOption {
type = types.nullOr types.bool;
default = null;
example = true;
description = ''
Some descriptive text
'';
};
# mkAliasOptionModule sets warnings, so this has to be defined.
warnings = mkOption {
internal = true;
default = [ ];
type = types.listOf types.str;
example = [ "The `foo' service is deprecated and will go away soon!" ];
description = ''
This option allows modules to show warnings to users during
the evaluation of the system configuration.
'';
};
};
imports = [
# Create an alias for the "enable" option.
(mkAliasOptionModule [ "enableAlias" ] [ "enable" ])
# Disable the aliased option with a high priority so it
# should override the next import.
(
{ config, lib, ... }:
{
enableAlias = mkForce false;
}
)
# Enable the normal (non-aliased) option.
(
{ config, lib, ... }:
{
enable = true;
}
)
];
}

64
lib/tests/modules/alias-with-priority.nix Normal file
View File

@@ -0,0 +1,64 @@
# This is a test to show that mkAliasOptionModule sets the priority correctly
# for aliased options.
#
# This test shows that an alias with a low priority is able to be overridden
# with a non-aliased option.
{ config, lib, ... }:
let
inherit (lib)
mkAliasOptionModule
mkDefault
mkOption
types
;
in
{
options = {
# A simple boolean option that can be enabled or disabled.
enable = mkOption {
type = types.nullOr types.bool;
default = null;
example = true;
description = ''
Some descriptive text
'';
};
# mkAliasOptionModule sets warnings, so this has to be defined.
warnings = mkOption {
internal = true;
default = [ ];
type = types.listOf types.str;
example = [ "The `foo' service is deprecated and will go away soon!" ];
description = ''
This option allows modules to show warnings to users during
the evaluation of the system configuration.
'';
};
};
imports = [
# Create an alias for the "enable" option.
(mkAliasOptionModule [ "enableAlias" ] [ "enable" ])
# Disable the aliased option, but with a default (low) priority so it
# should be able to be overridden by the next import.
(
{ config, lib, ... }:
{
enableAlias = mkDefault false;
}
)
# Enable the normal (non-aliased) option.
(
{ config, lib, ... }:
{
enable = true;
}
)
];
}

3
lib/tests/modules/assert-module-class-is-nixos.nix Normal file
View File

@@ -0,0 +1,3 @@
{ _class, ... }:
assert _class == "nixos";
{ }

8
lib/tests/modules/attrsOf-conditional-check.nix Normal file
View File

@@ -0,0 +1,8 @@
{ lib, config, ... }:
{
options.conditionalWorks = lib.mkOption {
default = !config.value ? foo;
};
config.value.foo = lib.mkIf false "should not be defined";
}

8
lib/tests/modules/attrsOf-lazy-check.nix Normal file
View File

@@ -0,0 +1,8 @@
{ lib, config, ... }:
{
options.isLazy = lib.mkOption {
default = !config.value ? foo;
};
config.value.bar = throw "is not lazy";
}

26
lib/tests/modules/boolByOr.nix Normal file
View File

@@ -0,0 +1,26 @@
{ lib, ... }:
{
options.value = lib.mkOption {
type = lib.types.lazyAttrsOf lib.types.boolByOr;
};
config.value = {
falseFalse = lib.mkMerge [
false
false
];
trueFalse = lib.mkMerge [
true
false
];
falseTrue = lib.mkMerge [
false
true
];
trueTrue = lib.mkMerge [
true
true
];
};
}

82
lib/tests/modules/class-check.nix Normal file
View File

@@ -0,0 +1,82 @@
{ lib, ... }:
{
options = {
sub = {
nixosOk = lib.mkOption {
type = lib.types.submoduleWith {
class = "nixos";
modules = [
./assert-module-class-is-nixos.nix
];
};
};
# Same but will have bad definition
nixosFail = lib.mkOption {
type = lib.types.submoduleWith {
class = "nixos";
modules = [ ];
};
};
mergeFail = lib.mkOption {
type = lib.types.submoduleWith {
class = "nixos";
modules = [ ];
};
default = { };
};
};
};
imports = [
{
options = {
sub = {
mergeFail = lib.mkOption {
type = lib.types.submoduleWith {
class = "darwin";
modules = [ ];
};
};
};
};
}
];
config = {
_module.freeformType = lib.types.anything;
ok = lib.evalModules {
class = "nixos";
modules = [
./module-class-is-nixos.nix
./assert-module-class-is-nixos.nix
];
};
fail = lib.evalModules {
class = "nixos";
modules = [
./module-class-is-nixos.nix
./module-class-is-darwin.nix
];
};
fail-anon = lib.evalModules {
class = "nixos";
modules = [
./module-class-is-nixos.nix
{
_file = "foo.nix#darwinModules.default";
_class = "darwin";
config = { };
imports = [ ];
}
];
};
sub.nixosOk = {
_class = "nixos";
};
sub.nixosFail = {
imports = [ ./module-class-is-darwin.nix ];
};
};
}

75
lib/tests/modules/composed-types-valueMeta.nix Normal file
View File

@@ -0,0 +1,75 @@
{ lib, ... }:
let
inherit (lib) types mkOption;
attrsOfModule = mkOption {
type = types.attrsOf (
types.submodule {
options.bar = mkOption {
type = types.int;
};
}
);
};
listOfModule = mkOption {
type = types.listOf (
types.submodule {
options.bar = mkOption {
type = types.int;
};
}
);
};
in
{
imports = [
# Module A
{
options.attrsOfModule = attrsOfModule;
options.mergedAttrsOfModule = attrsOfModule;
options.listOfModule = listOfModule;
options.mergedListOfModule = listOfModule;
}
# Module B
{
options.mergedAttrsOfModule = attrsOfModule;
options.mergedListOfModule = listOfModule;
}
# Values
# It is important that the value is defined in a separate module
# Without valueMeta the actual value and sub-options wouldn't be accessible via:
# options.attrsOfModule.type.getSubOptions
{
attrsOfModule = {
foo.bar = 42;
};
mergedAttrsOfModule = {
foo.bar = 42;
};
}
(
{ options, ... }:
{
config.listOfModule = [
{
bar = 42;
}
];
config.mergedListOfModule = [
{
bar = 42;
}
];
# Result options to expose the list module to bash as plain attribute path
options.listResult = mkOption {
default = (builtins.head options.listOfModule.valueMeta.list).configuration.options.bar.value;
};
options.mergedListResult = mkOption {
default = (builtins.head options.mergedListOfModule.valueMeta.list).configuration.options.bar.value;
};
}
)
];
}

57
lib/tests/modules/declaration-positions.nix Normal file
View File

@@ -0,0 +1,57 @@
{ lib, options, ... }:
let
discardPositions = lib.mapAttrs (k: v: v);
in
# unsafeGetAttrPos is unspecified best-effort behavior, so we only want to consider this test on an evaluator that satisfies some basic assumptions about this function.
assert builtins.unsafeGetAttrPos "a" { a = true; } != null;
assert
builtins.unsafeGetAttrPos "a" (discardPositions {
a = true;
}) == null;
{
imports = [
{
options.imported.line14 = lib.mkOption {
type = lib.types.int;
};
# Simulates various patterns of generating modules such as
# programs.firefox.nativeMessagingHosts.ff2mpv. We don't expect to get
# line numbers for these, but we can fall back on knowing the file.
options.generated = discardPositions {
line22 = lib.mkOption {
type = lib.types.int;
};
};
options.submoduleLine38.extraOptLine27 = lib.mkOption {
default = 1;
type = lib.types.int;
};
}
];
options.nested.nestedLine34 = lib.mkOption {
type = lib.types.int;
};
options.submoduleLine38 = lib.mkOption {
default = { };
type = lib.types.submoduleWith {
modules = [
(
{ options, ... }:
{
options.submodDeclLine45 = lib.mkOption { };
}
)
{ freeformType = with lib.types; lazyAttrsOf (uniq unspecified); }
];
};
};
config = {
submoduleLine38.submodDeclLine45 =
(options.submoduleLine38.type.getSubOptions [ ]).submodDeclLine45.declarationPositions;
};
}

13
lib/tests/modules/declare-attrsOf.nix Normal file
View File

@@ -0,0 +1,13 @@
{ lib, ... }:
let
deathtrapArgs = lib.mapAttrs (
k: _: throw "The module system is too strict, accessing an unused option's ${k} mkOption-attribute."
) (lib.functionArgs lib.mkOption);
in
{
options.value = lib.mkOption {
type = lib.types.attrsOf lib.types.str;
default = { };
};
options.testing-laziness-so-don't-read-me = lib.mkOption deathtrapArgs;
}

31
lib/tests/modules/declare-attrsOfSub-any-enable.nix Normal file
View File

@@ -0,0 +1,31 @@
{ lib, ... }:
let
submod =
{ ... }:
{
options = {
enable = lib.mkOption {
default = false;
example = true;
type = lib.types.bool;
description = ''
Some descriptive text
'';
};
};
};
in
{
options = {
attrsOfSub = lib.mkOption {
default = { };
example = { };
type = lib.types.attrsOf (lib.types.submodule [ submod ]);
description = ''
Some descriptive text
'';
};
};
}

10
lib/tests/modules/declare-bare-submodule-deep-option-duplicate.nix Normal file
View File

@@ -0,0 +1,10 @@
{ lib, ... }:
let
inherit (lib) mkOption types;
in
{
options.bare-submodule.deep = mkOption {
type = types.int;
default = 2;
};
}

10
lib/tests/modules/declare-bare-submodule-deep-option.nix Normal file
View File

@@ -0,0 +1,10 @@
{ lib, ... }:
let
inherit (lib) mkOption types;
in
{
options.bare-submodule.deep = mkOption {
type = types.int;
default = 2;
};
}

19
lib/tests/modules/declare-bare-submodule-nested-option.nix Normal file
View File

@@ -0,0 +1,19 @@
{ config, lib, ... }:
let
inherit (lib) mkOption types;
in
{
options.bare-submodule = mkOption {
type = types.submoduleWith {
shorthandOnlyDefinesConfig = config.shorthandOnlyDefinesConfig;
modules = [
{
options.nested = mkOption {
type = types.int;
default = 1;
};
}
];
};
};
}

18
lib/tests/modules/declare-bare-submodule.nix Normal file
View File

@@ -0,0 +1,18 @@
{ config, lib, ... }:
let
inherit (lib) mkOption types;
in
{
options.bare-submodule = mkOption {
type = types.submoduleWith {
modules = [ ];
shorthandOnlyDefinesConfig = config.shorthandOnlyDefinesConfig;
};
default = { };
};
# config-dependent options: won't recommend, but useful for making this test parameterized
options.shorthandOnlyDefinesConfig = mkOption {
default = false;
};
}

9
lib/tests/modules/declare-coerced-value-no-default.nix Normal file
View File

@@ -0,0 +1,9 @@
{ lib, ... }:
{
options = {
value = lib.mkOption {
type = lib.types.coercedTo lib.types.int toString lib.types.str;
};
};
}

10
lib/tests/modules/declare-coerced-value-unsound.nix Normal file
View File

@@ -0,0 +1,10 @@
{ lib, ... }:
{
options = {
value = lib.mkOption {
default = "12";
type = lib.types.coercedTo lib.types.str lib.toInt lib.types.ints.s8;
};
};
}

10
lib/tests/modules/declare-coerced-value.nix Normal file
View File

@@ -0,0 +1,10 @@
{ lib, ... }:
{
options = {
value = lib.mkOption {
default = 42;
type = lib.types.coercedTo lib.types.int toString lib.types.str;
};
};
}

6
lib/tests/modules/declare-either.nix Normal file
View File

@@ -0,0 +1,6 @@
{ lib, ... }:
{
options.value = lib.mkOption {
type = lib.types.either lib.types.int lib.types.str;
};
}

14
lib/tests/modules/declare-enable-nested.nix Normal file
View File

@@ -0,0 +1,14 @@
{ lib, ... }:
{
options.set = {
enable = lib.mkOption {
default = false;
example = true;
type = lib.types.bool;
description = ''
Some descriptive text
'';
};
};
}

14
lib/tests/modules/declare-enable.nix Normal file
View File

@@ -0,0 +1,14 @@
{ lib, ... }:
{
options = {
enable = lib.mkOption {
default = false;
example = true;
type = lib.types.bool;
description = ''
Some descriptive text
'';
};
};
}

9
lib/tests/modules/declare-int-between-value.nix Normal file
View File

@@ -0,0 +1,9 @@
{ lib, ... }:
{
options = {
value = lib.mkOption {
type = lib.types.ints.between (-21) 43;
};
};
}

9
lib/tests/modules/declare-int-positive-value-nested.nix Normal file
View File

@@ -0,0 +1,9 @@
{ lib, ... }:
{
options.set = {
value = lib.mkOption {
type = lib.types.ints.positive;
};
};
}

9
lib/tests/modules/declare-int-positive-value.nix Normal file
View File

@@ -0,0 +1,9 @@
{ lib, ... }:
{
options = {
value = lib.mkOption {
type = lib.types.ints.positive;
};
};
}

9
lib/tests/modules/declare-int-unsigned-value.nix Normal file
View File

@@ -0,0 +1,9 @@
{ lib, ... }:
{
options = {
value = lib.mkOption {
type = lib.types.ints.unsigned;
};
};
}

7
lib/tests/modules/declare-lazyAttrsOf.nix Normal file
View File

@@ -0,0 +1,7 @@
{ lib, ... }:
{
options.value = lib.mkOption {
type = lib.types.lazyAttrsOf (lib.types.str // { emptyValue.value = "empty"; });
default = { };
};
}

75
lib/tests/modules/declare-mkPackageOption.nix Normal file
View File

@@ -0,0 +1,75 @@
{ lib, ... }:
let
pkgs.hello = {
type = "derivation";
pname = "hello";
};
in
{
options = {
package = lib.mkPackageOption pkgs "hello" { };
namedPackage = lib.mkPackageOption pkgs "Hello" {
default = [ "hello" ];
};
namedPackageSingletonDefault = lib.mkPackageOption pkgs "Hello" {
default = "hello";
};
pathPackage = lib.mkPackageOption pkgs [ "hello" ] { };
packageWithExample = lib.mkPackageOption pkgs "hello" {
example = "pkgs.hello.override { stdenv = pkgs.clangStdenv; }";
};
packageWithPathExample = lib.mkPackageOption pkgs "hello" {
example = [ "hello" ];
};
packageWithExtraDescription = lib.mkPackageOption pkgs "hello" {
extraDescription = "Example extra description.";
};
undefinedPackage = lib.mkPackageOption pkgs "hello" {
default = null;
};
nullablePackage = lib.mkPackageOption pkgs "hello" {
nullable = true;
default = null;
};
nullablePackageWithDefault = lib.mkPackageOption pkgs "hello" {
nullable = true;
};
packageWithPkgsText = lib.mkPackageOption pkgs "hello" {
pkgsText = "myPkgs";
};
packageFromOtherSet =
let
myPkgs = {
hello = pkgs.hello // {
pname = "hello-other";
};
};
in
lib.mkPackageOption myPkgs "hello" { };
packageInvalidIdentifier =
let
myPkgs."123"."with\"quote" = { inherit (pkgs) hello; };
in
lib.mkPackageOption myPkgs [ "123" "with\"quote" "hello" ] { };
packageInvalidIdentifierExample = lib.mkPackageOption pkgs "hello" {
example = [
"123"
"with\"quote"
"hello"
];
};
};
}

10
lib/tests/modules/declare-oneOf.nix Normal file
View File

@@ -0,0 +1,10 @@
{ lib, ... }:
{
options.value = lib.mkOption {
type = lib.types.oneOf [
lib.types.int
(lib.types.listOf lib.types.int)
lib.types.str
];
};
}

14
lib/tests/modules/declare-set.nix Normal file
View File

@@ -0,0 +1,14 @@
{ lib, ... }:
{
options.set = lib.mkOption {
default = { };
example = {
a = 1;
};
type = lib.types.attrsOf lib.types.int;
description = ''
Some descriptive text
'';
};
}

35
lib/tests/modules/declare-submodule-via-evalModules.nix Normal file
View File

@@ -0,0 +1,35 @@
{ lib, ... }:
{
options.submodule = lib.mkOption {
inherit
(lib.evalModules {
modules = [
{
options.inner = lib.mkOption {
type = lib.types.bool;
default = false;
};
}
];
})
type
;
default = { };
};
config.submodule = lib.mkMerge [
(
{ lib, ... }:
{
options.outer = lib.mkOption {
type = lib.types.bool;
default = false;
};
}
)
{
inner = true;
outer = true;
}
];
}

32
lib/tests/modules/declare-submoduleWith-modules.nix Normal file
View File

@@ -0,0 +1,32 @@
{ lib, ... }:
{
options.submodule = lib.mkOption {
type = lib.types.submoduleWith {
modules = [
{
options.inner = lib.mkOption {
type = lib.types.bool;
default = false;
};
}
];
};
default = { };
};
config.submodule = lib.mkMerge [
(
{ lib, ... }:
{
options.outer = lib.mkOption {
type = lib.types.bool;
default = false;
};
}
)
{
inner = true;
outer = true;
}
];
}

15
lib/tests/modules/declare-submoduleWith-noshorthand.nix Normal file
View File

@@ -0,0 +1,15 @@
{ lib, ... }:
let
sub.options.config = lib.mkOption {
type = lib.types.bool;
default = false;
};
in
{
options.submodule = lib.mkOption {
type = lib.types.submoduleWith {
modules = [ sub ];
};
default = { };
};
}

13
lib/tests/modules/declare-submoduleWith-path.nix Normal file
View File

@@ -0,0 +1,13 @@
{ lib, ... }:
{
options.submodule = lib.mkOption {
type = lib.types.submoduleWith {
modules = [
./declare-enable.nix
];
};
default = { };
};
config.submodule = ./define-enable.nix;
}

16
lib/tests/modules/declare-submoduleWith-shorthand.nix Normal file
View File

@@ -0,0 +1,16 @@
{ lib, ... }:
let
sub.options.config = lib.mkOption {
type = lib.types.bool;
default = false;
};
in
{
options.submodule = lib.mkOption {
type = lib.types.submoduleWith {
modules = [ sub ];
shorthandOnlyDefinesConfig = true;
};
default = { };
};
}

21
lib/tests/modules/declare-submoduleWith-special.nix Normal file
View File

@@ -0,0 +1,21 @@
{ lib, ... }:
{
options.submodule = lib.mkOption {
type = lib.types.submoduleWith {
modules = [
(
{ lib, ... }:
{
options.foo = lib.mkOption {
default = lib.foo;
};
}
)
];
specialArgs.lib = lib // {
foo = "foo";
};
};
default = { };
};
}

10
lib/tests/modules/declare-variants.nix Normal file
View File

@@ -0,0 +1,10 @@
{ lib, moduleType, ... }:
let
inherit (lib) mkOption types;
in
{
options.variants = mkOption {
type = types.lazyAttrsOf moduleType;
default = { };
};
}

28
lib/tests/modules/default-type-merge-both.nix Normal file
View File

@@ -0,0 +1,28 @@
{ lib, options, ... }:
let
foo = lib.mkOptionType {
name = "foo";
functor = lib.types.defaultFunctor "foo" // {
wrapped = lib.types.int;
payload = 10;
};
};
in
{
imports = [
{
options.foo = lib.mkOption {
type = foo;
};
}
{
options.foo = lib.mkOption {
type = foo;
};
}
];
options.result = lib.mkOption {
default = builtins.seq options.foo null;
};
}

Some files were not shown because too many files have changed in this diff Show More