push sheeet
Some checks failed
Periodic Merges (6h) / master → staging-nixos (push) Failing after 12m50s
Periodic Merges (6h) / master → staging-next (push) Failing after 12m54s
Periodic Merges (24h) / merge-base(master,staging) → haskell-updates (push) Failing after 11m54s
Periodic Merges (6h) / staging-next → staging (push) Failing after 12m13s
Periodic Merges (24h) / staging-next-25.05 → staging-25.05 (push) Failing after 13m24s
Periodic Merges (24h) / release-25.05 → staging-next-25.05 (push) Failing after 14m28s

This commit is contained in:
Dark Steveneq
2025-10-09 14:15:47 +02:00
commit 646b892680
49168 changed files with 5897842 additions and 0 deletions

View File

@@ -0,0 +1,145 @@
# Modular Services
This directory defines a modular service infrastructure for NixOS.
See the [Modular Services chapter] in the manual [[source]](../../doc/manual/development/modular-services.md).
[Modular Services chapter]: https://nixos.org/manual/nixos/unstable/#modular-services
# Design decision log
## Initial design
- `system.services.<name>`. Alternatives considered
- `systemServices`: similar to does not allow importing a composition of services into `system`. Not sure if that's a good idea in the first place, but I've kept the possibility open.
- `services.abstract`: used in https://github.com/NixOS/nixpkgs/pull/267111, but too weird. Service modules should fit naturally into the configuration system.
Also "abstract" is wrong, because it has submodules - in other words, evalModules results, concrete services - not abstract at all.
- `services.modular`: only slightly better than `services.abstract`, but still weird
- No `daemon.*` options. https://github.com/NixOS/nixpkgs/pull/267111/files#r1723206521
- For now, do not add an `enable` option, because it's ambiguous. Does it disable at the Nix level (not generate anything) or at the systemd level (generate a service that is disabled)?
- Move all process options into a `process` option tree. Putting this at the root is messy, because we also have sub-services at that level. Those are rather distinct. Grouping them "by kind" should raise fewer questions.
- `modules/system/service/systemd/system.nix` has `system` twice. Not great, but
- they have different meanings
1. These are system-provided modules, provided by the configuration manager
2. `systemd/system` configures SystemD _system units_.
- This reserves `modules/service` for actual service modules, at least until those are lifted out of NixOS, potentially
## Configuration Data (`configData`) Design
Without a mechanism for adding files, all configuration had to go through `process.*`, requiring process restarts even when those would have been avoidable.
Many services implement automatic reloading or reloading on e.g. `SIGUSR1`, but those mechanisms need files to read. `configData` provides such files.
### Naming and Terminology
- **`configData` instead of `environment.etc`**: The name `configData` is service manager agnostic. While systemd system services can use `/etc`, other service managers may expose configuration data differently (e.g., different directory, relative paths).
- **`path` attribute**: Each `configData` entry automatically gets a `path` attribute set by the service manager implementation, allowing services to reference the location of their configuration files. These paths themselves are not subject to change from generation to generation; only their contents are.
- **`name` attribute**: In `environment.etc` this would be `target` but that's confusing, especially for symlinks, as it's not the symlink's target.
### Service Manager Integration
- **Portable base**: The `configData` interface is declared in `portable/config-data.nix`, making it available to all service manager implementations.
- **Systemd integration**: The systemd implementation (`systemd/system.nix`) maps `configData` entries to `environment.etc` entries under `/etc/system-services/`.
- **Path computation**: `systemd/config-data-path.nix` recursively computes unique paths for services and sub-services (e.g., `/etc/system-services/webserver/` vs `/etc/system-services/webserver-api/`).
Fun fact: for the module system it is a completely normal module, despite its recursive definition.
If we parameterize `/etc/system-services`, it will have to become an `importApply` style module nonetheless (function returning module).
- **Simple attribute structure**: Unlike `environment.etc`, `configData` uses a simpler structure with just `enable`, `name`, `text`, `source`, and `path` attributes. Complex ownership options were omitted for simplicity and portability.
Per-service user creation is still TBD.
## No `pkgs` module argument
The modular service infrastructure avoids exposing `pkgs` as a module argument to service modules. Instead, derivations and builder functions are provided through lexical closure, making dependency relationships explicit and avoiding uncertainty about where dependencies come from.
### Benefits
- **Explicit dependencies**: Services declare what they need rather than implicitly depending on `pkgs`
- **No interference**: Service modules can be reused in different contexts without assuming a specific `pkgs` instance. An unexpected `pkgs` version is not a failure mode anymore.
- **Clarity**: With fewer ways to do things, there's no ambiguity about where dependencies come from (from the module, not the OS or service manager)
### Implementation
- **Portable layer**: Service modules in `portable/` do not receive `pkgs` as a module argument. Any required derivations must be provided by the caller.
- **Systemd integration**: The `systemd/system.nix` module imports `config-data.nix` as a function, providing `pkgs` in lexical closure:
```nix
(import ../portable/config-data.nix { inherit pkgs; })
```
- **Service modules**:
1. Should explicitly declare their package dependencies as options rather than using `pkgs` defaults:
```nix
{
# Bad: uses pkgs module argument
foo.package = mkOption {
default = pkgs.python3;
# ...
};
}
```
```nix
{
# Good: caller provides the package
foo.package = mkOption {
type = types.package;
description = "Python package to use";
defaultText = lib.literalMD "The package that provided this module.";
};
}
```
2. `passthru.services` can still provide a complete module using the package's lexical scope, making the module truly self-contained:
**Package (`package.nix`):**
```nix
{
lib,
writeScript,
runtimeShell,
# ... other dependencies
}:
stdenv.mkDerivation (finalAttrs: {
# ... package definition
passthru.services.default = {
imports = [
(lib.modules.importApply ./service.nix {
inherit writeScript runtimeShell;
})
];
someService.package = finalAttrs.finalPackage;
};
})
```
**Service module (`service.nix`):**
```nix
# Non-module dependencies (importApply)
{ writeScript, runtimeShell }:
# Service module
{
lib,
config,
options,
...
}:
{
# Service definition using writeScript, runtimeShell from lexical scope
process.argv = [
(writeScript "wrapper" ''
#!${runtimeShell}
# ... wrapper logic
'')
# ... other args
];
}
```

View File

@@ -0,0 +1,65 @@
# Tests in: ../../../../tests/modular-service-etc/test.nix
# This file is a function that returns a module.
pkgs:
{
lib,
name,
config,
options,
...
}:
let
inherit (lib) mkOption types;
in
{
options = {
enable = mkOption {
type = types.bool;
default = true;
description = ''
Whether this configuration file should be generated.
This option allows specific configuration files to be disabled.
'';
};
name = mkOption {
type = types.str;
description = ''
Name of the configuration file (relative to the service's configuration directory). Defaults to the attribute name.
'';
};
path = mkOption {
type = types.str;
readOnly = true;
description = ''
The actual path where this configuration file will be available.
This is determined by the service manager implementation.
On NixOS it is an absolute path.
Other service managers may provide a relative path, in order to be unprivileged and/or relocatable.
'';
};
text = mkOption {
default = null;
type = types.nullOr types.lines;
description = "Text content of the configuration file.";
};
source = mkOption {
type = types.path;
description = "Path of the source file.";
};
};
config = {
name = lib.mkDefault name;
source = lib.mkIf (config.text != null) (
let
name' = "service-configdata-" + lib.replaceStrings [ "/" ] [ "-" ] name;
in
lib.mkDerivedConfig options.text (pkgs.writeText name')
);
};
}

View File

@@ -0,0 +1,47 @@
# Tests in: ../../../../tests/modular-service-etc/test.nix
# Non-modular context provided by the modular services integration.
{ pkgs }:
# Configuration data support for portable services
# This module provides configData for services, enabling configuration reloading
# without terminating and restarting the service process.
{
lib,
...
}:
let
inherit (lib) mkOption types;
inherit (lib.modules) importApply;
in
{
options = {
configData = mkOption {
default = { };
example = lib.literalExpression ''
{
"server.conf" = {
text = '''
port = 8080
workers = 4
''';
};
"ssl/cert.pem" = {
source = ./cert.pem;
};
}
'';
description = ''
Configuration data files for the service
These files are made available to the service and can be updated without restarting the service process, enabling configuration reloading.
The service manager implementation determines how these files are exposed to the service (e.g., via a specific directory path).
This path is available in the `path` sub-option for each `configData.<name>` entry.
This is particularly useful for services that support configuration reloading via signals (e.g., SIGHUP) or which pick up changes automatically, so that no downtime is required in order to reload the service.
'';
type = types.lazyAttrsOf (types.submodule (importApply ./config-data-item.nix pkgs));
};
};
}

View File

@@ -0,0 +1,77 @@
{ lib, ... }:
let
inherit (lib)
concatLists
mapAttrsToList
showOption
types
;
in
rec {
flattenMapServicesConfigToList =
f: loc: config:
f loc config
++ concatLists (
mapAttrsToList (
k: v:
flattenMapServicesConfigToList f (
loc
++ [
"services"
k
]
) v
) config.services
);
getWarnings = flattenMapServicesConfigToList (
loc: config: map (msg: "in ${showOption loc}: ${msg}") config.warnings
);
getAssertions = flattenMapServicesConfigToList (
loc: config:
map (ass: {
message = "in ${showOption loc}: ${ass.message}";
assertion = ass.assertion;
}) config.assertions
);
/**
This is the entrypoint for the portable part of modular services.
It provides the various options that are consumed by service manager implementations.
# Inputs
`serviceManagerPkgs`: A Nixpkgs instance which will be used for built-in logic such as converting `configData.<path>.text` to a store path.
`extraRootModules`: Modules to be loaded into the "root" service submodule, but not into its sub-`services`. That's the modules' own responsibility.
`extraRootSpecialArgs`: Fixed module arguments that are provided in a similar manner to `extraRootModules`.
# Output
An attribute set.
`serviceSubmodule`: a Module System option type which is a `submodule` with the portable modules and this function's inputs loaded into it.
*/
configure =
{
serviceManagerPkgs,
extraRootModules ? [ ],
extraRootSpecialArgs ? { },
}:
let
modules = [
(lib.modules.importApply ./service.nix { pkgs = serviceManagerPkgs; })
];
serviceSubmodule = types.submoduleWith {
class = "service";
modules = modules ++ extraRootModules;
specialArgs = extraRootSpecialArgs;
};
in
{
inherit serviceSubmodule;
};
}

View File

@@ -0,0 +1,54 @@
# Non-module arguments
# These are separate from the module arguments to avoid implicit dependencies.
# This makes service modules self-contains, allowing mixing of Nixpkgs versions.
{ pkgs }:
# The module
{
lib,
...
}:
let
inherit (lib) mkOption types;
pathOrStr = types.coercedTo types.path (x: "${x}") types.str;
in
{
# https://nixos.org/manual/nixos/unstable/#modular-services
_class = "service";
imports = [
../../../../../modules/generic/meta-maintainers.nix
../../../misc/assertions.nix
(lib.modules.importApply ./config-data.nix { inherit pkgs; })
];
options = {
services = mkOption {
type = types.attrsOf (
types.submoduleWith {
modules = [
(lib.modules.importApply ./service.nix { inherit pkgs; })
];
}
);
description = ''
A collection of [modular services](https://nixos.org/manual/nixos/unstable/#modular-services) that are configured in one go.
You could consider the sub-service relationship to be an ownership relation.
It **does not** automatically create any other relationship between services (e.g. systemd slices), unless perhaps such a behavior is explicitly defined and enabled in another option.
'';
default = { };
visible = "shallow";
};
process = {
argv = lib.mkOption {
type = types.listOf pathOrStr;
example = lib.literalExpression ''[ (lib.getExe config.package) "--nobackground" ]'';
description = ''
Command filename and arguments for starting this service.
This is a raw command-line that should not contain any shell escaping.
If expansion of environmental variables is required then use
a shell script or `importas` from `pkgs.execline`.
'';
};
};
};
}

View File

@@ -0,0 +1,191 @@
# Run:
# nix-instantiate --eval nixos/modules/system/service/portable/test.nix
let
lib = import ../../../../../lib;
inherit (lib) mkOption types;
portable-lib = import ./lib.nix { inherit lib; };
configured = portable-lib.configure {
serviceManagerPkgs = throw "do not use pkgs in this test";
extraRootModules = [ ];
extraRootSpecialArgs = { };
};
dummyPkg =
name:
derivation {
system = "dummy";
name = name;
builder = "/bin/false";
};
exampleConfig = {
_file = "${__curPos.file}:${toString __curPos.line}";
services = {
service1 = {
process = {
argv = [
"/usr/bin/echo" # *giggles*
"hello"
];
};
assertions = [
{
assertion = false;
message = "you can't enable this for that reason";
}
];
warnings = [
"The `foo' service is deprecated and will go away soon!"
];
};
service2 = {
process = {
# No meta.mainProgram, because it's supposedly an executable script _file_,
# not a directory with a bin directory containing the main program.
argv = [
(dummyPkg "cowsay.sh")
"world"
];
};
};
service3 = {
process = {
argv = [ "/bin/false" ];
};
services.exclacow = {
process = {
argv = [
(lib.getExe (
dummyPkg "cowsay-ng"
// {
meta.mainProgram = "cowsay";
}
))
"!"
];
};
assertions = [
{
assertion = false;
message = "you can't enable this for such reason";
}
];
warnings = [
"The `bar' service is deprecated and will go away soon!"
];
};
};
};
};
exampleEval = lib.evalModules {
modules = [
{
options.services = mkOption {
type = types.attrsOf configured.serviceSubmodule;
};
}
exampleConfig
];
};
filterEval =
config:
lib.optionalAttrs (config ? process) {
inherit (config) assertions warnings process;
}
// {
services = lib.mapAttrs (k: filterEval) config.services;
};
test =
assert
filterEval exampleEval.config == {
services = {
service1 = {
process = {
argv = [
"/usr/bin/echo"
"hello"
];
};
services = { };
assertions = [
{
assertion = false;
message = "you can't enable this for that reason";
}
];
warnings = [
"The `foo' service is deprecated and will go away soon!"
];
};
service2 = {
process = {
argv = [
"${dummyPkg "cowsay.sh"}"
"world"
];
};
services = { };
assertions = [ ];
warnings = [ ];
};
service3 = {
process = {
argv = [ "/bin/false" ];
};
services.exclacow = {
process = {
argv = [
"${dummyPkg "cowsay-ng"}/bin/cowsay"
"!"
];
};
services = { };
assertions = [
{
assertion = false;
message = "you can't enable this for such reason";
}
];
warnings = [ "The `bar' service is deprecated and will go away soon!" ];
};
assertions = [ ];
warnings = [ ];
};
};
};
assert
portable-lib.getWarnings [ "service1" ] exampleEval.config.services.service1 == [
"in service1: The `foo' service is deprecated and will go away soon!"
];
assert
portable-lib.getAssertions [ "service1" ] exampleEval.config.services.service1 == [
{
message = "in service1: you can't enable this for that reason";
assertion = false;
}
];
assert
portable-lib.getWarnings [ "service3" ] exampleEval.config.services.service3 == [
"in service3.services.exclacow: The `bar' service is deprecated and will go away soon!"
];
assert
portable-lib.getAssertions [ "service3" ] exampleEval.config.services.service3 == [
{
message = "in service3.services.exclacow: you can't enable this for such reason";
assertion = false;
}
];
"ok";
in
test

View File

@@ -0,0 +1,39 @@
# Tests in: ../../tests/modular-service-etc/test.nix
# This module sets the path for configData entries in systemd services
let
setPathsModule =
prefix:
{ lib, name, ... }:
let
inherit (lib) mkOption types;
servicePrefix = "${prefix}${name}";
in
{
_class = "service";
options = {
# Extend portable configData option
configData = mkOption {
type = types.lazyAttrsOf (
types.submodule (
{ config, ... }:
{
config = {
path = lib.mkDefault "/etc/system-services/${servicePrefix}/${config.name}";
};
}
)
);
};
services = mkOption {
type = types.attrsOf (
types.submoduleWith {
modules = [
(setPathsModule "${servicePrefix}-")
];
}
);
};
};
};
in
setPathsModule ""

View File

@@ -0,0 +1,123 @@
{
lib,
config,
systemdPackage,
...
}:
let
inherit (lib)
concatMapStringsSep
isDerivation
isInt
isFloat
isPath
isString
mkOption
replaceStrings
types
;
inherit (builtins) toJSON;
# Local copy of systemd exec argument escaping function.
# TODO: This could perhaps be deduplicated, but it is unclear where it should go.
# Preferably, we don't create a hard dependency on NixOS here, so that this
# module can be reused in a non-NixOS context, such as mutaable services
# in /run/systemd/system.
# Quotes an argument for use in Exec* service lines.
# systemd accepts "-quoted strings with escape sequences, toJSON produces
# a subset of these.
# Additionally we escape % to disallow expansion of % specifiers. Any lone ;
# in the input will be turned it ";" and thus lose its special meaning.
# Every $ is escaped to $$, this makes it unnecessary to disable environment
# substitution for the directive.
escapeSystemdExecArg =
arg:
let
s =
if isPath arg then
"${arg}"
else if isString arg then
arg
else if isInt arg || isFloat arg || isDerivation arg then
toString arg
else
throw "escapeSystemdExecArg only allows strings, paths, numbers and derivations";
in
replaceStrings [ "%" "$" ] [ "%%" "$$" ] (toJSON s);
# Quotes a list of arguments into a single string for use in a Exec*
# line.
escapeSystemdExecArgs = concatMapStringsSep " " escapeSystemdExecArg;
in
{
_class = "service";
imports = [
(lib.mkAliasOptionModule [ "systemd" "service" ] [ "systemd" "services" "" ])
(lib.mkAliasOptionModule [ "systemd" "socket" ] [ "systemd" "sockets" "" ])
];
options = {
systemd.services = mkOption {
description = ''
This module configures systemd services, with the notable difference that their unit names will be prefixed with the abstract service name.
This option's value is not suitable for reading, but you can define a module here that interacts with just the unit configuration in the host system configuration.
Note that this option contains _deferred_ modules.
This means that the module has not been combined with the system configuration yet, no values can be read from this option.
What you can do instead is define a module that reads from the module arguments (such as `config`) that are available when the module is merged into the system configuration.
'';
type = types.lazyAttrsOf (
types.deferredModuleWith {
staticModules = [
# TODO: Add modules for the purpose of generating documentation?
];
}
);
default = { };
};
systemd.sockets = mkOption {
description = ''
Declares systemd socket units. Names will be prefixed by the service name / path.
See {option}`systemd.services`.
'';
type = types.lazyAttrsOf types.deferredModule;
default = { };
};
# Also import systemd logic into sub-services
# extends the portable `services` option
services = mkOption {
type = types.attrsOf (
types.submoduleWith {
class = "service";
modules = [
./service.nix
];
specialArgs = {
inherit systemdPackage;
};
}
);
# Rendered by the portable docs instead.
visible = false;
};
};
config = {
# Note that this is the systemd.services option above, not the system one.
systemd.services."" = {
# TODO description;
wantedBy = lib.mkDefault [ "multi-user.target" ];
serviceConfig = {
Type = lib.mkDefault "simple";
Restart = lib.mkDefault "always";
RestartSec = lib.mkDefault "5";
ExecStart = [
(escapeSystemdExecArgs config.process.argv)
];
};
};
};
}

View File

@@ -0,0 +1,116 @@
{
lib,
config,
options,
pkgs,
...
}:
let
inherit (lib)
concatMapAttrs
mkOption
types
concatLists
mapAttrsToList
;
portable-lib = import ../portable/lib.nix { inherit lib; };
dash =
before: after:
if after == "" then
before
else if before == "" then
after
else
"${before}-${after}";
makeNixosEtcFiles =
prefix: service:
let
# Convert configData entries to environment.etc entries
serviceConfigData = lib.mapAttrs' (name: cfg: {
name =
# cfg.path is read only and prefixed with unique service name; see ./config-data-path.nix
assert lib.hasPrefix "/etc/system-services" cfg.path;
lib.removePrefix "/etc/" cfg.path;
value = {
inherit (cfg) enable source;
};
}) (service.configData or { });
# Recursively process sub-services
subServiceConfigData = concatMapAttrs (
subServiceName: subService: makeNixosEtcFiles (dash prefix subServiceName) subService
) service.services;
in
serviceConfigData // subServiceConfigData;
makeUnits =
unitType: prefix: service:
concatMapAttrs (unitName: unitModule: {
"${dash prefix unitName}" =
{ ... }:
{
imports = [ unitModule ];
};
}) service.systemd.${unitType}
// concatMapAttrs (
subServiceName: subService: makeUnits unitType (dash prefix subServiceName) subService
) service.services;
modularServiceConfiguration = portable-lib.configure {
serviceManagerPkgs = pkgs;
extraRootModules = [
./service.nix
./config-data-path.nix
];
extraRootSpecialArgs = {
systemdPackage = config.systemd.package;
};
};
in
{
_class = "nixos";
# First half of the magic: mix systemd logic into the otherwise abstract services
options = {
system.services = mkOption {
description = ''
A collection of NixOS [modular services](https://nixos.org/manual/nixos/unstable/#modular-services) that are configured as systemd services.
'';
type = types.attrsOf modularServiceConfiguration.serviceSubmodule;
default = { };
visible = "shallow";
};
};
# Second half of the magic: siphon units that were defined in isolation to the system
config = {
assertions = concatLists (
mapAttrsToList (
name: cfg: portable-lib.getAssertions (options.system.services.loc ++ [ name ]) cfg
) config.system.services
);
warnings = concatLists (
mapAttrsToList (
name: cfg: portable-lib.getWarnings (options.system.services.loc ++ [ name ]) cfg
) config.system.services
);
systemd.services = concatMapAttrs (
serviceName: topLevelService: makeUnits "services" serviceName topLevelService
) config.system.services;
systemd.sockets = concatMapAttrs (
serviceName: topLevelService: makeUnits "sockets" serviceName topLevelService
) config.system.services;
environment.etc = concatMapAttrs (
serviceName: topLevelService: makeNixosEtcFiles serviceName topLevelService
) config.system.services;
};
}

View File

@@ -0,0 +1,92 @@
# Run:
# nix-build -A nixosTests.modularService
{
evalSystem,
runCommand,
hello,
...
}:
let
machine = evalSystem (
{ lib, ... }:
let
hello' = lib.getExe hello;
in
{
# Test input
system.services.foo = {
process = {
argv = [
hello'
"--greeting"
"hoi"
];
};
};
system.services.bar = {
process = {
argv = [
hello'
"--greeting"
"hoi"
];
};
systemd.service = {
serviceConfig.X-Bar = "lol crossbar whatever";
};
services.db = {
process = {
argv = [
hello'
"--greeting"
"Hi, I'm a database, would you believe it"
];
};
systemd.service = {
serviceConfig.RestartSec = "42";
};
};
};
# irrelevant stuff
system.stateVersion = "25.05";
fileSystems."/".device = "/test/dummy";
boot.loader.grub.enable = false;
}
);
inherit (machine.config.system.build) toplevel;
in
runCommand "test-modular-service-systemd-units"
{
passthru = {
inherit
machine
toplevel
;
};
}
''
echo ${toplevel}/etc/systemd/system/foo.service:
cat -n ${toplevel}/etc/systemd/system/foo.service
(
set -x
grep -F 'ExecStart="${hello}/bin/hello" "--greeting" "hoi"' ${toplevel}/etc/systemd/system/foo.service >/dev/null
grep -F 'ExecStart="${hello}/bin/hello" "--greeting" "hoi"' ${toplevel}/etc/systemd/system/bar.service >/dev/null
grep -F 'X-Bar=lol crossbar whatever' ${toplevel}/etc/systemd/system/bar.service >/dev/null
grep 'ExecStart="${hello}/bin/hello" "--greeting" ".*database.*"' ${toplevel}/etc/systemd/system/bar-db.service >/dev/null
grep -F 'RestartSec=42' ${toplevel}/etc/systemd/system/bar-db.service >/dev/null
[[ ! -e ${toplevel}/etc/systemd/system/foo.socket ]]
[[ ! -e ${toplevel}/etc/systemd/system/bar.socket ]]
[[ ! -e ${toplevel}/etc/systemd/system/bar-db.socket ]]
)
echo 🐬👍
touch $out
''

View File

@@ -0,0 +1,3 @@
# TBD, analogous to system.nix but for user units
{
}