Merge branch 'release-21.05' into staging-21.05

This commit is contained in:
Vladimír Čunát
2021-06-05 07:30:40 +02:00
123 changed files with 1590 additions and 738 deletions
@@ -436,6 +436,18 @@ machine.systemctl("list-jobs --no-pager", "any-user") # spawns a shell for `any-
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<methodname>shell_interact</methodname>
</term>
<listitem>
<para>
Allows you to directly interact with the guest shell.
This should only be used during test development, not in production tests.
Killing the interactive session with <literal>Ctrl-d</literal> or <literal>Ctrl-c</literal> also ends the guest session.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
+8 -8
View File
@@ -14,7 +14,7 @@
<para>
<emphasis>Stable channels</emphasis>, such as
<literal
xlink:href="https://nixos.org/channels/nixos-20.09">nixos-20.09</literal>.
xlink:href="https://nixos.org/channels/nixos-21.05">nixos-21.05</literal>.
These only get conservative bug fixes and package upgrades. For instance,
a channel update may cause the Linux kernel on your system to be upgraded
from 4.19.34 to 4.19.38 (a minor bug fix), but not from
@@ -38,7 +38,7 @@
<para>
<emphasis>Small channels</emphasis>, such as
<literal
xlink:href="https://nixos.org/channels/nixos-20.09-small">nixos-20.09-small</literal>
xlink:href="https://nixos.org/channels/nixos-21.05-small">nixos-21.05-small</literal>
or
<literal
xlink:href="https://nixos.org/channels/nixos-unstable-small">nixos-unstable-small</literal>.
@@ -63,8 +63,8 @@
<para>
When you first install NixOS, youre automatically subscribed to the NixOS
channel that corresponds to your installation source. For instance, if you
installed from a 20.09 ISO, you will be subscribed to the
<literal>nixos-20.09</literal> channel. To see which NixOS channel youre
installed from a 21.05 ISO, you will be subscribed to the
<literal>nixos-21.05</literal> channel. To see which NixOS channel youre
subscribed to, run the following as root:
<screen>
<prompt># </prompt>nix-channel --list | grep nixos
@@ -75,13 +75,13 @@ nixos https://nixos.org/channels/nixos-unstable
<prompt># </prompt>nix-channel --add https://nixos.org/channels/<replaceable>channel-name</replaceable> nixos
</screen>
(Be sure to include the <literal>nixos</literal> parameter at the end.) For
instance, to use the NixOS 20.09 stable channel:
instance, to use the NixOS 21.05 stable channel:
<screen>
<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09 nixos
<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-21.05 nixos
</screen>
If you have a server, you may want to use the “small” channel instead:
<screen>
<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09-small nixos
<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-21.05-small nixos
</screen>
And if you want to live on the bleeding edge:
<screen>
@@ -132,7 +132,7 @@ nixos https://nixos.org/channels/nixos-unstable
kernel, initrd or kernel modules.
You can also specify a channel explicitly, e.g.
<programlisting>
<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-20.09;
<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-21.05;
</programlisting>
</para>
</section>
+190 -104
View File
@@ -3,8 +3,11 @@
xmlns:xi="http://www.w3.org/2001/XInclude"
version="5.0"
xml:id="sec-release-21.05">
<title>Release 21.05 (“Okapi”, 2021.05/??)</title>
<title>Release 21.05 (“Okapi”, 2021.05/31)</title>
<para>
Support is planned until the end of December 2021, handing over to 21.11.
</para>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
@@ -18,114 +21,81 @@
</para>
<itemizedlist>
<listitem>
<para>
Support is planned until the end of December 2021, handing over to 21.11.
Core version changes:
</para>
<itemizedlist>
<listitem>
<para>
gcc: 9.3.0 -> 10.3.0
</para>
</listitem>
<listitem>
<para>
glibc: 2.30 -> 2.32
</para>
</listitem>
<listitem>
<para>
default linux: 5.4 -> 5.10, all supported kernels available
</para>
</listitem>
<listitem>
<para>
mesa: 20.1.7 -> 21.0.1
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
Desktop Environments:
</para>
<itemizedlist>
<listitem>
<para>
GNOME: 3.36 -> 40, see its <link xlink:href="https://help.gnome.org/misc/release-notes/40.0/">release notes</link>
</para>
</listitem>
<listitem>
<para>
Plasma5: 5.18.5 -> 5.21.3
</para>
</listitem>
<listitem>
<para>
kdeApplications: 20.08.1 -> 20.12.3
</para>
</listitem>
<listitem>
<para>
cinnamon: 4.6 -> 4.8.1
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
Programming Languages and Frameworks:
</para>
<itemizedlist>
<listitem>
<para>
Python optimizations were disabled again. Builds with optimizations enabled
are not reproducible. Optimizations can now be enabled with an option.
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>The default Linux kernel was updated to the 5.10 LTS series, coming from the 5.4 LTS series.</para>
<para>The <package>linux_latest</package> kernel was updated to the 5.12 series. It currently is not officially supported for use with the zfs filesystem. If you use zfs, you should use a different kernel version (either the LTS kernel, or track a specific one). </para>
</listitem>
<listitem>
<para>GNOME desktop environment was upgraded to 40, see the release notes for <link xlink:href="https://help.gnome.org/misc/release-notes/40.0/">40.0</link> and <link xlink:href="https://help.gnome.org/misc/release-notes/3.38/">3.38</link>. The <code>gnome3</code> attribute set has been renamed to <code>gnome</code> and so have been the NixOS options.</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.gnuradio.org/">GNURadio</link> 3.8 was
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/82263">finally</link>
packaged, along with a rewrite to the Nix expressions, allowing users to
override the features upstream supports selecting to compile or not to.
Additionally, the attribute <code>gnuradio</code> and <code>gnuradio3_7</code>
now point to an externally wrapped by default derivations, that allow you to
also add `extraPythonPackages` to the Python interpreter used by GNURadio.
Missing environmental variables needed for operational GUI were also added
(<link xlink:href="https://github.com/NixOS/nixpkgs/issues/75478">#75478</link>).
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.gnuradio.org/">GNURadio</link> has a
<code>pkgs</code> attribute set, and there's a <code>gnuradio.callPackage</code>
function that extends <code>pkgs</code> with a <code>mkDerivation</code>, and a
<code>mkDerivationWith</code>, like Qt5. Now all <code>gnuradio.pkgs</code> are
defined with <code>gnuradio.callPackage</code> and some packages that depend
on gnuradio are defined with this as well.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.privoxy.org/">Privoxy</link> has been updated
to version 3.0.32 (See <link xlink:href="https://lists.privoxy.org/pipermail/privoxy-announce/2021-February/000007.html">announcement</link>).
Compared to the previous release, Privoxy has gained support for HTTPS
inspection (still experimental), Brotli decompression, several new filters
and lots of bug fixes, including security ones. In addition, the package
is now built with compression and external filters support, which were
previously disabled.
</para>
<para>
Regarding the NixOS module, new options for HTTPS inspection have been added
and <option>services.privoxy.extraConfig</option> has been replaced by the new
<xref linkend="opt-services.privoxy.settings"/>
(See <link xlink:href="https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md">RFC 0042</link>
for the motivation).
</para>
</listitem>
<listitem>
<para>
Python optimizations were disabled again. Builds with optimizations enabled
are not reproducible. Optimizations can now be enabled with an option.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://kodi.tv/">Kodi</link> has been updated to version 19.1 "Matrix". See
the <link xlink:href="https://kodi.tv/article/kodi-190-matrix-release">announcement</link> for
further details.
</para>
</listitem>
<listitem>
<para>
The <option>services.packagekit.backend</option> option has been removed as
it only supported a single setting which would always be the default.
Instead new <link
xlink:href="https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md">RFC
0042</link> compliant <xref linkend="opt-services.packagekit.settings"/>
and <xref linkend="opt-services.packagekit.vendorSettings"/> options have
been introduced.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://nginx.org">Nginx</link> has been updated to stable version 1.20.0.
Now nginx uses the zlib-ng library by default.
</para>
</listitem>
<listitem>
<para>
KDE Gear (formerly KDE Applications) is upgraded to 21.04, see its
<link xlink:href="https://kde.org/announcements/gear/21.04/">release
notes</link> for details.
</para>
<para>
The <code>kdeApplications</code> package set is now <code>kdeGear</code>,
in keeping with the new name. The old name remains for compatibility, but
it is deprecated.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://libreswan.org/">Libreswan</link> has been updated
to version 4.4. The package now includes example configurations and manual
pages by default. The NixOS module has been changed to use the upstream
systemd units and write the configuration in the <literal>/etc/ipsec.d/
</literal> directory. In addition, two new options have been added to
specify connection policies
(<xref linkend="opt-services.libreswan.policies"/>)
and disable send/receive redirects
(<xref linkend="opt-services.libreswan.disableRedirects"/>).
</para>
</listitem>
</itemizedlist>
</section>
@@ -141,6 +111,20 @@
</para>
<itemizedlist>
<listitem>
<para>
<link xlink:href="https://www.gnuradio.org/">GNURadio</link> 3.8 was
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/82263">finally</link>
packaged, along with a rewrite to the Nix expressions, allowing users to
override the features upstream supports selecting to compile or not to.
Additionally, the attribute <code>gnuradio</code> and <code>gnuradio3_7</code>
now point to an externally wrapped by default derivations, that allow you to
also add `extraPythonPackages` to the Python interpreter used by GNURadio.
Missing environmental variables needed for operational GUI were also added
(<link xlink:href="https://github.com/NixOS/nixpkgs/issues/75478">#75478</link>).
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.keycloak.org/">Keycloak</link>,
@@ -193,6 +177,10 @@
</para>
<itemizedlist>
<listitem>
<para>GNOME desktop environment was upgraded to 40, see the release notes for <link xlink:href="https://help.gnome.org/misc/release-notes/40.0/">40.0</link> and <link xlink:href="https://help.gnome.org/misc/release-notes/3.38/">3.38</link>. The <code>gnome3</code> attribute set has been renamed to <code>gnome</code> and so have been the NixOS options.</para>
</listitem>
<listitem>
<para>
If you are using <option>services.udev.extraRules</option> to assign
@@ -591,7 +579,7 @@ http://some.json-exporter.host:7979/probe?target=https://example.com/some/json/e
<programlisting>
self: super:
{
mpi = super.mpich;
mpi = super.mpich;
}
</programlisting>
</para>
@@ -795,6 +783,16 @@ environment.systemPackages = [
the deprecated <option>services.radicale.config</option> is used.
</para>
</listitem>
<listitem>
<para>
In the <option>security.acme</option> module, use of <literal>--reuse-key</literal>
parameter for Lego has been removed. It was introduced for HKPK, but this security
feature is now deprecated. It is a better security practice to rotate key pairs
instead of always keeping the same. If you need to keep this parameter, you can add
it back using <literal>extraLegoRenewFlags</literal> as an option for the
appropriate certificate.
</para>
</listitem>
</itemizedlist>
</section>
@@ -814,6 +812,85 @@ environment.systemPackages = [
for details.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.gnuradio.org/">GNURadio</link> has a
<code>pkgs</code> attribute set, and there's a <code>gnuradio.callPackage</code>
function that extends <code>pkgs</code> with a <code>mkDerivation</code>, and a
<code>mkDerivationWith</code>, like Qt5. Now all <code>gnuradio.pkgs</code> are
defined with <code>gnuradio.callPackage</code> and some packages that depend
on gnuradio are defined with this as well.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://www.privoxy.org/">Privoxy</link> has been updated
to version 3.0.32 (See <link xlink:href="https://lists.privoxy.org/pipermail/privoxy-announce/2021-February/000007.html">announcement</link>).
Compared to the previous release, Privoxy has gained support for HTTPS
inspection (still experimental), Brotli decompression, several new filters
and lots of bug fixes, including security ones. In addition, the package
is now built with compression and external filters support, which were
previously disabled.
</para>
<para>
Regarding the NixOS module, new options for HTTPS inspection have been added
and <option>services.privoxy.extraConfig</option> has been replaced by the new
<xref linkend="opt-services.privoxy.settings"/>
(See <link xlink:href="https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md">RFC 0042</link>
for the motivation).
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://kodi.tv/">Kodi</link> has been updated to version 19.1 "Matrix". See
the <link xlink:href="https://kodi.tv/article/kodi-190-matrix-release">announcement</link> for
further details.
</para>
</listitem>
<listitem>
<para>
The <option>services.packagekit.backend</option> option has been removed as
it only supported a single setting which would always be the default.
Instead new <link
xlink:href="https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md">RFC
0042</link> compliant <xref linkend="opt-services.packagekit.settings"/>
and <xref linkend="opt-services.packagekit.vendorSettings"/> options have
been introduced.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://nginx.org">Nginx</link> has been updated to stable version 1.20.0.
Now nginx uses the zlib-ng library by default.
</para>
</listitem>
<listitem>
<para>
KDE Gear (formerly KDE Applications) is upgraded to 21.04, see its
<link xlink:href="https://kde.org/announcements/gear/21.04/">release
notes</link> for details.
</para>
<para>
The <code>kdeApplications</code> package set is now <code>kdeGear</code>,
in keeping with the new name. The old name remains for compatibility, but
it is deprecated.
</para>
</listitem>
<listitem>
<para>
<link xlink:href="https://libreswan.org/">Libreswan</link> has been updated
to version 4.4. The package now includes example configurations and manual
pages by default. The NixOS module has been changed to use the upstream
systemd units and write the configuration in the <literal>/etc/ipsec.d/
</literal> directory. In addition, two new options have been added to
specify connection policies
(<xref linkend="opt-services.libreswan.policies"/>)
and disable send/receive redirects
(<xref linkend="opt-services.libreswan.disableRedirects"/>).
</para>
</listitem>
<listitem>
<para>
The Mailman NixOS module (<literal>services.mailman</literal>) has a new
@@ -975,7 +1052,8 @@ environment.systemPackages = [
PulseAudio was upgraded to 14.0, with changes to the handling of default sinks.
See its <link xlink:href="https://www.freedesktop.org/wiki/Software/PulseAudio/Notes/14.0/">release notes</link>.
</para>
</listitem>
<listitem>
<para>
GNOME users may wish to delete their <literal>~/.config/pulse</literal> due to the changes to stream routing
logic. See <link xlink:href="https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/issues/832">PulseAudio bug 832</link>
@@ -1166,6 +1244,14 @@ environment.systemPackages = [
Nixpkgs now contains <link xlink:href="https://github.com/NixOS/nixpkgs/pull/118232">automatically packaged GNOME Shell extensions</link> from the <link xlink:href="https://extensions.gnome.org/">GNOME Extensions</link> portal. You can find them, filed by their UUID, under <literal>gnome38Extensions</literal> attribute for GNOME 3.38 and under <literal>gnome40Extensions</literal> for GNOME 40. Finally, the <literal>gnomeExtensions</literal> attribute contains extensions for the latest GNOME Shell version in Nixpkgs, listed under a more human-friendly name. The unqualified attribute scope also contains manually packaged extensions. Note that the automatically packaged extensions are provided for convenience and are not checked or guaranteed to work.
</para>
</listitem>
<listitem>
<para>
Erlang/OTP versions older than R21 got dropped. We also dropped the cuter package, as it was purely an example of how to build a package.
We also dropped <literal>lfe_1_2</literal> as it could not build with R21+.
Moving forward, we expect to only support 3 yearly releases of OTP.
</para>
</listitem>
</itemizedlist>
</section>
</section>
+11
View File
@@ -21,6 +21,7 @@ import shutil
import socket
import subprocess
import sys
import telnetlib
import tempfile
import time
import traceback
@@ -455,6 +456,16 @@ class Machine:
return (status_code, output)
output += chunk
def shell_interact(self) -> None:
"""Allows you to interact with the guest shell
Should only be used during test development, not in the production test."""
self.connect()
self.log("Terminal is ready (there is no prompt):")
telnet = telnetlib.Telnet()
telnet.sock = self.shell # type: ignore
telnet.interact()
def succeed(self, *commands: str) -> str:
"""Execute each command and check that it succeeds."""
output = ""
+31 -8
View File
@@ -182,13 +182,29 @@ let
# Menu configuration
#
# Search using a "marker file"
search --set=root --file /EFI/nixos-installer-image
insmod gfxterm
insmod png
set gfxpayload=keep
set gfxmode=${concatStringsSep "," [
# GRUB will use the first valid mode listed here.
# `auto` will sometimes choose the smallest valid mode it detects.
# So instead we'll list a lot of possibly valid modes :/
#"3840x2160"
#"2560x1440"
"1920x1080"
"1366x768"
"1280x720"
"1024x768"
"800x600"
"auto"
]}
# Fonts can be loaded?
# (This font is assumed to always be provided as a fallback by NixOS)
if loadfont /EFI/boot/unicode.pf2; then
if loadfont (\$root)/EFI/boot/unicode.pf2; then
set with_fonts=true
fi
if [ "\$textmode" != "true" -a "\$with_fonts" == "true" ]; then
@@ -212,11 +228,11 @@ let
${ # When there is a theme configured, use it, otherwise use the background image.
if config.isoImage.grubTheme != null then ''
# Sets theme.
set theme=/EFI/boot/grub-theme/theme.txt
set theme=(\$root)/EFI/boot/grub-theme/theme.txt
# Load theme fonts
$(find ${config.isoImage.grubTheme} -iname '*.pf2' -printf "loadfont /EFI/boot/grub-theme/%P\n")
$(find ${config.isoImage.grubTheme} -iname '*.pf2' -printf "loadfont (\$root)/EFI/boot/grub-theme/%P\n")
'' else ''
if background_image /EFI/boot/efi-background.png; then
if background_image (\$root)/EFI/boot/efi-background.png; then
# Black background means transparent background when there
# is a background image set... This seems undocumented :(
set color_normal=black/black
@@ -239,6 +255,9 @@ let
} ''
mkdir -p $out/EFI/boot/
# Add a marker so GRUB can find the filesystem.
touch $out/EFI/nixos-installer-image
# ALWAYS required modules.
MODULES="fat iso9660 part_gpt part_msdos \
normal boot linux configfile loopback chain halt \
@@ -294,12 +313,12 @@ let
${grubMenuCfg}
hiddenentry 'Text mode' --hotkey 't' {
loadfont /EFI/boot/unicode.pf2
loadfont (\$root)/EFI/boot/unicode.pf2
set textmode=true
terminal_output gfxterm console
}
hiddenentry 'GUI mode' --hotkey 'g' {
$(find ${config.isoImage.grubTheme} -iname '*.pf2' -printf "loadfont /EFI/boot/grub-theme/%P\n")
$(find ${config.isoImage.grubTheme} -iname '*.pf2' -printf "loadfont (\$root)/EFI/boot/grub-theme/%P\n")
set textmode=false
terminal_output gfxterm
}
@@ -370,8 +389,10 @@ let
${lib.optionalString (refindBinary != null) ''
# GRUB apparently cannot do "chainloader" operations on "CD".
if [ "\$root" != "cd0" ]; then
# Force root to be the FAT partition
# Otherwise it breaks rEFInd's boot
search --set=root --no-floppy --fs-uuid 1234-5678
menuentry 'rEFInd' --class refind {
# \$root defaults to the drive the EFI is found on.
chainloader (\$root)/EFI/boot/${refindBinary}
}
fi
@@ -403,7 +424,9 @@ let
mkdir ./boot
cp -p "${config.boot.kernelPackages.kernel}/${config.system.boot.loader.kernelFile}" \
"${config.system.build.initialRamdisk}/${config.system.boot.loader.initrdFile}" ./boot/
touch --date=@0 ./EFI ./boot
# Rewrite dates for everything in the FS
find . -exec touch --date=2000-01-01 {} +
usage_size=$(du -sb --apparent-size . | tr -cd '[:digit:]')
# Make the image 110% as big as the files need to make up for FAT overhead
+1
View File
@@ -1111,6 +1111,7 @@
./virtualisation/openvswitch.nix
./virtualisation/parallels-guest.nix
./virtualisation/podman.nix
./virtualisation/podman-network-socket-ghostunnel.nix
./virtualisation/qemu-guest-agent.nix
./virtualisation/railcar.nix
./virtualisation/spice-usb-redirection.nix
+1 -1
View File
@@ -152,7 +152,7 @@ let
);
renewOpts = escapeShellArgs (
commonOpts
++ [ "renew" "--reuse-key" ]
++ [ "renew" ]
++ optionals data.ocspMustStaple [ "--must-staple" ]
++ data.extraLegoRenewFlags
);
@@ -35,13 +35,28 @@ in
{
<xref linkend="opt-services.prometheus.exporters.dovecot.enable" /> = true;
<xref linkend="opt-services.prometheus.exporters.dovecot.socketPath" /> = "/var/run/dovecot2/old-stats";
<xref linkend="opt-services.dovecot2.mailPlugins.globally.enable" /> = [ "old_stats" ];
<xref linkend="opt-services.dovecot2.extraConfig" /> = '''
mail_plugins = $mail_plugins old_stats
service old-stats {
unix_listener old-stats {
user = dovecot-exporter
group = dovecot-exporter
mode = 0660
}
fifo_listener old-stats-mail {
mode = 0660
user = dovecot
group = dovecot
}
fifo_listener old-stats-user {
mode = 0660
user = dovecot
group = dovecot
}
}
plugin {
old_stats_refresh = 30 secs
old_stats_track_cmds = yes
}
''';
}
@@ -112,6 +112,24 @@ let
'';
description = ''
List of servers that should be probed.
<emphasis>Note:</emphasis> if your mailserver has <citerefentry>
<refentrytitle>rspamd</refentrytitle><manvolnum>8</manvolnum></citerefentry> configured,
it can happen that emails from this exporter are marked as spam.
It's possible to work around the issue with a config like this:
<programlisting>
{
<link linkend="opt-services.rspamd.locals._name_.text">services.rspamd.locals."multimap.conf".text</link> = '''
ALLOWLIST_PROMETHEUS {
filter = "email:domain:tld";
type = "from";
map = "''${pkgs.writeText "allowmap" "domain.tld"}";
score = -100.0;
}
''';
}
</programlisting>
'';
};
};
@@ -5,21 +5,19 @@ with lib;
let
cfg = config.services.prometheus.exporters.rspamd;
prettyJSON = conf:
pkgs.runCommand "rspamd-exporter-config.yml" { } ''
echo '${builtins.toJSON conf}' | ${pkgs.buildPackages.jq}/bin/jq '.' > $out
'';
mkFile = conf:
pkgs.writeText "rspamd-exporter-config.yml" (builtins.toJSON conf);
generateConfig = extraLabels: {
metrics = (map (path: {
name = "rspamd_${replaceStrings [ "." " " ] [ "_" "_" ] path}";
name = "rspamd_${replaceStrings [ "[" "." " " "]" "\\" "'" ] [ "_" "_" "_" "" "" "" ] path}";
path = "{ .${path} }";
labels = extraLabels;
}) [
"actions.'add header'"
"actions.'no action'"
"actions.'rewrite subject'"
"actions.'soft reject'"
"actions['add\\ header']"
"actions['no\\ action']"
"actions['rewrite\\ subject']"
"actions['soft\\ reject']"
"actions.greylist"
"actions.reject"
"bytes_allocated"
@@ -40,18 +38,18 @@ let
]) ++ [{
name = "rspamd_statfiles";
type = "object";
path = "$.statfiles[*]";
path = "{.statfiles[*]}";
labels = recursiveUpdate {
symbol = "$.symbol";
type = "$.type";
symbol = "{.symbol}";
type = "{.type}";
} extraLabels;
values = {
revision = "$.revision";
size = "$.size";
total = "$.total";
used = "$.used";
languages = "$.languages";
users = "$.users";
revision = "{.revision}";
size = "{.size}";
total = "{.total}";
used = "{.used}";
languages = "{.languages}";
users = "{.users}";
};
}];
};
@@ -76,7 +74,7 @@ in
};
serviceOpts.serviceConfig.ExecStart = ''
${pkgs.prometheus-json-exporter}/bin/json_exporter \
--config.file ${prettyJSON (generateConfig cfg.extraLabels)} \
--config.file ${mkFile (generateConfig cfg.extraLabels)} \
--web.listen-address "${cfg.listenAddress}:${toString cfg.port}" \
${concatStringsSep " \\\n " cfg.extraFlags}
'';
+21 -1
View File
@@ -48,6 +48,23 @@ in
description = "containers.conf configuration";
};
containersConf.cniPlugins = mkOption {
type = types.listOf types.package;
defaultText = ''
[
pkgs.cni-plugins
]
'';
example = lib.literalExample ''
[
pkgs.cniPlugins.dnsname
]
'';
description = ''
CNI plugins to install on the system.
'';
};
registries = {
search = mkOption {
type = types.listOf types.str;
@@ -97,8 +114,11 @@ in
};
config = lib.mkIf cfg.enable {
virtualisation.containers.containersConf.cniPlugins = [ pkgs.cni-plugins ];
virtualisation.containers.containersConf.settings = {
network.cni_plugin_dirs = [ "${pkgs.cni-plugins}/bin/" ];
network.cni_plugin_dirs = map (p: "${lib.getBin p}/bin") cfg.containersConf.cniPlugins;
engine = {
init_path = "${pkgs.catatonit}/bin/catatonit";
} // lib.optionalAttrs cfg.ociSeccompBpfHook.enable {
@@ -0,0 +1,36 @@
{ config, lib, pkgs, ... }:
let
inherit (lib)
mkOption
mkIf
types
;
cfg = config.virtualisation.podman;
in
{
options = {
virtualisation.podman = {
defaultNetwork.dnsname.enable = mkOption {
type = types.bool;
default = false;
description = ''
Enable DNS resolution in the default podman network.
'';
};
};
};
config = {
virtualisation.containers.containersConf.cniPlugins = mkIf cfg.defaultNetwork.dnsname.enable [ pkgs.dnsname-cni ];
virtualisation.podman.defaultNetwork.extraPlugins =
lib.optional cfg.defaultNetwork.dnsname.enable {
type = "dnsname";
domainName = "dns.podman";
capabilities.aliases = true;
};
};
}
@@ -0,0 +1,34 @@
{ config, lib, pkg, ... }:
let
inherit (lib)
mkOption
types
;
cfg = config.virtualisation.podman.networkSocket;
in
{
options.virtualisation.podman.networkSocket = {
server = mkOption {
type = types.enum [ "ghostunnel" ];
};
};
config = lib.mkIf (cfg.enable && cfg.server == "ghostunnel") {
services.ghostunnel = {
enable = true;
servers."podman-socket" = {
inherit (cfg.tls) cert key cacert;
listen = "${cfg.listenAddress}:${toString cfg.port}";
target = "unix:/run/podman/podman.sock";
allowAll = lib.mkDefault true;
};
};
systemd.services.ghostunnel-server-podman-socket.serviceConfig.SupplementaryGroups = ["podman"];
};
meta.maintainers = lib.teams.podman.members ++ [ lib.maintainers.roberth ];
}
@@ -0,0 +1,91 @@
{ config, lib, pkg, ... }:
let
inherit (lib)
mkOption
types
;
cfg = config.virtualisation.podman.networkSocket;
in
{
options.virtualisation.podman.networkSocket = {
enable = mkOption {
type = types.bool;
default = false;
description = ''
Make the Podman and Docker compatibility API available over the network
with TLS client certificate authentication.
This allows Docker clients to connect with the equivalents of the Docker
CLI <code>-H</code> and <code>--tls*</code> family of options.
For certificate setup, see https://docs.docker.com/engine/security/protect-access/
This option is independent of <xref linkend="opt-virtualisation.podman.dockerSocket.enable"/>.
'';
};
server = mkOption {
type = types.enum [];
description = ''
Choice of TLS proxy server.
'';
example = "ghostunnel";
};
openFirewall = mkOption {
type = types.bool;
default = false;
description = ''
Whether to open the port in the firewall.
'';
};
tls.cacert = mkOption {
type = types.path;
description = ''
Path to CA certificate to use for client authentication.
'';
};
tls.cert = mkOption {
type = types.path;
description = ''
Path to certificate describing the server.
'';
};
tls.key = mkOption {
type = types.path;
description = ''
Path to the private key corresponding to the server certificate.
Use a string for this setting. Otherwise it will be copied to the Nix
store first, where it is readable by any system process.
'';
};
port = mkOption {
type = types.port;
default = 2376;
description = ''
TCP port number for receiving TLS connections.
'';
};
listenAddress = mkOption {
type = types.str;
default = "0.0.0.0";
description = ''
Interface address for receiving TLS connections.
'';
};
};
config = {
networking.firewall.allowedTCPPorts =
lib.optional (cfg.enable && cfg.openFirewall) cfg.port;
};
meta.maintainers = lib.teams.podman.members ++ [ lib.maintainers.roberth ];
}
+62 -3
View File
@@ -1,7 +1,8 @@
{ config, lib, pkgs, utils, ... }:
{ config, lib, pkgs, ... }:
let
cfg = config.virtualisation.podman;
toml = pkgs.formats.toml { };
json = pkgs.formats.json { };
inherit (lib) mkOption types;
@@ -22,9 +23,24 @@ let
done
'';
net-conflist = pkgs.runCommand "87-podman-bridge.conflist" {
nativeBuildInputs = [ pkgs.jq ];
extraPlugins = builtins.toJSON cfg.defaultNetwork.extraPlugins;
jqScript = ''
. + { "plugins": (.plugins + $extraPlugins) }
'';
} ''
jq <${cfg.package}/etc/cni/net.d/87-podman-bridge.conflist \
--argjson extraPlugins "$extraPlugins" \
"$jqScript" \
>$out
'';
in
{
imports = [
./podman-dnsname.nix
./podman-network-socket.nix
(lib.mkRenamedOptionModule [ "virtualisation" "podman" "libpod" ] [ "virtualisation" "containers" "containersConf" ])
];
@@ -46,6 +62,20 @@ in
'';
};
dockerSocket.enable = mkOption {
type = types.bool;
default = false;
description = ''
Make the Podman socket available in place of the Docker socket, so
Docker tools can find the Podman socket.
Podman implements the Docker API.
Users must be in the <code>podman</code> group in order to connect. As
with Docker, members of this group can gain root access.
'';
};
dockerCompat = mkOption {
type = types.bool;
default = false;
@@ -84,6 +114,13 @@ in
'';
};
defaultNetwork.extraPlugins = lib.mkOption {
type = types.listOf json.type;
default = [];
description = ''
Extra CNI plugin configurations to add to podman's default network.
'';
};
};
@@ -92,7 +129,7 @@ in
environment.systemPackages = [ cfg.package ]
++ lib.optional cfg.dockerCompat dockerCompat;
environment.etc."cni/net.d/87-podman-bridge.conflist".source = utils.copyFile "${pkgs.podman-unwrapped.src}/cni/87-podman-bridge.conflist";
environment.etc."cni/net.d/87-podman-bridge.conflist".source = net-conflist;
virtualisation.containers = {
enable = true; # Enable common /etc/containers configuration
@@ -111,14 +148,36 @@ in
};
systemd.sockets.podman.wantedBy = [ "sockets.target" ];
systemd.sockets.podman.socketConfig.SocketGroup = "podman";
systemd.tmpfiles.packages = [ cfg.package ];
systemd.tmpfiles.packages = [
# The /run/podman rule interferes with our podman group, so we remove
# it and let the systemd socket logic take care of it.
(pkgs.runCommand "podman-tmpfiles-nixos" { package = cfg.package; } ''
mkdir -p $out/lib/tmpfiles.d/
grep -v 'D! /run/podman 0700 root root' \
<$package/lib/tmpfiles.d/podman.conf \
>$out/lib/tmpfiles.d/podman.conf
'') ];
systemd.tmpfiles.rules =
lib.optionals cfg.dockerSocket.enable [
"L! /run/docker.sock - - - - /run/podman/podman.sock"
];
users.groups.podman = {};
assertions = [
{
assertion = cfg.dockerCompat -> !config.virtualisation.docker.enable;
message = "Option dockerCompat conflicts with docker";
}
{
assertion = cfg.dockerSocket.enable -> !config.virtualisation.docker.enable;
message = ''
The options virtualisation.podman.dockerSocket.enable and virtualisation.docker.enable conflict, because only one can serve the socket.
'';
}
];
}
]);
+2
View File
@@ -335,6 +335,8 @@ in
plotinus = handleTest ./plotinus.nix {};
podgrab = handleTest ./podgrab.nix {};
podman = handleTestOn ["x86_64-linux"] ./podman.nix {};
podman-dnsname = handleTestOn ["x86_64-linux"] ./podman-dnsname.nix {};
podman-tls-ghostunnel = handleTestOn ["x86_64-linux"] ./podman-tls-ghostunnel.nix {};
pomerium = handleTestOn ["x86_64-linux"] ./pomerium.nix {};
postfix = handleTest ./postfix.nix {};
postfix-raise-smtpd-tls-security-level = handleTest ./postfix-raise-smtpd-tls-security-level.nix {};
+14
View File
@@ -20,6 +20,20 @@ import ./make-test-python.nix ({ pkgs, ... }: {
docker.wait_for_unit("sockets.target")
with subtest("includeStorePath"):
with subtest("assumption"):
docker.succeed("${examples.helloOnRoot} | docker load")
docker.succeed("set -euo pipefail; docker run --rm hello | grep -i hello")
docker.succeed("docker image rm hello:latest")
with subtest("includeStorePath = false; breaks example"):
docker.succeed("${examples.helloOnRootNoStore} | docker load")
docker.fail("set -euo pipefail; docker run --rm hello | grep -i hello")
docker.succeed("docker image rm hello:latest")
with subtest("includeStorePath = false; works with mounted store"):
docker.succeed("${examples.helloOnRootNoStore} | docker load")
docker.succeed("set -euo pipefail; docker run --rm --volume ${builtins.storeDir}:${builtins.storeDir}:ro hello | grep -i hello")
docker.succeed("docker image rm hello:latest")
with subtest("Ensure Docker images use a stable date by default"):
docker.succeed(
"docker load --input='${examples.bash}'"
+42
View File
@@ -0,0 +1,42 @@
import ./make-test-python.nix (
{ pkgs, lib, ... }:
let
inherit (pkgs) writeTextDir python3 curl;
webroot = writeTextDir "index.html" "<h1>Hi</h1>";
in
{
name = "podman-dnsname";
meta = {
maintainers = with lib.maintainers; [ roberth ] ++ lib.teams.podman.members;
};
nodes = {
podman = { pkgs, ... }: {
virtualisation.podman.enable = true;
virtualisation.podman.defaultNetwork.dnsname.enable = true;
};
};
testScript = ''
podman.wait_for_unit("sockets.target")
with subtest("DNS works"): # also tests inter-container tcp routing
podman.succeed("tar cv --files-from /dev/null | podman import - scratchimg")
podman.succeed(
"podman run -d --name=webserver -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin -w ${webroot} scratchimg ${python3}/bin/python -m http.server 8000"
)
podman.succeed("podman ps | grep webserver")
podman.succeed("""
for i in `seq 0 120`; do
podman run --rm --name=client -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg ${curl}/bin/curl http://webserver:8000 >/dev/console \
&& exit 0
sleep 0.5
done
exit 1
""")
podman.succeed("podman stop webserver")
podman.succeed("podman rm webserver")
'';
}
)
+150
View File
@@ -0,0 +1,150 @@
/*
This test runs podman as a backend for the Docker CLI.
*/
import ./make-test-python.nix (
{ pkgs, lib, ... }:
let gen-ca = pkgs.writeScript "gen-ca" ''
# Create CA
PATH="${pkgs.openssl}/bin:$PATH"
openssl genrsa -out ca-key.pem 4096
openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -subj '/C=NL/ST=Zuid-Holland/L=The Hague/O=Stevige Balken en Planken B.V./OU=OpSec/CN=Certificate Authority' -out ca.pem
# Create service
openssl genrsa -out podman-key.pem 4096
openssl req -subj '/CN=podman' -sha256 -new -key podman-key.pem -out service.csr
echo subjectAltName = DNS:podman,IP:127.0.0.1 >> extfile.cnf
echo extendedKeyUsage = serverAuth >> extfile.cnf
openssl x509 -req -days 365 -sha256 -in service.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out podman-cert.pem -extfile extfile.cnf
# Create client
openssl genrsa -out client-key.pem 4096
openssl req -subj '/CN=client' -new -key client-key.pem -out client.csr
echo extendedKeyUsage = clientAuth > extfile-client.cnf
openssl x509 -req -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -extfile extfile-client.cnf
# Create CA 2
PATH="${pkgs.openssl}/bin:$PATH"
openssl genrsa -out ca-2-key.pem 4096
openssl req -new -x509 -days 365 -key ca-2-key.pem -sha256 -subj '/C=NL/ST=Zuid-Holland/L=The Hague/O=Stevige Balken en Planken B.V./OU=OpSec/CN=Certificate Authority' -out ca-2.pem
# Create client signed by CA 2
openssl genrsa -out client-2-key.pem 4096
openssl req -subj '/CN=client' -new -key client-2-key.pem -out client-2.csr
echo extendedKeyUsage = clientAuth > extfile-client.cnf
openssl x509 -req -days 365 -sha256 -in client-2.csr -CA ca-2.pem -CAkey ca-2-key.pem -CAcreateserial -out client-2-cert.pem -extfile extfile-client.cnf
'';
in
{
name = "podman-tls-ghostunnel";
meta = {
maintainers = lib.teams.podman.members ++ [ lib.maintainers.roberth ];
};
nodes = {
podman =
{ pkgs, ... }:
{
virtualisation.podman.enable = true;
virtualisation.podman.dockerSocket.enable = true;
virtualisation.podman.networkSocket = {
enable = true;
openFirewall = true;
server = "ghostunnel";
tls.cert = "/root/podman-cert.pem";
tls.key = "/root/podman-key.pem";
tls.cacert = "/root/ca.pem";
};
environment.systemPackages = [
pkgs.docker-client
];
users.users.alice = {
isNormalUser = true;
home = "/home/alice";
description = "Alice Foobar";
extraGroups = ["podman"];
};
};
client = { ... }: {
environment.systemPackages = [
# Installs the docker _client_ only
# Normally, you'd want `virtualisation.docker.enable = true;`.
pkgs.docker-client
];
environment.variables.DOCKER_HOST = "podman:2376";
environment.variables.DOCKER_TLS_VERIFY = "1";
};
};
testScript = ''
import shlex
def su_cmd(user, cmd):
cmd = shlex.quote(cmd)
return f"su {user} -l -c {cmd}"
def cmd(command):
print(f"+{command}")
r = os.system(command)
if r != 0:
raise Exception(f"Command {command} failed with exit code {r}")
start_all()
cmd("${gen-ca}")
podman.copy_from_host("ca.pem", "/root/ca.pem")
podman.copy_from_host("podman-cert.pem", "/root/podman-cert.pem")
podman.copy_from_host("podman-key.pem", "/root/podman-key.pem")
client.copy_from_host("ca.pem", "/root/.docker/ca.pem")
# client.copy_from_host("podman-cert.pem", "/root/podman-cert.pem")
client.copy_from_host("client-cert.pem", "/root/.docker/cert.pem")
client.copy_from_host("client-key.pem", "/root/.docker/key.pem")
# TODO (ghostunnel): add file watchers so the restart isn't necessary
podman.succeed("systemctl reset-failed && systemctl restart ghostunnel-server-podman-socket.service")
podman.wait_for_unit("sockets.target")
podman.wait_for_unit("ghostunnel-server-podman-socket.service")
with subtest("Create default network"):
podman.succeed("docker network create default")
with subtest("Root docker cli also works"):
podman.succeed("docker version")
with subtest("A podman member can also still use the docker cli"):
podman.succeed(su_cmd("alice", "docker version"))
with subtest("Run container remotely via docker cli"):
client.succeed("docker version")
# via socket would be nicer
podman.succeed("tar cv --files-from /dev/null | podman import - scratchimg")
client.succeed(
"docker run -d --name=sleeping -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg /bin/sleep 10"
)
client.succeed("docker ps | grep sleeping")
podman.succeed("docker ps | grep sleeping")
client.succeed("docker stop sleeping")
client.succeed("docker rm sleeping")
with subtest("Clients without cert will be denied"):
client.succeed("rm /root/.docker/{cert,key}.pem")
client.fail("docker version")
with subtest("Clients with wrong cert will be denied"):
client.copy_from_host("client-2-cert.pem", "/root/.docker/cert.pem")
client.copy_from_host("client-2-key.pem", "/root/.docker/key.pem")
client.fail("docker version")
'';
}
)
+36 -2
View File
@@ -13,10 +13,23 @@ import ./make-test-python.nix (
{
virtualisation.podman.enable = true;
# To test docker socket support
virtualisation.podman.dockerSocket.enable = true;
environment.systemPackages = [
pkgs.docker-client
];
users.users.alice = {
isNormalUser = true;
home = "/home/alice";
description = "Alice Foobar";
extraGroups = [ "podman" ];
};
users.users.mallory = {
isNormalUser = true;
home = "/home/mallory";
description = "Mallory Foobar";
};
};
@@ -26,9 +39,9 @@ import ./make-test-python.nix (
import shlex
def su_cmd(cmd):
def su_cmd(cmd, user = "alice"):
cmd = shlex.quote(cmd)
return f"su alice -l -c {cmd}"
return f"su {user} -l -c {cmd}"
podman.wait_for_unit("sockets.target")
@@ -105,6 +118,27 @@ import ./make-test-python.nix (
assert pid == "1"
pid = podman.succeed("podman run --rm --init busybox readlink /proc/self").strip()
assert pid == "2"
with subtest("A podman member can use the docker cli"):
podman.succeed(su_cmd("docker version"))
with subtest("Run container via docker cli"):
podman.succeed("docker network create default")
podman.succeed("tar cv --files-from /dev/null | podman import - scratchimg")
podman.succeed(
"docker run -d --name=sleeping -v /nix/store:/nix/store -v /run/current-system/sw/bin:/bin scratchimg /bin/sleep 10"
)
podman.succeed("docker ps | grep sleeping")
podman.succeed("podman ps | grep sleeping")
podman.succeed("docker stop sleeping")
podman.succeed("docker rm sleeping")
podman.succeed("docker network rm default")
with subtest("A podman non-member can not use the docker cli"):
podman.fail(su_cmd("docker version", user="mallory"))
# TODO: add docker-compose test
'';
}
)