When the `paths` argument is too big `symlinkJoin` will fail with:
```
while setting up the build environment: executing '/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23/bin/bash': Argument list too long
```
This is fixed by passing `paths` as a file instead of as an
environment variable.
My build server which isn't using cache.nixos.org discovered an
outdated hash in servant:
```
trying https://github.com/haskell-servant/servant/archive/v0.16.2.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 132 0 132 0 0 616 0 --:--:-- --:--:-- --:--:-- 616
100 295k 0 295k 0 0 269k 0 --:--:-- 0:00:01 --:--:-- 384k
unpacking source archive /build/v0.16.2.tar.gz
hash mismatch in fixed-output derivation '/nix/store/i6qgxlqf599wl11rd44jasgmwb78wr6c-source':
wanted: sha256:0kqglih3rv12nmkzxvalhfaaafk4b2irvv9x5xmc48i1ns71y23l
got: sha256:0xk3czk3jhqjxhy0g8r2248m8yxgvmqhgn955k92z0h7p02lfs89
```
This applies [hydra PR #432](https://github.com/NixOS/hydra/pull/432)
to the NixOS module in nixpkgs:
```
commit 4efd078977e5ea20e1104783efc324cba11690bc
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date: Sun Dec 11 15:35:38 2016 +0100
Only set buildMachinesFiles when nix.buildMachines is defined
```
The following commit from 2016 in hydra removed the `--option
build-use-substitutes` from the hydra-queue-runner service:
```
commit ee2e9f5335c8c0288c102975b506f6b275793cfe
Author: Eelco Dolstra <edolstra@gmail.com>
Date: Fri Oct 7 20:23:05 2016 +0200
Update to reflect BinaryCacheStore changes
BinaryCacheStore no longer implements buildPaths() and ensurePath(),
so we need to use copyPath() / copyClosure().
```
It would be better if the hydra module in NixOS matches the upstream
module.
This is to fix the following error in the test on aarch64-linux:
store# [ 126.911144] thanos[739]: level=error ts=2019-06-16T14:00:26.59870538Z caller=main.go:182 msg="running command failed" err="error executing compaction: first pass of downsampling failed: create dir: mkdir /var/lib/thanos-compact/downsample: no space left on device"
store# [ 126.942655] systemd[1]: thanos-compact.service: Main process exited, code=exited, status=1/FAILURE
`sourceByRegex src regexes` should include a source file if one of the
regular expressions `regexes` matches the path of that file relative
to `src`.
However to compute this relative path `sourceByRegex` uses:
```
relPath = lib.removePrefix (toString src + "/") (toString path);
```
Note that `toString path` evaluates to an absolute file somewhere
under `src` and not under `/nix/store`.
The problem is that this doesn't work if `src` is a `cleanSourceWith`
invocation as well because `toString src` will then evaluate to
`src.outPath` which will evaluate to `builtins.filterSource ...` which
evaluates to a path in `/nix/store` which is not a prefix of `path`.
The solution is to replace `src` with `origSrc` where
```
origSrc = if isFiltered then src.origSrc else src;
isFiltered = src ? _isLibCleanSourceWith;
```
Test this by executing the following from the nixpkgs repo:
```
(cat << 'EOI'
let
pkgs = import ./. {};
in pkgs.runCommand "test-sourceByRegex" {
test_sourceByRegex =
let
src1 = pkgs.lib.sourceByRegex ./. [ "^test-sourceByRegex.nix$" ];
src2 = pkgs.lib.sourceByRegex src1 [ "^test-sourceByRegex.nix$" ];
in src2 + "/test-sourceByRegex.nix";
} ''
cp $test_sourceByRegex $out
''
EOI
) > test-sourceByRegex.nix
nix-build test-sourceByRegex.nix
```
mergeInputs is now simply defined in terms of `concatLists` and
`catAttrs` instead of a more complicated `foldr`.
Note that the order of PATH has also changed. For example running the
following with nix-shell:
let
pkgs = import <nixpkgs> {};
shell1 = pkgs.mkShell {
buildInputs = [ pkgs.htop ];
};
shell2 = pkgs.mkShell {
buildInputs = [ pkgs.hello ];
};
shell3 = pkgs.mkShell {
inputsFrom = [ shell1 shell2 ];
buildInputs = [ pkgs.tree ];
};
in shell3
Results in the following PATH:
$ echo $PATH
...
/nix/store/yifq4bikf7m07160bpia7z48ciqddbfi-tree-1.8.0/bin:
/nix/store/vhxqk81234ivqw1a7j200a1c69k8mywi-htop-2.2.0/bin:
/nix/store/n9vm3m58y1n3rg3mlll17wanc9hln58k-hello-2.10/bin
...
Previously the order was:
/nix/store/n9vm3m58y1n3rg3mlll17wanc9hln58k-hello-2.10/bin
/nix/store/vhxqk81234ivqw1a7j200a1c69k8mywi-htop-2.2.0/bin:
/nix/store/yifq4bikf7m07160bpia7z48ciqddbfi-tree-1.8.0/bin:
I think the new order makes more sense because it allows to override
the PATH in the outermost mkShell.
Running the following expression with nix-shell:
let
pkgs = import <nixpkgs> {};
shell1 = pkgs.mkShell {
shellHook = ''
echo shell1
'';
};
shell2 = pkgs.mkShell {
shellHook = ''
echo shell2
'';
};
shell3 = pkgs.mkShell {
inputsFrom = [ shell1 shell2 ];
shellHook = ''
echo shell3
'';
};
in shell3
Will now results in:
shell2
shell1
shell3
Note that packages in the front of inputsFrom have precedence over
packages in the back. The outermost mkShell has precedence over all.
Bas van Dijk