Third time's a charm. Not sure how I thought that three different files
would download and have the same hash, but that's what the old code did.
This one should have the three distinct platform hashes.
When transplanting the expression in 845d1b28bf
I accidentally switched pants 1.3 to use the old pathspec, which breaks
at runtime. This should fix it.
In the extremely unlikely case that our store hash path ends in several
digits (as is the case right now), the Darwin ld will try to interpret
those digits as a version number and barf. To avoid that, we pass in the
SDK version explicitly to stop it from trying to figure it out from iffy
context.
(this is reapplying 6b1957d17a, which got
lost in a merge)
Including apple_sdk.sdk is generally a recipe for a bad time on LLVM 3.8
and above, since you end up with bad headers in the wrong place that hurt
the new libc++ in 3.8 and above. In this case, qt only wanted the super-
generic SDK for CUPS headers, which we can just depend on directly now.
This changes much of the make-disk-image.nix logic (and thus most NixOS
image building) to use LKL to set up the target directory structure rather
than a Linux VM. The only work we still do in a VM is less IO-heavy stuff
that while still time-consuming, is less of the overall load. The goal is
to kill more of that stuff, but that will require deeper changes to NixOS
activation scripts and switch-to-configuration.pl, and I don't want to
bite off too much at once.
The key distinction I'm drawing is that there's a component that deals
with the store of the machine being built, and another component for
the store building it. The inner part of it assumes nothing from the
builder (doesn't need chroot or root powers) so it can run comfortably
inside a Nix build, as well as nixos-rebuild. I have some upcoming work
that will use that to significantly speed up and streamline image builds
for NixOS, especially on virtualized hosts like EC2, but it's also a
reasonable speedup on native hosts.
Not sure if the things I made optional on Darwin are really worth keeping
at all, but I don't have a Linux box handy so I'm minimizing potential
damage.
Including apple_sdk.sdk is generally a recipe for a bad time on LLVM 3.8
and above, since you end up with bad headers in the wrong place that hurt
the new libc++ in 3.8 and above. In this case, qt only wanted the super-
generic SDK for CUPS headers, which we can just depend on directly now.
This also fixes a missing header in the SDK that rtags needs to work
properly. The underlying cause is that C++ headers got shuffled around a
lot in libc++ 3.8 (I believe) and became more standards-compliant, which
led to a lot of C-compatible passthrough header files being added to it
like math.h, which defines some C++-compatible versions of standard
functions like signbit, while #include_next'ing the system math.h. In
this case, including the SDK was stuffing another math.h in front of the
libc++ shim, which led to all sorts of mysterious failures.
This is in preparation for the LLVM 4 upgrade (which gets more strict
about e.g., return false in xcbuild itself) and also for using xcbuild
more extensively in the Darwin stdenv bootstrap process, which is why I
killed the unnecessary gcc dependency in the toolchain. llvm-cov pretends
to be gcov anyway, so we're fine.
This sort of thing is going to get revamped to be less hackish soon but
for now I want it to work. In this particular case, libc++ 4 (and maybe
earlier) gets very upset if we're imprecise about our const markers, and
I guess libauto was careless. This fixes it (PtrPtrMap) to be correct.
This should eliminate the branched logic for gfortran on Darwin, as well
as preventing accidental inclusion of impure paths in gcc and gfortran
builds.
Splits outputs in clang like we do in 3.8 and 4.0 to avoid runtime
dependency on Python in the main derivation.
I also disable TSAN on Darwin to maintain consistency with 4.0, which
disables it because it forces an unfree dependency in the stdenv.
Split outputs because there's no point in keeping a reference to Python
and it causes trouble during the Darwin stdenv bootstrap. There's also
an unnecessary dependency on LLVM in libc++ which causes us to rebuild
LLVM several more times than necessary during bootstrap, and an awkward
dependency on XPC in the TSAN that we turn off. This is in preparation
for using LLVM 4 in the Darwin stdenv and by default across nixpkgs.
We should really move to a source build of openjdk on Darwin, but that's
tricky so for now we continue to hack around it and do stuff like this,
even though crypto export polices are stupid and make no sense. No, I'm
not bitter at all that I need to do this 😩
This is a slightly less ambitious version of the (now reverted) commit
377cef8d16, which had a bunch of issues
that I don't have time to resolve right now.
Linux behavior should be largely unchanged but we now build minikube
ourselves. Unfortunately localkube is still tricky to build so I pull in
a binary version from upstream.
1) The forking behavior of `buildbot start` is temporarily broken for
mysterious reasons that I'm still looking into
2) Let systemd do the forking: no point in using two different process
startup wait loops
The nixbld group belongs to nix-daemon and you really don't want to be
in it. If you are in it, nix-daemon will kill your processes when you
least expect it :)
Let's just be consistent with 8. At some point we'll get a proper build
of it from source but until then we might as well standardize on 8, which
has been out for years now and seems to work fine on Darwin.
Before I was just grabbing the immediate dependencies. I _think_ this
will do the right thing by using the pre-existing setup hook to avoid
having to compute the transitive closure myself.
Some changes to be more idiomatic and use stdenv building blocks more.
I also added a `buildbot.withPlugins` instead of the current plugins
mechanism, which forces an unnecessary rebuild of the package and reruns
all the tests. This should be equivalent and more pleasant to use in
practice.
The implicit behavior of pulling it out of the classpath seemed not
to work properly and could be thrown off by other things on the
classpath also providing the properties file. This guarantees that
our settings stick.
We now make it happen later in the boot process so that multi-user
has already activated, so as to not run afoul of the logic in
switch-to-configuration.pl. It's not my favorite solution, but at
least it works. Also added a check to the VM test to catch the failure
so we don't break in future.
Fixes#23121
This makes make-disk-image.nix slightly more consistent with other image
builders we have. Unfortunately I duplicated some code in doing so, but
this is temporary duplication on the path to consolidating everything.
See https://github.com/NixOS/nixpkgs/issues/23052 for more details on that.
I'm also exposing the option in the amazon-image.nix maintainer module.
The initialization code is now a systemd service that explicitly
waits for network-online, so the occasional failure I was seeing
because the `nixos-rebuild` couldn't get anything from the binary
cache should stop. I hope!
Dan Peebles