You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

178 lines
4.8 KiB

* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
13 years ago
  1. #! /somewhere/perl -w
  2. use strict;
  3. use Machine;
  4. use Term::ReadLine;
  5. use IO::File;
  6. use IO::Pty;
  7. use Logger;
  8. use Cwd;
  9. use POSIX qw(_exit dup2);
  10. $SIG{PIPE} = 'IGNORE'; # because Unix domain sockets may die unexpectedly
  11. STDERR->autoflush(1);
  12. my $log = new Logger;
  13. # Start vde_switch for each network required by the test.
  14. my %vlans;
  15. foreach my $vlan (split / /, $ENV{VLANS} || "") {
  16. next if defined $vlans{$vlan};
  17. # Start vde_switch as a child process. We don't run it in daemon
  18. # mode because we want the child process to be cleaned up when we
  19. # die. Since we have to make sure that the control socket is
  20. # ready, we send a dummy command to vde_switch (via stdin) and
  21. # wait for a reply. Note that vde_switch requires stdin to be a
  22. # TTY, so we create one.
  23. $log->log("starting VDE switch for network $vlan");
  24. my $socket = Cwd::abs_path "./vde$vlan.ctl";
  25. my $pty = new IO::Pty;
  26. my ($stdoutR, $stdoutW); pipe $stdoutR, $stdoutW;
  27. my $pid = fork(); die "cannot fork" unless defined $pid;
  28. if ($pid == 0) {
  29. dup2(fileno($pty->slave), 0);
  30. dup2(fileno($stdoutW), 1);
  31. exec "vde_switch -s $socket" or _exit(1);
  32. }
  33. close $stdoutW;
  34. print $pty "version\n";
  35. readline $stdoutR or die "cannot start vde_switch";
  36. $ENV{"QEMU_VDE_SOCKET_$vlan"} = $socket;
  37. $vlans{$vlan} = $pty;
  38. die unless -e "$socket/ctl";
  39. }
  40. my %vms;
  41. my $context = "";
  42. sub createMachine {
  43. my ($args) = @_;
  44. my $vm = Machine->new({%{$args}, log => $log, redirectSerial => ($ENV{USE_SERIAL} // "0") ne "1"});
  45. $vms{$vm->name} = $vm;
  46. return $vm;
  47. }
  48. foreach my $vmScript (@ARGV) {
  49. my $vm = createMachine({startCommand => $vmScript});
  50. $context .= "my \$" . $vm->name . " = \$vms{'" . $vm->name . "'}; ";
  51. }
  52. sub startAll {
  53. $log->nest("starting all VMs", sub {
  54. $_->start foreach values %vms;
  55. });
  56. }
  57. # Wait until all VMs have terminated.
  58. sub joinAll {
  59. $log->nest("waiting for all VMs to finish", sub {
  60. $_->waitForShutdown foreach values %vms;
  61. });
  62. }
  63. # In interactive tests, this allows the non-interactive test script to
  64. # be executed conveniently.
  65. sub testScript {
  66. eval "$context $ENV{testScript};\n";
  67. warn $@ if $@;
  68. }
  69. my $nrTests = 0;
  70. my $nrSucceeded = 0;
  71. sub subtest {
  72. my ($name, $coderef) = @_;
  73. $log->nest("subtest: $name", sub {
  74. $nrTests++;
  75. eval { &$coderef };
  76. if ($@) {
  77. $log->log("error: $@", { error => 1 });
  78. } else {
  79. $nrSucceeded++;
  80. }
  81. });
  82. }
  83. sub runTests {
  84. if (defined $ENV{tests}) {
  85. $log->nest("running the VM test script", sub {
  86. eval "$context $ENV{tests}";
  87. if ($@) {
  88. $log->log("error: $@", { error => 1 });
  89. die $@;
  90. }
  91. }, { expanded => 1 });
  92. } else {
  93. my $term = Term::ReadLine->new('nixos-vm-test');
  94. $term->ReadHistory;
  95. while (defined ($_ = $term->readline("> "))) {
  96. eval "$context $_\n";
  97. warn $@ if $@;
  98. }
  99. $term->WriteHistory;
  100. }
  101. # Copy the kernel coverage data for each machine, if the kernel
  102. # has been compiled with coverage instrumentation.
  103. $log->nest("collecting coverage data", sub {
  104. foreach my $vm (values %vms) {
  105. my $gcovDir = "/sys/kernel/debug/gcov";
  106. next unless $vm->isUp();
  107. my ($status, $out) = $vm->execute("test -e $gcovDir");
  108. next if $status != 0;
  109. # Figure out where to put the *.gcda files so that the
  110. # report generator can find the corresponding kernel
  111. # sources.
  112. my $kernelDir = $vm->mustSucceed("echo \$(dirname \$(readlink -f /run/current-system/kernel))/.build/linux-*");
  113. chomp $kernelDir;
  114. my $coverageDir = "/tmp/xchg/coverage-data/$kernelDir";
  115. # Copy all the *.gcda files.
  116. $vm->execute("for d in $gcovDir/nix/store/*/.build/linux-*; do for i in \$(cd \$d && find -name '*.gcda'); do echo \$i; mkdir -p $coverageDir/\$(dirname \$i); cp -v \$d/\$i $coverageDir/\$i; done; done");
  117. }
  118. });
  119. if ($nrTests != 0) {
  120. $log->log("$nrSucceeded out of $nrTests tests succeeded",
  121. ($nrSucceeded < $nrTests ? { error => 1 } : { }));
  122. }
  123. }
  124. # Create an empty raw virtual disk with the given name and size (in
  125. # MiB).
  126. sub createDisk {
  127. my ($name, $size) = @_;
  128. system("qemu-img create -f raw $name ${size}M") == 0
  129. or die "cannot create image of size $size";
  130. }
  131. END {
  132. $log->nest("cleaning up", sub {
  133. foreach my $vm (values %vms) {
  134. if ($vm->{pid}) {
  135. $log->log("killing " . $vm->{name} . " (pid " . $vm->{pid} . ")");
  136. kill 9, $vm->{pid};
  137. }
  138. }
  139. });
  140. $log->close();
  141. }
  142. runTests;
  143. exit ($nrSucceeded < $nrTests ? 1 : 0);