public/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/master' into hardened-stdenv

Browse Source
This commit is contained in:
Robin Gloster
2016-07-15 14:41:01 +00:00
parent 0761573507 f402c6321a
commit 5185bc1773
1603 changed files with 132258 additions and 39568 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/default.nix
+2 -2
View File
@@ -57,11 +57,11 @@ stdenv.mkDerivation {
outputFile = "./languages-frameworks/haskell.xml";
}
+ toDocbook {
inputFile = ./../pkgs/development/idris-modules/README.md;
inputFile = ../pkgs/development/idris-modules/README.md;
outputFile = "languages-frameworks/idris.xml";
}
+ toDocbook {
inputFile = ./../pkgs/development/r-modules/README.md;
inputFile = ../pkgs/development/r-modules/README.md;
outputFile = "languages-frameworks/r.xml";
}
+ ''
doc/functions.xml
+66 -70
View File
@@ -89,27 +89,27 @@ in ...</programlisting>
<title>&lt;pkg&gt;.overrideDerivation</title>
<warning>
<para>Do not use this function in Nixpkgs. Because it breaks
package abstraction and doesn’t provide error checking for
function arguments, it is only intended for ad-hoc customisation
(such as in <filename>~/.nixpkgs/config.nix</filename>).
</para>
<para>
Additionally, <varname>overrideDerivation</varname> forces an evaluation
of the Derivation which can be quite a performance penalty if there are many
overrides used.
<para>Do not use this function in Nixpkgs as it evaluates a Derivation
before modifying it, which breaks package abstraction and removes
error-checking of function arguments. In addition, this
evaluation-per-function application incurs a performance penalty,
which can become a problem if many overrides are used.
It is only intended for ad-hoc customisation, such as in
<filename>~/.nixpkgs/config.nix</filename>.
</para>
</warning>
<para>
The function <varname>overrideDerivation</varname> is usually available for all the
derivations in the nixpkgs expression (<varname>pkgs</varname>).
The function <varname>overrideDerivation</varname> creates a new derivation
based on an existing one by overriding the original's attributes with
the attribute set produced by the specified function.
This function is available on all
derivations defined using the <varname>makeOverridable</varname> function.
Most standard derivation-producing functions, such as
<varname>stdenv.mkDerivation</varname>, are defined using this
function, which means most packages in the nixpkgs expression,
<varname>pkgs</varname>, have this function.
</para>
<para>
It is used to create a new derivation by overriding the attributes of
the original derivation according to the given function.
</para>
<para>
Example usage:
@@ -125,9 +125,9 @@ in ...</programlisting>
</para>
<para>
In the above example, the name, src and patches of the derivation
will be overridden, while all other attributes will be retained from the
original derivation.
In the above example, the <varname>name</varname>, <varname>src</varname>,
and <varname>patches</varname> of the derivation will be overridden, while
all other attributes will be retained from the original derivation.
</para>
<para>
@@ -135,6 +135,20 @@ in ...</programlisting>
the original derivation.
</para>
<note>
<para>
A package's attributes are evaluated *before* being modified by
the <varname>overrideDerivation</varname> function.
For example, the <varname>name</varname> attribute reference
in <varname>url = "mirror://gnu/hello/${name}.tar.gz";</varname>
is filled-in *before* the <varname>overrideDerivation</varname> function
modifies the attribute set. This means that overriding the
<varname>name</varname> attribute, in this example, *will not* change the
value of the <varname>url</varname> attribute. Instead, we need to override
both the <varname>name</varname> *and* <varname>url</varname> attributes.
</para>
</note>
</section>
<section xml:id="sec-lib-makeOverridable">
@@ -171,42 +185,18 @@ c = lib.makeOverridable f { a = 1; b = 2; }</programlisting>
<section xml:id="sec-fhs-environments">
<title>buildFHSChrootEnv/buildFHSUserEnv</title>
<title>buildFHSUserEnv</title>
<para>
<function>buildFHSChrootEnv</function> and
<function>buildFHSUserEnv</function> provide a way to build and run
FHS-compatible lightweight sandboxes. They get their own isolated root with
binded <filename>/nix/store</filename>, so their footprint in terms of disk
<function>buildFHSUserEnv</function> provides a way to build and run
FHS-compatible lightweight sandboxes. It creates an isolated root with
bound <filename>/nix/store</filename>, so its footprint in terms of disk
space needed is quite small. This allows one to run software which is hard or
unfeasible to patch for NixOS -- 3rd-party source trees with FHS assumptions,
games distributed as tarballs, software with integrity checking and/or external
self-updated binaries.
</para>
<para>
<function>buildFHSChrootEnv</function> allows to create persistent
environments, which can be constructed, deconstructed and entered by
multiple users at once. A downside is that it requires
<literal>root</literal> access for both those who create and destroy and
those who enter it. It can be useful to create environments for daemons that
one can enter and observe.
</para>
<para>
<function>buildFHSUserEnv</function> uses Linux namespaces feature to create
self-updated binaries. It uses Linux namespaces feature to create
temporary lightweight environments which are destroyed after all child
processes exit. It does not require root access, and can be useful to create
sandboxes and wrap applications.
</para>
<para>
Those functions both rely on <function>buildFHSEnv</function>, which creates
an actual directory structure given a list of necessary packages and extra
build commands.
<function>buildFHSChrootEnv</function> and <function>buildFHSUserEnv</function>
both accept those arguments which are passed to
<function>buildFHSEnv</function>:
processes exit, without root user rights requirement. Accepted arguments are:
</para>
<variablelist>
@@ -220,14 +210,16 @@ c = lib.makeOverridable f { a = 1; b = 2; }</programlisting>
<term><literal>targetPkgs</literal></term>
<listitem><para>Packages to be installed for the main host's architecture
(i.e. x86_64 on x86_64 installations).</para></listitem>
(i.e. x86_64 on x86_64 installations). Along with libraries binaries are also
installed.</para></listitem>
</varlistentry>
<varlistentry>
<term><literal>multiPkgs</literal></term>
<listitem><para>Packages to be installed for all architectures supported by
a host (i.e. i686 and x86_64 on x86_64 installations).</para></listitem>
a host (i.e. i686 and x86_64 on x86_64 installations). Only libraries are
installed by default.</para></listitem>
</varlistentry>
<varlistentry>
@@ -240,29 +232,33 @@ c = lib.makeOverridable f { a = 1; b = 2; }</programlisting>
<varlistentry>
<term><literal>extraBuildCommandsMulti</literal></term>
<listitem><para>Like <literal>extraBuildCommandsMulti</literal>, but
<listitem><para>Like <literal>extraBuildCommands</literal>, but
executed only on multilib architectures.</para></listitem>
</varlistentry>
<varlistentry>
<term><literal>extraOutputsToInstall</literal></term>
<listitem><para>Additional derivation outputs to be linked for both
target and multi-architecture packages.</para></listitem>
</varlistentry>
<varlistentry>
<term><literal>extraInstallCommands</literal></term>
<listitem><para>Additional commands to be executed for finalizing the
derivation with runner script.</para></listitem>
</varlistentry>
<varlistentry>
<term><literal>runScript</literal></term>
<listitem><para>A command that would be executed inside the sandbox and
passed all the command line arguments. It defaults to
<literal>bash</literal>.</para></listitem>
</varlistentry>
</variablelist>
<para>
Additionally, <function>buildFHSUserEnv</function> accepts
<literal>runScript</literal> parameter, which is a command that would be
executed inside the sandbox and passed all the command line arguments. It
default to <literal>bash</literal>.
</para>
<para>
It also uses <literal>CHROOTENV_EXTRA_BINDS</literal> environment variable
for binding extra directories in the sandbox to outside places. The format of
the variable is <literal>/mnt=test-mnt:/data</literal>, where
<literal>/mnt</literal> would be mounted as <literal>/test-mnt</literal>
and <literal>/data</literal> would be mounted as <literal>/data</literal>.
<literal>extraBindMounts</literal> array argument to
<function>buildFHSUserEnv</function> function is prepended to this variable.
Latter entries take priority if defined several times -- i.e. in case of
<literal>/data=data1:/data=data2</literal> the actual bind path would be
<literal>/data2</literal>.
</para>
<para>
One can create a simple environment using a <literal>shell.nix</literal>
like that:
doc/languages-frameworks/go.xml
+100 -48
View File
@@ -5,27 +5,29 @@
<title>Go</title>
<para>The function <varname>buildGoPackage</varname> builds
standard Go packages.
standard Go programs.
</para>
<example xml:id='ex-buildGoPackage'><title>buildGoPackage</title>
<programlisting>
net = buildGoPackage rec {
name = "go.net-${rev}";
goPackagePath = "golang.org/x/net"; <co xml:id='ex-buildGoPackage-1' />
subPackages = [ "ipv4" "ipv6" ]; <co xml:id='ex-buildGoPackage-2' />
rev = "e0403b4e005";
deis = buildGoPackage rec {
name = "deis-${version}";
version = "1.13.0";
goPackagePath = "github.com/deis/deis"; <co xml:id='ex-buildGoPackage-1' />
subPackages = [ "client" ]; <co xml:id='ex-buildGoPackage-2' />
src = fetchFromGitHub {
inherit rev;
owner = "golang";
repo = "net";
sha256 = "1g7cjzw4g4301a3yqpbk8n1d4s97sfby2aysl275x04g0zh8jxqp";
owner = "deis";
repo = "deis";
rev = "v${version}";
sha256 = "1qv9lxqx7m18029lj8cw3k7jngvxs4iciwrypdy0gd2nnghc68sw";
};
goPackageAliases = [ "code.google.com/p/go.net" ]; <co xml:id='ex-buildGoPackage-3' />
propagatedBuildInputs = [ goPackages.text ]; <co xml:id='ex-buildGoPackage-4' />
buildFlags = "--tags release"; <co xml:id='ex-buildGoPackage-5' />
disabled = isGo13;<co xml:id='ex-buildGoPackage-6' />
};
goDeps = ./deps.json; <co xml:id='ex-buildGoPackage-3' />
buildFlags = "--tags release"; <co xml:id='ex-buildGoPackage-4' />
}
</programlisting>
</example>
@@ -47,50 +49,90 @@ the following arguments are of special significance to the function:
packages will be built.
</para>
<para>
In this example only <literal>code.google.com/p/go.net/ipv4</literal> and
<literal>code.google.com/p/go.net/ipv6</literal> will be built.
In this example only <literal>github.com/deis/deis/client</literal> will be built.
</para>
</callout>
<callout arearefs='ex-buildGoPackage-3'>
<para>
<varname>goPackageAliases</varname> is a list of alternative import paths
that are valid for this library.
Packages that depend on this library will automatically rename
import paths that match any of the aliases to <literal>goPackagePath</literal>.
</para>
<para>
In this example imports will be renamed from
<literal>code.google.com/p/go.net</literal> to
<literal>golang.org/x/net</literal> in every package that depend on the
<literal>go.net</literal> library.
<varname>goDeps</varname> is where the Go dependencies of a Go program are listed
in a JSON format described below.
</para>
</callout>
<callout arearefs='ex-buildGoPackage-4'>
<para>
<varname>propagatedBuildInputs</varname> is where the dependencies of a Go library are
listed. Only libraries should list <varname>propagatedBuildInputs</varname>. If a standalone
program is being built instead, use <varname>buildInputs</varname>. If a library's tests require
additional dependencies that are not propagated, they should be listed in <varname>buildInputs</varname>.
</para>
</callout>
<callout arearefs='ex-buildGoPackage-5'>
<para>
<varname>buildFlags</varname> is a list of flags passed to the go build command.
</para>
</callout>
<callout arearefs='ex-buildGoPackage-6'>
</calloutlist>
</para>
<para>The <varname>goDeps</varname> attribute should point to a JSON file that defines which Go libraries
are needed and should be included in <varname>GOPATH</varname> for <varname>buildPhase</varname>.
</para>
<example xml:id='ex-goDeps'><title>deps.json</title>
<programlisting>
[ <co xml:id='ex-goDeps-1' />
{
"goPackagePath": "gopkg.in/yaml.v2", <co xml:id='ex-goDeps-2' />
"fetch": {
"type": "git", <co xml:id='ex-goDeps-3' />
"url": "https://gopkg.in/yaml.v2",
"rev": "a83829b6f1293c91addabc89d0571c246397bbf4",
"sha256": "1m4dsmk90sbi17571h6pld44zxz7jc4lrnl4f27dpd1l8g5xvjhh"
}
},
{
"include": "../../libs.json", <co xml:id='ex-goDeps-4' />
"packages": [ <co xml:id='ex-goDeps-5' />
"github.com/docopt/docopt-go",
"golang.org/x/crypto",
]
}
]
</programlisting>
</example>
<para>
<calloutlist>
<callout arearefs='ex-goDeps-1'>
<para>
If <varname>disabled</varname> is <literal>true</literal>,
nix will refuse to build this package.
<varname>goDeps</varname> is a list of Go dependencies.
</para>
</callout>
<callout arearefs='ex-goDeps-2'>
<para>
In this example the package will not be built for go 1.3. The <literal>isGo13</literal>
is an utility function that returns <literal>true</literal> if go used to build the
package has version 1.3.x.
<varname>goPackagePath</varname> specifies Go package import path.
</para>
</callout>
<callout arearefs='ex-goDeps-3'>
<para>
<varname>fetch type</varname> that needs to be used to get package source. If <varname>git</varname>
is used there should be <varname>url</varname>, <varname>rev</varname> and <varname>sha256</varname>
defined next to it.
</para>
</callout>
<callout arearefs='ex-goDeps-4'>
<para>
<varname>include</varname> could be used to reuse <varname>goDeps</varname> between Go programs.
There is a common libs set in <varname>&lt;nixpkgs/pkgs/development/go-modules/libs.json&gt;</varname>
with pinned versions of many packages that you can reuse.
</para>
</callout>
<callout arearefs='ex-goDeps-5'>
<para>
<varname>packages</varname> enumerates all Go packages that will be imported from included file.
</para>
</callout>
@@ -99,12 +141,21 @@ the following arguments are of special significance to the function:
</para>
<para>
Reusable Go libraries may be found in the <varname>goPackages</varname> set. You can test
build a Go package as follows:
<varname>buildGoPackage</varname> produces <xref linkend='chap-multiple-output' xrefstyle="select: title" />
where <varname>bin</varname> includes program binaries. You can test build a Go binary as follows:
<screen>
$ nix-build -A goPackages.net
</screen>
<screen>
$ nix-build -A deis.bin
</screen>
or build all outputs with:
<screen>
$ nix-build -A deis.all
</screen>
<varname>bin</varname> output will be installed by default with <varname>nix-env -i</varname>
or <varname>systemPackages</varname>.
</para>
@@ -119,6 +170,7 @@ done
</screen>
</para>
<para>To extract dependency information from a Go package in automated way use <link xlink:href="https://github.com/kamilchm/go2nix">go2nix</link>.</para>
<para>To extract dependency information from a Go package in automated way use <link xlink:href="https://github.com/kamilchm/go2nix">go2nix</link>.
It can produce complete derivation and <varname>goDeps</varname> file for Go programs.</para>
</section>
doc/languages-frameworks/haskell.md
+18 -1
View File
@@ -378,6 +378,23 @@ special options turned on:
buildInputs = [ R zeromq zlib ];
}
You can select a particular GHC version to compile with by setting the
`ghc` attribute as an argument to `buildStackProject`. Better yet, let
Stack choose what GHC version it wants based on the snapshot specified
in `stack.yaml` (only works with Stack >= 1.1.3):
{nixpkgs ? import <nixpkgs> { }, ghc ? nixpkgs.ghc}
with nixpkgs;
let R = pkgs.R.override { enableStrictBarrier = true; };
in
haskell.lib.buildStackProject {
name = "HaskellR";
buildInputs = [ R zeromq zlib ];
inherit ghc;
}
[stack-nix-doc]: http://docs.haskellstack.org/en/stable/nix_integration.html
### How to create ad hoc environments for `nix-shell`
@@ -636,7 +653,7 @@ then you have to download and re-install `foo` and all its dependents from
scratch:
# nix-store -q --referrers /nix/store/*-haskell-text-1.2.0.4 \
| xargs -L 1 nix-store --repair-path --option binary-caches http://hydra.nixos.org
| xargs -L 1 nix-store --repair-path
If you're using additional Hydra servers other than `hydra.nixos.org`, then it
might be necessary to purge the local caches that store data from those
doc/languages-frameworks/python.md
+51
View File
@@ -532,6 +532,7 @@ All parameters from `mkDerivation` function are still supported.
* `makeWrapperArgs`: A list of strings. Arguments to be passed to `makeWrapper`, which wraps generated binaries. By default, the arguments to `makeWrapper` set `PATH` and `PYTHONPATH` environment variables before calling the binary. Additional arguments here can allow a developer to set environment variables which will be available when the binary is run. For example, `makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.
* `installFlags`: A list of strings. Arguments to be passed to `pip install`. To pass options to `python setup.py install`, use `--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"].
* `format`: Format of the source. Options are `setup` for when the source has a `setup.py` and `setuptools` is used to build a wheel, and `wheel` in case the source is already a binary wheel. The default value is `setup`.
* `catchConflicts` If `true`, abort package build if a package name appears more than once in dependency tree. Default is `true`.
#### `buildPythonApplication` function
@@ -648,6 +649,56 @@ community to help save time. No tool is preferred at the moment.
## FAQ
### How can I install a working Python environment?
As explained in the user's guide installing individual Python packages
imperatively with `nix-env -i` or declaratively in `environment.systemPackages`
is not supported. However, it is possible to install a Python environment with packages (`python.buildEnv`).
In the following examples we create an environment with Python 3.5, `numpy` and `ipython`.
As you might imagine there is one limitation here, and that's you can install
only one environment at a time. You will notice the complaints about collisions
when you try to install a second environment.
#### Environment defined in separate `.nix` file
Create a file, e.g. `build.nix`, with the following expression
```nix
with import <nixpkgs> {};
with python35Packages;
python.withPackages (ps: with ps; [ numpy ipython ])
```
and install it in your profile with
```
nix-env -if build.nix
```
Now you can use the Python interpreter, as well as the extra packages that you added to the environment.
#### Environment defined in `~/.nixpkgs/config.nix`
If you prefer to, you could also add the environment as a package override to the Nixpkgs set.
```
packageOverrides = pkgs: with pkgs; with python35Packages; {
myEnv = python.withPackages (ps: with ps; [ numpy ipython ]);
};
```
and install it in your profile with
```
nix-env -iA nixos.blogEnv
```
Note that I'm using the attribute path here.
#### Environment defined in `/etc/nixos/configuration.nix`
For the sake of completeness, here's another example how to install the environment system-wide.
```nix
environment.systemPackages = with pkgs; [
(python35Packages.python.withPackages (ps: callPackage ../packages/common-python-packages.nix { pythonPackages = ps; }))
];
```
### How to solve circular dependencies?
Consider the packages `A` and `B` that depend on each other. When packaging `B`,
doc/stdenv.xml
+18 -4
View File
@@ -1196,10 +1196,24 @@ echo @foo@
<term><function>stripHash</function>
<replaceable>path</replaceable></term>
<listitem><para>Strips the directory and hash part of a store
path, and prints (on standard output) only the name part. For
instance, <literal>stripHash
/nix/store/68afga4khv0w...-coreutils-6.12</literal> print
<literal>coreutils-6.12</literal>.</para></listitem>
path, storing the name part in the environment variable
<literal>strippedName</literal>. For example:
<programlisting>
stripHash "/nix/store/9s9r019176g7cvn2nvcw41gsp862y6b4-coreutils-8.24"
# prints coreutils-8.24
echo $strippedName
</programlisting>
If you wish to store the result in another variable, then the
following idiom may be useful:
<programlisting>
name="/nix/store/9s9r019176g7cvn2nvcw41gsp862y6b4-coreutils-8.24"
someVar=$(stripHash $name; echo $strippedName)
</programlisting>
</para></listitem>
</varlistentry>