Browse Source

derivations: Add #:substitutable?, distinguished from #:local-build?.

Fixes <http://bugs.gnu.org/18747>.

* guix/derivations.scm (substitutable-derivation?): Rewrite to check for
  "allowSubstitutes".
  (derivation): Add #:substitutable? parameter.
  [user+system-env-vars]: Honor it.
  (build-expression->derivation): Add #:substitutable? and honor it.
* guix/gexp.scm (gexp->derivation): Likewise.
* tests/derivations.scm ("derivation-prerequisites-to-build and substitutes,
  non-substitutable build"): Use #:substitutable? instead of #:local-build?.
  ("substitutable-derivation?", "derivation-prerequisites-to-build and
  substitutes, local build"): New tests.
* guix/download.scm (url-fetch): Adjust comment.
* guix/git-download.scm (git-fetch): Likewise.
* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): Use #:substitutable?
  instead of #:local-build?.
* doc/guix.texi (Derivations, G-Expressions): Adjust accordingly.
version-0.8.3
Ludovic Courtès 7 years ago
parent
commit
4a6aeb670f
  1. 15
      doc/guix.texi
  2. 10
      guix/build-system/gnu.scm
  3. 29
      guix/derivations.scm
  4. 4
      guix/download.scm
  5. 5
      guix/gexp.scm
  6. 1
      guix/git-download.scm
  7. 44
      tests/derivations.scm

15
doc/guix.texi

@ -2582,7 +2582,8 @@ a derivation is the @code{derivation} procedure:
@var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:inputs '()] [#:env-vars '()] @
[#:system (%current-system)] [#:references-graphs #f] @
[#:allowed-references #f] [#:leaked-env-vars #f] [#:local-build? #f]
[#:allowed-references #f] [#:leaked-env-vars #f] [#:local-build? #f] @
[#:substitutable? #t]
Build a derivation with the given arguments, and return the resulting
@code{<derivation>} object.
@ -2612,6 +2613,11 @@ When @var{local-build?} is true, declare that the derivation is not a
good candidate for offloading and should rather be built locally
(@pxref{Daemon Offload Setup}). This is the case for small derivations
where the costs of data transfers would outweigh the benefits.
When @var{substitutable?} is false, declare that substitutes of the
derivation's output should not be used (@pxref{Substitutes}). This is
useful, for instance, when building packages that capture details of the
host CPU instruction set.
@end deffn
@noindent
@ -2651,7 +2657,7 @@ is now deprecated in favor of the much nicer @code{gexp->derivation}.
[#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:env-vars '()] [#:modules '()] @
[#:references-graphs #f] [#:allowed-references #f] @
[#:local-build? #f] [#:guile-for-build #f]
[#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f]
Return a derivation that executes Scheme expression @var{exp} as a
builder for derivation @var{name}. @var{inputs} must be a list of
@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
@ -2674,7 +2680,8 @@ terminates by passing the result of @var{exp} to @code{exit}; thus, when
@code{%guile-for-build} fluid is used instead.
See the @code{derivation} procedure for the meaning of
@var{references-graphs}, @var{allowed-references}, and @var{local-build?}.
@var{references-graphs}, @var{allowed-references}, @var{local-build?},
and @var{substitutable?}.
@end deffn
@noindent
@ -3163,7 +3170,7 @@ information about monads.)
[#:module-path @var{%load-path}] @
[#:references-graphs #f] [#:allowed-references #f] @
[#:leaked-env-vars #f] @
[#:local-build? #f] [#:guile-for-build #f]
[#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f]
Return a derivation @var{name} that runs @var{exp} (a gexp) with
@var{guile-for-build} (a derivation) on @var{system}. When @var{target}
is true, it is used as the cross-compilation target triplet for packages

10
guix/build-system/gnu.scm

@ -367,10 +367,7 @@ are allowed to refer to."
#:inputs input-drvs
#:outputs outputs
#:modules imported-modules
;; XXX: Update when
;; <http://bugs.gnu.org/18747> is fixed.
#:local-build? (not substitutable?)
#:substitutable? substitutable?
#:allowed-references
(and allowed-references
@ -513,10 +510,7 @@ platform."
#:inputs (append native-drvs target-drvs)
#:outputs outputs
#:modules imported-modules
;; XXX: Update when
;; <http://bugs.gnu.org/18747> is fixed.
#:local-build? (not substitutable?)
#:substitutable? substitutable?
#:allowed-references
(and allowed-references

29
guix/derivations.scm

@ -223,10 +223,13 @@ result is the set of prerequisites of DRV not already in valid."
(("preferLocalBuild" . "1") #f)
(_ #t)))
(define substitutable-derivation?
;; Return #t if the derivation can be substituted. Currently the two are
;; synonymous, see <http://bugs.gnu.org/18747>.
offloadable-derivation?)
(define (substitutable-derivation? drv)
"Return #t if DRV can be substituted."
(match (assoc "allowSubstitutes"
(derivation-builder-environment-vars drv))
(("allowSubstitutes" . value)
(string=? value "1"))
(_ #t)))
(define (derivation-output-paths drv sub-drvs)
"Return the output paths of outputs SUB-DRVS of DRV."
@ -692,7 +695,8 @@ HASH-ALGO, of the derivation NAME. RECURSIVE? has the same meaning as for
(inputs '()) (outputs '("out"))
hash hash-algo recursive?
references-graphs allowed-references
leaked-env-vars local-build?)
leaked-env-vars local-build?
(substitutable? #t))
"Build a derivation with the given arguments, and return the resulting
<derivation> object. When HASH and HASH-ALGO are given, a
fixed-output derivation is created---i.e., one whose result is known in
@ -715,7 +719,10 @@ as \"http_proxy\" to be passed to derivations that download files.
When LOCAL-BUILD? is true, declare that the derivation is not a good candidate
for offloading and should rather be built locally. This is the case for small
derivations where the costs of data transfers would outweigh the benefits."
derivations where the costs of data transfers would outweigh the benefits.
When SUBSTITUTABLE? is false, declare that substitutes of the derivation's
output should not be used."
(define (add-output-paths drv)
;; Return DRV with an actual store path for each of its output and the
;; corresponding environment variable.
@ -753,6 +760,9 @@ derivations where the costs of data transfers would outweigh the benefits."
(let ((env-vars `(,@(if local-build?
`(("preferLocalBuild" . "1"))
'())
,@(if (not substitutable?)
`(("allowSubstitutes" . "0"))
'())
,@(if allowed-references
`(("allowedReferences"
. ,(string-join allowed-references)))
@ -1173,7 +1183,7 @@ applied."
guile-for-build
references-graphs
allowed-references
local-build?)
local-build? (substitutable? #t))
"Return a derivation that executes Scheme expression EXP as a builder
for derivation NAME. INPUTS must be a list of (NAME DRV-PATH SUB-DRV)
tuples; when SUB-DRV is omitted, \"out\" is assumed. MODULES is a list
@ -1193,7 +1203,7 @@ EXP is built using GUILE-FOR-BUILD (a derivation). When GUILE-FOR-BUILD is
omitted or is #f, the value of the `%guile-for-build' fluid is used instead.
See the `derivation' procedure for the meaning of REFERENCES-GRAPHS,
ALLOWED-REFERENCES, and LOCAL-BUILD?."
ALLOWED-REFERENCES, LOCAL-BUILD?, and SUBSTITUTABLE?."
(define guile-drv
(or guile-for-build (%guile-for-build)))
@ -1319,7 +1329,8 @@ ALLOWED-REFERENCES, and LOCAL-BUILD?."
#:outputs outputs
#:references-graphs references-graphs
#:allowed-references allowed-references
#:local-build? local-build?)))
#:local-build? local-build?
#:substitutable? substitutable?)))
;;;

4
guix/download.scm

@ -277,8 +277,8 @@ in the store."
;; In general, offloading downloads is not a good idea.
;;#:local-build? #t
;; FIXME: The above would also disable use of
;; substitutes, so comment it out; see
;; <https://bugs.gnu.org/18747>.
;; substitutes on old daemons, so comment it out;
;; see <https://bugs.gnu.org/18747>.
)))))
(define* (download-to-store store url #:optional (name (basename url))

5
guix/gexp.scm

@ -320,7 +320,7 @@ names and file names suitable for the #:allowed-references argument to
references-graphs
allowed-references
leaked-env-vars
local-build?)
local-build? (substitutable? #t))
"Return a derivation NAME that runs EXP (a gexp) with GUILE-FOR-BUILD (a
derivation) on SYSTEM. When TARGET is true, it is used as the
cross-compilation target triplet for packages referred to by EXP.
@ -439,7 +439,8 @@ The other arguments are as for 'derivation'."
#:references-graphs (and=> graphs graphs-file-names)
#:allowed-references allowed
#:leaked-env-vars leaked-env-vars
#:local-build? local-build?))))
#:local-build? local-build?
#:substitutable? substitutable?))))
(define* (gexp-inputs exp #:key native?)
"Return the input list for EXP. When NATIVE? is true, return only native

1
guix/git-download.scm

@ -90,6 +90,7 @@ HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f."
(gexp->derivation (or name "git-checkout") build
#:system system
;; FIXME: See <https://bugs.gnu.org/18747>.
;; Uncomment when fixed daemons are widely deployed.
;;#:local-build? #t
#:hash-algo hash-algo
#:hash hash

44
tests/derivations.scm

@ -188,10 +188,22 @@
(test-assert "offloadable-derivation?"
(and (offloadable-derivation? (derivation %store "foo" %bash '()))
(offloadable-derivation? ;see <http://bugs.gnu.org/18747>
(derivation %store "foo" %bash '()
#:substitutable? #f))
(not (offloadable-derivation?
(derivation %store "foo" %bash '()
#:local-build? #t)))))
(test-assert "substitutable-derivation?"
(and (substitutable-derivation? (derivation %store "foo" %bash '()))
(substitutable-derivation? ;see <http://bugs.gnu.org/18747>
(derivation %store "foo" %bash '()
#:local-build? #f))
(not (substitutable-derivation?
(derivation %store "foo" %bash '()
#:substitutable? #f)))))
(test-assert "fixed-output-derivation?"
(let* ((builder (add-text-to-store %store "my-fixed-builder.sh"
"echo -n hello > $out" '()))
@ -614,14 +626,11 @@
(null? download*)
(null? build*))))))
(test-assert "derivation-prerequisites-to-build and substitutes, local build"
(test-assert "derivation-prerequisites-to-build and substitutes, non-substitutable build"
(let* ((store (open-connection))
(drv (build-expression->derivation store "prereq-subst-local"
(drv (build-expression->derivation store "prereq-no-subst"
(random 1000)
;; XXX: Adjust once
;; <http://bugs.gnu.org/18747>
;; is fixed.
#:local-build? #t))
#:substitutable? #f))
(output (derivation->output-path drv)))
;; Make sure substitutes are usable.
@ -631,13 +640,34 @@
(let-values (((build download)
(derivation-prerequisites-to-build store drv)))
;; Despite being available as a substitute, DRV will be built locally
;; due to #:local-build?.
;; due to #:substitutable? #f.
(and (null? download)
(match build
(((? derivation-input? input))
(string=? (derivation-input-path input)
(derivation-file-name drv)))))))))
(test-assert "derivation-prerequisites-to-build and substitutes, local build"
(with-store store
(let* ((drv (build-expression->derivation store "prereq-subst-local"
(random 1000)
#:local-build? #t))
(output (derivation->output-path drv)))
;; Make sure substitutes are usable.
(set-build-options store #:use-substitutes? #t)
(with-derivation-narinfo drv
(let-values (((build download)
(derivation-prerequisites-to-build store drv)))
;; #:local-build? is not be synonymous with #:substitutable?, so we
;; must be able to substitute DRV's output.
;; See <http://bugs.gnu.org/18747>.
(and (null? build)
(match download
(((? string? item))
(string=? item (derivation->output-path drv))))))))))
(test-assert "build-expression->derivation with expression returning #f"
(let* ((builder '(begin
(mkdir %output)

Loading…
Cancel
Save