You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

292 lines
12 KiB

Switch to Guile-Gcrypt. This removes (guix hash) and (guix pk-crypto), which now live as part of Guile-Gcrypt (version 0.1.0.) * guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm, tests/hash.scm, tests/pk-crypto.scm: Remove. * configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and LIBGCRYPT_LIBDIR assignments. * m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove. * README: Add Guile-Gcrypt to the dependencies; move libgcrypt as "required unless --disable-daemon". * doc/guix.texi (Requirements): Likewise. * gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm, guix/git.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm, guix/import/gnu.scm, guix/import/hackage.scm, guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm, guix/pki.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/download.scm, guix/scripts/hash.scm, guix/scripts/pack.scm, guix/scripts/publish.scm, guix/scripts/refresh.scm, guix/scripts/substitute.scm, guix/store.scm, guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm, tests/builders.scm, tests/challenge.scm, tests/cpan.scm, tests/crate.scm, tests/derivations.scm, tests/gem.scm, tests/nar.scm, tests/opam.scm, tests/pki.scm, tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm, tests/store.scm, tests/substitute.scm: Adjust imports. * gnu/system/vm.scm: Likewise. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (expression->derivation-in-linux-vm)[config]: Remove. (iso9660-image)[config]: Remove. (qemu-image)[config]: Remove. (system-docker-image)[config]: Remove. * guix/scripts/pack.scm: Adjust imports. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (self-contained-tarball)[build]: Call 'make-config.scm' without #:libgcrypt argument. (squashfs-image)[libgcrypt]: Remove. [build]: Call 'make-config.scm' without #:libgcrypt. (docker-image)[config, json]: Remove. [build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from the imported modules. * guix/self.scm (specification->package): Remove "libgcrypt", add "guile-gcrypt". (compiled-guix): Remove #:libgcrypt. [guile-gcrypt]: New variable. [dependencies]: Add it. [*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call. Add #:extensions. [*config*]: Remove #:libgcrypt from 'make-config.scm' call. (%dependency-variables): Remove %libgcrypt. (make-config.scm): Remove #:libgcrypt. * build-aux/build-self.scm (guile-gcrypt): New variable. (make-config.scm): Remove #:libgcrypt. (build-program)[fake-gcrypt-hash]: New variable. Add (gcrypt hash) to the imported modules. Adjust load path assignments. * gnu/packages/package-management.scm (guix)[propagated-inputs]: Add GUILE-GCRYPT. [arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search path.
3 years ago
Switch to Guile-Gcrypt. This removes (guix hash) and (guix pk-crypto), which now live as part of Guile-Gcrypt (version 0.1.0.) * guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm, tests/hash.scm, tests/pk-crypto.scm: Remove. * configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and LIBGCRYPT_LIBDIR assignments. * m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove. * README: Add Guile-Gcrypt to the dependencies; move libgcrypt as "required unless --disable-daemon". * doc/guix.texi (Requirements): Likewise. * gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm, guix/git.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm, guix/import/gnu.scm, guix/import/hackage.scm, guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm, guix/pki.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/download.scm, guix/scripts/hash.scm, guix/scripts/pack.scm, guix/scripts/publish.scm, guix/scripts/refresh.scm, guix/scripts/substitute.scm, guix/store.scm, guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm, tests/builders.scm, tests/challenge.scm, tests/cpan.scm, tests/crate.scm, tests/derivations.scm, tests/gem.scm, tests/nar.scm, tests/opam.scm, tests/pki.scm, tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm, tests/store.scm, tests/substitute.scm: Adjust imports. * gnu/system/vm.scm: Likewise. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (expression->derivation-in-linux-vm)[config]: Remove. (iso9660-image)[config]: Remove. (qemu-image)[config]: Remove. (system-docker-image)[config]: Remove. * guix/scripts/pack.scm: Adjust imports. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (self-contained-tarball)[build]: Call 'make-config.scm' without #:libgcrypt argument. (squashfs-image)[libgcrypt]: Remove. [build]: Call 'make-config.scm' without #:libgcrypt. (docker-image)[config, json]: Remove. [build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from the imported modules. * guix/self.scm (specification->package): Remove "libgcrypt", add "guile-gcrypt". (compiled-guix): Remove #:libgcrypt. [guile-gcrypt]: New variable. [dependencies]: Add it. [*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call. Add #:extensions. [*config*]: Remove #:libgcrypt from 'make-config.scm' call. (%dependency-variables): Remove %libgcrypt. (make-config.scm): Remove #:libgcrypt. * build-aux/build-self.scm (guile-gcrypt): New variable. (make-config.scm): Remove #:libgcrypt. (build-program)[fake-gcrypt-hash]: New variable. Add (gcrypt hash) to the imported modules. Adjust load path assignments. * gnu/packages/package-management.scm (guix)[propagated-inputs]: Add GUILE-GCRYPT. [arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search path.
3 years ago
ui: Rename '_' to 'G_'. This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
4 years ago
ui: Rename '_' to 'G_'. This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
4 years ago
  1. ;;; GNU Guix --- Functional package management for GNU
  2. ;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2018, 2019 Ludovic Courtès <ludo@gnu.org>
  3. ;;; Copyright © 2014 Mark H Weaver <mhw@netris.org>
  4. ;;;
  5. ;;; This file is part of GNU Guix.
  6. ;;;
  7. ;;; GNU Guix is free software; you can redistribute it and/or modify it
  8. ;;; under the terms of the GNU General Public License as published by
  9. ;;; the Free Software Foundation; either version 3 of the License, or (at
  10. ;;; your option) any later version.
  11. ;;;
  12. ;;; GNU Guix is distributed in the hope that it will be useful, but
  13. ;;; WITHOUT ANY WARRANTY; without even the implied warranty of
  14. ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  15. ;;; GNU General Public License for more details.
  16. ;;;
  17. ;;; You should have received a copy of the GNU General Public License
  18. ;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
  19. (define-module (guix nar)
  20. #:use-module (guix serialization)
  21. #:use-module (guix build syscalls)
  22. #:use-module ((guix build utils)
  23. #:select (delete-file-recursively with-directory-excursion))
  24. ;; XXX: Eventually we should use (guix store database) exclusively, and not
  25. ;; (guix store) since this is "daemon-side" code.
  26. #:use-module (guix store)
  27. #:use-module (guix store database)
  28. #:use-module ((guix build store-copy) #:select (store-info))
  29. #:use-module (guix i18n)
  30. #:use-module (gcrypt hash)
  31. #:use-module (guix pki)
  32. #:use-module (gcrypt pk-crypto)
  33. #:use-module (srfi srfi-1)
  34. #:use-module (srfi srfi-11)
  35. #:use-module (srfi srfi-26)
  36. #:use-module (srfi srfi-34)
  37. #:use-module (srfi srfi-35)
  38. #:export (nar-invalid-hash-error?
  39. nar-invalid-hash-error-expected
  40. nar-invalid-hash-error-actual
  41. nar-signature-error?
  42. nar-signature-error-signature
  43. restore-file-set))
  44. ;;; Comment:
  45. ;;;
  46. ;;; Read and write Nix archives, aka. ‘nar’.
  47. ;;;
  48. ;;; Code:
  49. (define-condition-type &nar-signature-error &nar-error
  50. nar-signature-error?
  51. (signature nar-signature-error-signature)) ; faulty signature or #f
  52. (define-condition-type &nar-invalid-hash-error &nar-signature-error
  53. nar-invalid-hash-error?
  54. (expected nar-invalid-hash-error-expected) ; expected hash (a bytevector)
  55. (actual nar-invalid-hash-error-actual)) ; actual hash
  56. ;;;
  57. ;;; Restoring a file set into the store.
  58. ;;;
  59. ;; The code below accesses the store directly and is meant to be run from
  60. ;; "build hooks", which cannot invoke the daemon's 'import-paths' RPC since
  61. ;; (1) the locks on the files to be restored as already held, and (2) the
  62. ;; $NIX_HELD_LOCKS hackish environment variable cannot be set.
  63. ;;
  64. ;; So we're really duplicating that functionality of the daemon (well, until
  65. ;; most of the daemon is in Scheme :-)). But note that we do use a couple of
  66. ;; RPCs for functionality not available otherwise, like 'valid-path?'.
  67. (define* (finalize-store-file source target
  68. #:key (references '()) deriver (lock? #t))
  69. "Rename SOURCE to TARGET and register TARGET as a valid store item, with
  70. REFERENCES and DERIVER. When LOCK? is true, acquire exclusive locks on TARGET
  71. before attempting to register it; otherwise, assume TARGET's locks are already
  72. held."
  73. ;; TODO: make this reusable
  74. (define (acquire-lock file)
  75. (let ((port (lock-file file)))
  76. ;; There is an inherent race condition between opening the lock file and
  77. ;; attempting to acquire the lock on it, and because we like deleting
  78. ;; these lock files when we release them, only the first successful
  79. ;; acquisition on a given lock file matters. To make it easier to tell
  80. ;; when an acquisition is and isn't the first, the first to acquire it
  81. ;; writes a deletion token (arbitrary character) prior to releasing the
  82. ;; lock.
  83. (if (zero? (stat:size (stat port)))
  84. port
  85. ;; if FILE is non-empty, that's because it contains the deletion
  86. ;; token, so we aren't the first to acquire it. So try again!
  87. (begin
  88. (close port)
  89. (acquire-lock file)))))
  90. (with-database %default-database-file db
  91. (unless (path-id db target)
  92. (let ((lock (and lock?
  93. (acquire-lock (string-append target ".lock")))))
  94. (unless (path-id db target)
  95. ;; If FILE already exists, delete it (it's invalid anyway.)
  96. (when (file-exists? target)
  97. (delete-file-recursively target))
  98. ;; Install the new TARGET.
  99. (rename-file source target)
  100. ;; Register TARGET. As a side effect, it resets the timestamps of all
  101. ;; its files, recursively, and runs a deduplication pass.
  102. (register-items db
  103. (list (store-info target deriver references))))
  104. (when lock?
  105. (delete-file (string-append target ".lock"))
  106. ;; Write the deletion token to inform anyone who acquires the lock
  107. ;; on this particular file next that they aren't the first to
  108. ;; acquire it, so they should retry.
  109. (display "d" lock)
  110. (force-output lock)
  111. (unlock-file lock))))))
  112. (define (temporary-store-file)
  113. "Return the file name of a temporary file created in the store."
  114. (let* ((template (string-append (%store-prefix) "/guix-XXXXXX"))
  115. (port (mkstemp! template)))
  116. (close-port port)
  117. template))
  118. (define-syntax-rule (with-temporary-store-file name body ...)
  119. "Evaluate BODY with NAME bound to the file name of a temporary store item
  120. protected from GC."
  121. (with-store store
  122. (let loop ((name (temporary-store-file)))
  123. ;; Add NAME to the current process' roots. (Opening this connection to
  124. ;; the daemon allows us to reuse its code that deals with the
  125. ;; per-process roots file.)
  126. (add-temp-root store name)
  127. ;; There's a window during which GC could delete NAME. Try again when
  128. ;; that happens.
  129. (if (file-exists? name)
  130. (begin
  131. (delete-file name)
  132. body ...)
  133. (loop (temporary-store-file))))))
  134. (define* (restore-one-item port
  135. #:key acl (verify-signature? #t) (lock? #t)
  136. (log-port (current-error-port)))
  137. "Restore one store item from PORT; return its file name on success."
  138. (define (assert-valid-signature signature hash file)
  139. ;; Bail out if SIGNATURE, which must be a string as produced by
  140. ;; 'canonical-sexp->string', doesn't match HASH, a bytevector containing
  141. ;; the expected hash for FILE.
  142. (let ((signature (catch 'gcry-error
  143. (lambda ()
  144. (string->canonical-sexp signature))
  145. (lambda (key proc err)
  146. (raise (condition
  147. (&message
  148. (message "signature is not a valid \
  149. s-expression"))
  150. (&nar-signature-error
  151. (file file)
  152. (signature signature) (port port))))))))
  153. (signature-case (signature hash (current-acl))
  154. (valid-signature #t)
  155. (invalid-signature
  156. (raise (condition
  157. (&message (message "invalid signature"))
  158. (&nar-signature-error
  159. (file file) (signature signature) (port port)))))
  160. (hash-mismatch
  161. (raise (condition (&message (message "invalid hash"))
  162. (&nar-invalid-hash-error
  163. (port port) (file file)
  164. (signature signature)
  165. (expected (hash-data->bytevector
  166. (signature-signed-data signature)))
  167. (actual hash)))))
  168. (unauthorized-key
  169. (raise (condition (&message (message "unauthorized public key"))
  170. (&nar-signature-error
  171. (signature signature) (file file) (port port)))))
  172. (corrupt-signature
  173. (raise (condition
  174. (&message (message "corrupt signature data"))
  175. (&nar-signature-error
  176. (signature signature) (file file) (port port))))))))
  177. (define %export-magic
  178. ;; Number used to identify genuine file set archives.
  179. #x4558494e)
  180. (define port*
  181. ;; Keep that one around, for error conditions.
  182. port)
  183. (let-values (((port get-hash)
  184. (open-sha256-input-port port)))
  185. (with-temporary-store-file temp
  186. (restore-file port temp)
  187. (let ((magic (read-int port)))
  188. (unless (= magic %export-magic)
  189. (raise (condition
  190. (&message (message "corrupt file set archive"))
  191. (&nar-read-error
  192. (port port*) (file #f) (token #f))))))
  193. (let ((file (read-store-path port))
  194. (refs (read-store-path-list port))
  195. (deriver (read-string port))
  196. (hash (get-hash))
  197. (has-sig? (= 1 (read-int port))))
  198. (format log-port
  199. (G_ "importing file or directory '~a'...~%")
  200. file)
  201. ;; The signature may contain characters that are meant to be
  202. ;; interpreted as bytes in a 'char *', so read them as a ISO-8859-1.
  203. (let ((sig (and has-sig? (read-latin1-string port))))
  204. (when verify-signature?
  205. (if sig
  206. (begin
  207. (assert-valid-signature sig hash file)
  208. (format log-port
  209. (G_ "found valid signature for '~a'~%")
  210. file)
  211. (finalize-store-file temp file
  212. #:references refs
  213. #:deriver deriver
  214. #:lock? lock?))
  215. (raise (condition
  216. (&message (message "imported file lacks \
  217. a signature"))
  218. (&nar-signature-error
  219. (port port*) (file file) (signature #f))))))
  220. file)))))
  221. (define* (restore-file-set port
  222. #:key (verify-signature? #t) (lock? #t)
  223. (log-port (current-error-port)))
  224. "Restore the file set read from PORT to the store. The format of the data
  225. on PORT must be as created by 'export-paths'---i.e., a series of Nar-formatted
  226. archives with interspersed meta-data joining them together, possibly with a
  227. digital signature at the end. Log progress to LOG-PORT. Return the list of
  228. files restored.
  229. When LOCK? is #f, assume locks for the files to be restored are already held.
  230. This is the case when the daemon calls a build hook.
  231. Note that this procedure accesses the store directly, so it's only meant to be
  232. used by the daemon's build hooks since they cannot call back to the daemon
  233. while the locks are held."
  234. (define acl
  235. (current-acl))
  236. (let loop ((n (read-long-long port))
  237. (files '()))
  238. (case n
  239. ((0)
  240. (reverse files))
  241. ((1)
  242. (let ((file
  243. (restore-one-item port
  244. #:acl acl #:verify-signature? verify-signature?
  245. #:lock? lock? #:log-port log-port)))
  246. (loop (read-long-long port)
  247. (cons file files))))
  248. (else
  249. ;; Neither 0 nor 1.
  250. (raise (condition
  251. (&message (message "invalid inter-file archive mark"))
  252. (&nar-read-error
  253. (port port) (file #f) (token #f))))))))
  254. ;;; Local Variables:
  255. ;;; eval: (put 'with-temporary-store-file 'scheme-indent-function 1)
  256. ;;; End:
  257. ;;; nar.scm ends here