diff options
author | Collin J. Doering | 2024-12-11 18:23:33 -0500 |
---|---|---|
committer | Collin J. Doering | 2024-12-11 18:23:33 -0500 |
commit | c56b24f422d5d5ea36241b897137787c932e3bdc (patch) | |
tree | 092ab7a47c22fbcedce1d1513a2d2009c1cec5fd | |
parent | 60b96a6acd3870e48f8f60b8bb95cd806fefe406 (diff) | |
download | guix-north-america-c56b24f422d5d5ea36241b897137787c932e3bdc.tar.gz |
Add to draft blog article about guix-na and its setup
-rw-r--r-- | docs/blog-Tennnessee-build-farm.org | 62 |
1 files changed, 50 insertions, 12 deletions
diff --git a/docs/blog-Tennnessee-build-farm.org b/docs/blog-Tennnessee-build-farm.org index 8b3ede9..0163377 100644 --- a/docs/blog-Tennnessee-build-farm.org +++ b/docs/blog-Tennnessee-build-farm.org @@ -1,12 +1,12 @@ #+TITLE: Setup of a simple Guix build farm and substitute server #+AUTHOR: Collin J. Doering -A few months ago [[https://lists.gnu.org/archive/html/guix-devel/2024-07/msg00033.html][I announced on the guix mailing list]] that there was a new North American -based Guix substitute server and build farm, cuirass.genenetwork.org. This article provides -further information about how the build farm and substitute server was setup, and how you can -do so for yourself or your organization. Having more Guix substitutes servers available -improves build diversity (which can be checked with [[https://guix.gnu.org/manual/en/html_node/Invoking-guix-challenge.html][guix challenge]]), as well as substitute -availability and improved response times due to server locality. +Earlier this year [[https://lists.gnu.org/archive/html/guix-devel/2024-07/msg00033.html][I announced on the guix mailing list]] that a new North American based Guix +substitute server and build farm, cuirass.genenetwork.org was available for general use. This +article provides further information about how the build farm and substitute server was +setup, and how you can do so for yourself or your organization. Having more Guix substitutes +servers available improves build diversity (which can be checked with [[https://guix.gnu.org/manual/en/html_node/Invoking-guix-challenge.html][guix challenge]]), as +well as substitute availability and improved response times due to server locality. * TODO note inspiration, and in cases direct copy from https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/berlin.scm @@ -19,10 +19,10 @@ necessary functionality: - [[https://guix.gnu.org/cuirass/][Cuirass]] :: Watches the the guix channel repository for changes, and manages building of derivations, packages, etc.. - [[https://guix.gnu.org/manual/en/html_node/Invoking-guix-publish.html][guix-publish]] :: Provides substitute archives for consumption by users (indirectly via nginx - as a local proxy). + as a local reverse proxy). - nginx :: Acts as a reverse proxy for Cuirass and guix-publish. - certbot :: Fetches ssl certificates so cuirass and substitutes can be served over https. -- anonip :: Anatomizes http logs to preserver user privacy. +- anonip :: Anatomizes http logs to preserve user privacy. How each of these components are setup is detailed below, component-by-component. You can see the full source-code for the Tennessee build farm at @@ -30,8 +30,17 @@ https://git.genenetwork.org/guix-north-america/. ** Cuirass - building packages +Cuirass is the GNU Guix continuous integration software, and is responsible for watching for +changes to one or more VCS repositories (usually [[https://guix.gnu.org/manual/en/html_node/Channels.html][Guix channels]]), executing build jobs for +packages that have changed, and finally, storing build results in its database. + *** Define Cuirass Specs +In order to run Cuirass via the ~cuirass-service-type~, we need to define what we +want Cuirass to build. In the case of guix-na, we want to build the GNU Guix distribution and +its packages, so we declare ~%cuirass-specs~ as a [[https://guix.gnu.org/manual/en/html_node/G_002dExpressions.html][G-Expression]] that will return a list of +[[https://guix.gnu.org/cuirass/manual/html_node/Specifications.html][cuirass specifications]] with a single entry named "guix", which does exactly that! + #+begin_src scheme (define %cuirass-specs #~(list (specification @@ -41,8 +50,22 @@ https://git.genenetwork.org/guix-north-america/. (channels %default-channels)))) #+end_src +If you have custom channels you wish to build, you would add a new ~specification~ to the +list. Cuirass can build more then just channels and their packages, it can also build images, +tarballs, a specific set of packages, a manifest, and more! See the [[https://guix.gnu.org/cuirass/manual/html_node/Specifications.html][Cuirass specification]] +documentation for more details. + +Its worth noting that if you're following along by reviewing the full source code of the +Tennessee build farm that there is an additional specification listed (for guix-na itself!). +More details on that in [[*Guix Configuration as a Channel][Guix Configuration as a Channel]]. + *** Setup Cuirass Service +Now that we have defined what we want Cuirass to build, we need to enable its guix service +which in turn will run Cuirass. We are going to later setup nginx as a reverse proxy for +cuirass, so we'll set its host to localhost, and pass along the specifications we defined +earlier. + #+begin_src scheme (service cuirass-service-type (cuirass-configuration @@ -52,15 +75,28 @@ https://git.genenetwork.org/guix-north-america/. ** Provide Substitutes using Guix Publish +With Cuirass configured and the guix store being populated with package builds as the guix +channel changes, we now turn our attention to serving these builds as substitutes to Guix +users. This is done using [[https://guix.gnu.org/manual/en/html_node/Invoking-guix-publish.html][guix publish]], which Guix provides the [[file:~/.org/roam/20221129213953-advent_of_code.org::*Day 2][guix-publish-service-type]] in +order to configure and run. + #+begin_src scheme - (service guix-publish-service-type - (guix-publish-configuration - (port 3000) - (cache "/var/cache/guix/publish"))) + (service guix-publish-service-type + (guix-publish-configuration + (port 3000) + (cache "/var/cache/guix/publish") + (ttl (* 90 24 3600)) #+end_src ** Anonomize IPs (anonip) +Guix users care about their privacy, and though this is necessarily a requirement, +anonomizing nginx access logs using the anonip-service is implemented by all public Guix +sponsored build farms, so keeping with this privacy preserving trend, cuirass.genenetwork.org +implements the same log anonomization. + +To anonomize nginx access logs, the [[https://guix.gnu.org/manual/devel/en/html_node/Log-Rotation.html][anonip-service-type]] is configured and used. + #+begin_src scheme (define (anonip-service file) (service anonip-service-type @@ -491,6 +527,8 @@ https://git.genenetwork.org/guix-north-america/. TODO: talk about setup of Tennessee Guix Build Farm and Substitute Server specifics (eg. remote install) +** Guix Configuration as a Channel + * Conclusion TODO: ... |