Fill in parts of the guix-publish nginx section

1 files changed, 110 insertions, 39 deletions

diff --git a/docs/blog-Tennnessee-build-farm.org b/docs/blog-Tennnessee-build-farm.org
index dfe738c..22d4808 100644
--- a/docs/blog-Tennnessee-build-farm.org
+++ b/docs/blog-Tennnessee-build-farm.org
@@ -217,23 +217,42 @@ configuring Nginx tls.
 
 *** TODO Configure Nginx Location block for ~guix-publish~
 
-Lets define a function that given a url, produces the appropriate nginx location blocks to
-enable guix-publish running on some provided URL.
-
-, starting from the definition below that will return a list
-of ~nginx-location-configuration~ blocks
+Lets define a function that given a url, produces a list of appropriate nginx location blocks
+to enable guix-publish running on some provided URL.
 
 #+begin_src scheme
   (define (publish-locations url)
     "Return the nginx location blocks for 'guix publish' running on URL."
-    (list (nginx-location-configuration
-           ...) ...)
+    (list (nginx-location-configuration ...) ...)
 #+end_src
 
-Lets look and explain the purpose of each location-configuration.
+Starting from the definition above, lets fill in and explain the purpose of each
+location-configuration in the list that will be returned from our function.
 
 - ~/nix-cache-info~
 
+  ~guix-publish~ provides a route ~/nix-cache-info~ which returns text/plain content of
+  key/value pairs, used by cooperating clients (which is normally the case, when using the
+  ~guix ...~ command). Given away by the reference to [[https://nixos.org/][Nix]] in its name, this route (and some
+  aspects of the design of guix-publish) are inspired by nix-serve, and the [[https://nixos.wiki/wiki/Binary_Cache][Nix Binary Cache]].
+
+  At the time of writing this article, the following is returned from
+  cuirass.genenetwork.org/nix-cache-info, and will be returned from every guix-published
+  based substitute server.
+
+  #+begin_src text
+    StoreDir: /gnu/store
+    WantMassQuery: 0
+    Priority: 100
+  #+end_src
+
+  FIXME: Its interesting to note that outside of the expected difference of ~StoreDir~
+  varying on Nix and Guix (on nix its ~/nix/store~), ~WantMassQuery~ and ~Priority~ are both
+  set to static values for Nix and Guix, but they vary!
+
+  Now that we have some more context on the route, here is the nginx-location-configuration
+  we will return to proxy requests appropriately.
+
   #+begin_src scheme
     (nginx-location-configuration
      (uri "/nix-cache-info")
@@ -254,44 +273,28 @@ Lets look and explain the purpose of each location-configuration.
        "proxy_ignore_headers Set-Cookie;")))
   #+end_src
 
-- ~/nar/~
+- ~~ \\.narinfo$~
 
-  #+begin_src scheme
-    (nginx-location-configuration
-     (uri "/nar/")
-     (body
-      (list
-       (string-append "proxy_pass " url ";")
-       "client_body_buffer_size 256k;"
+  [[https://nix.dev/manual/nix/2.22/protocols/nix-archive][NAR (Nix Archive Format)]] is the format used by Guix for cached substitutes. To get a sense
+  for how substitutes are downloaded from a guix-publish substitute server, lets take a
+  moment to do so manually for the [[https://packages.guix.gnu.org/packages/hello/2.12.1/][hello]] package.
 
-       ;; Be more tolerant of delays when fetching a nar.
-       "proxy_read_timeout 60s;"
-       "proxy_send_timeout 60s;"
+  First, lets find the store path of the package (but without actually building it).
 
-       ;; Enable caching for nar files, to avoid reconstructing and
-       ;; recompressing archives.
-       "proxy_cache nar;"
-       "proxy_cache_valid 200 30d;" ; cache hits for 1 month
-       "proxy_cache_valid 504 3m;"  ; timeout, when hydra.gnu.org is overloaded
-       "proxy_cache_valid any 1h;"  ; cache misses/others for 1h.
-
-       "proxy_ignore_client_abort on;"
+  #+begin_src shell :results output
+    guix build --dry-run hello
+  #+end_src
 
-       ;; Nars are already compressed.
-       "gzip off;"
+  #+RESULTS:
+  : /gnu/store/8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1
 
-       ;; We need to hide and ignore the Set-Cookie header to enable
-       ;; caching.
-       "proxy_hide_header    Set-Cookie;"
-       "proxy_ignore_headers Set-Cookie;"
+  The result is composed of a few parts:
 
-       ;; Provide a 'content-length' header so that 'guix
-       ;; substitute-binary' knows upfront how much it is downloading.
-       ;; "add_header Content-Length $body_bytes_sent;"
-       )))
-  #+end_src
+  - ~/gnu/store/~ :: the guix store path
+  - ~8bjy9g0cssjrw9ljz2r8ww1sma95isfj~ :: a hash uniquely identifying the store item
+  - ~-hello-2.12.1~ :: the package-name and version, separated by dashes
 
-- ~~ \\.narinfo$~
+  We now have enough context to define our route matching anything that ends in ~.narinfo~.
 
   #+begin_src scheme
     (nginx-location-configuration
@@ -332,8 +335,76 @@ Lets look and explain the purpose of each location-configuration.
        "proxy_ignore_headers Set-Cookie;")))
   #+end_src
 
+- ~/nar/~
+
+  As part of defining the nginx-location-configuration for ~*.narinfo~ routes, we started to
+  manually fetch a substitute. Here we will continue, using the provided hash to query to
+  query the substitute server for a corresponding ~.narinfo~ file.
+
+  #+begin_src shell :results output
+    curl https://cuirass.genenetwork.org/8bjy9g0cssjrw9ljz2r8ww1sma95isfj.narinfo
+  #+end_src
+
+  #+RESULTS:
+  #+begin_example
+  StorePath: /gnu/store/8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1
+  NarHash: sha256:0f94l0bl09i2igkhklzkawqbbdn4kkxl90wbb4y7f0dnni4f6ljh
+  NarSize: 235240
+  References: 8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1 zvlp3n8iwa1svxmwv4q22pv1pb1c9pjq-glibc-2.39 zzpbp6rr43smwxzvzd4qd317z5j7qblj-gcc-11.4.0-lib
+  Deriver: 79dhya6sngg4rf53m1cyxlhn8y4pnw2n-hello-2.12.1.drv
+  Signature: 1;balg02;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjNjE0ODFDNDUzMkU3RTIyOUEzMDlDREVBRDM2MkE2Qzk4QjU0RkFDNEUyQjA1ODEzQ0ZDOEI1NzQ2RUY0NjYxMiMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDQ2NTA3Q0FBNUJFNEY1QUQxRUE0NzUwQzlEMjgzMjQ5NDMwMDQ1OEIzRTM5QUJDOTBFMjZGNkU0MTA0RjMwNCMpCiAgIChzICMwQjdERDlCRUE5ODA0MTkyQ0E2OTUwQzFGRUYzRDdEQ0M3RTMyQzNEMENGNDg3NkY4RkRBMzEwRTUzNkYwNEVBIykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjOTU3OEFENkNEQjIzQkE1MUY5QzQxODVENUQ1QTMyQTdFRUI0N0FDREQ1NUYxQ0NCOENFRTRFMDU3MEZCRjk2MSMpCiAgICkKICApCiApCg==
+  URL: nar/gzip/8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1
+  Compression: gzip
+  FileSize: 73331
+  URL: nar/zstd/8bjy9g0cssjrw9ljz2r8ww1sma95isfj-hello-2.12.1
+  Compression: zstd
+  FileSize: 65480
+  #+end_example
+
+  If the package is not available, this would return a ~404~ not found error. However, in our
+  case the substitute is found, and various details about it are provided back to the
+  requester. The ~URL~ field is notable, as it will be used next to download the substitute,
+  which is why we define another proxied route for ~/nar/~ below.
+
+  #+begin_src scheme
+    (nginx-location-configuration
+     (uri "/nar/")
+     (body
+      (list
+       (string-append "proxy_pass " url ";")
+       "client_body_buffer_size 256k;"
+
+       ;; Be more tolerant of delays when fetching a nar.
+       "proxy_read_timeout 60s;"
+       "proxy_send_timeout 60s;"
+
+       ;; Enable caching for nar files, to avoid reconstructing and
+       ;; recompressing archives.
+       "proxy_cache nar;"
+       "proxy_cache_valid 200 30d;" ; cache hits for 1 month
+       "proxy_cache_valid 504 3m;"  ; timeout, when hydra.gnu.org is overloaded
+       "proxy_cache_valid any 1h;"  ; cache misses/others for 1h.
+
+       "proxy_ignore_client_abort on;"
+
+       ;; Nars are already compressed.
+       "gzip off;"
+
+       ;; We need to hide and ignore the Set-Cookie header to enable
+       ;; caching.
+       "proxy_hide_header    Set-Cookie;"
+       "proxy_ignore_headers Set-Cookie;"
+
+       ;; Provide a 'content-length' header so that 'guix
+       ;; substitute-binary' knows upfront how much it is downloading.
+       ;; "add_header Content-Length $body_bytes_sent;"
+       )))
+  #+end_src
+
 - ~/file/~
 
+  FIXME: provide more context
+
   #+begin_src scheme
     ;; Content-addressed files served by 'guix publish'.
     (nginx-location-configuration