From 54fe5ce5c20194b468631e906a3fcf5f9e52779d Mon Sep 17 00:00:00 2001 From: Munyoki Kilyungi Date: Wed, 27 Aug 2025 10:01:20 +0300 Subject: Update CI/CD notes. Signed-off-by: Munyoki Kilyungi --- issues/CI-CD/failing-services-startup.gmi | 40 +++++++++++++------------------ 1 file changed, 16 insertions(+), 24 deletions(-) (limited to 'issues') diff --git a/issues/CI-CD/failing-services-startup.gmi b/issues/CI-CD/failing-services-startup.gmi index 169bccf..f820b72 100644 --- a/issues/CI-CD/failing-services-startup.gmi +++ b/issues/CI-CD/failing-services-startup.gmi @@ -11,7 +11,7 @@ ## Description -On rebuild of the CI/CD container with guix channel pinned at commit `34453b97005ff86355399df89c8827c57839d9c7`, some services fail to start and the error messages we get are as follows: +Upgrading guix to `34453b97005ff86355399df89c8827c57839d9c7` for CI/CD fails with: ``` 2025-08-20 16:05:20 Backtrace: @@ -41,43 +41,35 @@ On rebuild of the CI/CD container with guix channel pinned at commit `34453b9700 2025-08-20 16:05:20 Wrong type (expecting exact integer): #f ``` -The services that fail are: +Failing services: * genenetwork3: consistently * genenetwork2: consistently * gn-auth: intermittently -After digging further into this issue, and I think I have the beginnings of an idea of why the issue is comming up. Looking at: -=> https://codeberg.org/guix/guix/src/commit/34453b97005ff86355399df89c8827c57839d9c7/guix/build/syscalls.scm#L1218-L1233 +## Troubleshooting Notes -We see the documentation says: - -> Note that CLONE_NEWUSER requires that the calling process be single-threaded, -> which is possible if and only if libgc is running a single marker thread; this -> can be achieved by setting the GC_MARKERS environment variable to 1. If the -> calling process is multi-threaded, this throws to 'system-error' with EINVAL. - -and looking at the error we are getting: +To reproduce, set the default directory for all GN cloning to 'gn-repositories': ``` -⋮ -2025-08-20 15:17:38 guix/build/syscalls.scm:1231:10: In procedure unshare: 268566528: Invalid argument -⋮ +@@ -775,38 +775,43 @@ described by CONFIG, a object." + #~(make-forkexec-constructor + (list #$(least-authority-wrapper + (program-file "genenetwork2" + (genenetwork2-cd-gexp config)) + #:name "genenetwork2-pola-wrapper" ++ #:directory gn-repositories ``` -Now, looking at -=>https://codeberg.org/guix/guix/src/commit/34453b97005ff86355399df89c8827c57839d9c7/gnu/build/linux-container.scm#L321 where `unshare` is called - - -we could come to the conclusion that, perhaps the calling process for `unshare` for "genenetwork3" and "genenetwork2" is consistently multi-threaded, leading to the error above. - -It might also explain why the /gn-auth/ service will **sometimes** throw the same error when the container is restarted, but other times, it'll just start with no error. +On the first run, gn3/gn-auth/gn-guile start normally. gn2 fails (still under investigation). -I (currently) have no idea why the calling process would be multi-threaded. +After restarting the container, gn3/gn-auth/gn-guile fail with above error. -Or maybe, I'm overthinking this whole thing. +Removing "gn-repositories" and restarting the container makes everything run normally. ## Also See => https://issues.guix.gnu.org/78356 Broken system and home containers +=> https://codeberg.org/guix/guix/src/commit/34453b97005ff86355399df89c8827c57839d9c7/guix/build/syscalls.scm#L1218-L1233 How "unshare" is defined +=> https://codeberg.org/guix/guix/src/commit/34453b97005ff86355399df89c8827c57839d9c7/gnu/build/linux-container.scm#L321 Where `unshare` is called -- cgit 1.4.1