aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-05-31Add trailing slash to GN_SERVER_URL.Frederick Muriuki Muriithi
Without the trailing slash, the urllib.parse.urljoin function produces the wrong endpoint. * genenetwork-development.scm (genenetwork2-cd-gexp): Add trailing slash to GN_SERVER_URL. Signed-off-by: Arun Isaac <arunisaac@systemreboot.net>
2023-05-22Document getting a shell into containerFrederick Muriuki Muriithi
For some tasks (e.g. debugging the xapian index build), we need to manually queue the job, which means we need a shell into the container. Getting a shell using the default command displayed actually fails with: nsenter: failed to execute /bin/bash: No such file or directory so we need to provide the correct path to bash, and optionally, initialise the shell to setup the correct paths.
2023-05-05Run migrations after tests and before restarting genenetwork3.Frederick Muriuki Muriithi
Add a CI job that applies the auth database migrations. This job is run after the genenetwork3 tests have passed but before genenetwork3 is restarted. * genenetwork-development-deploy.sh: Share /export/data/genenetwork-sqlite. * genenetwork-development.scm: Import yoyo-migrations from (gnu packages databases). (<genenetwork-configuration>)[auth-db-path]: New field. (genenetwork3-auth-migrations-genenetwork, genenetwork3-auth-migrations-laminar, genenetwork-activation): New functions. (genenetwork-projects): Add genenetwork3-auth-migrations CI job. Trigger this job after genenetwork3 tests. (genenetwork3-cd-gexp): Configure AUTH_DB_PATH in gn3.conf. (genenetwork-service-type): Extend activation-service-type with genenetwork-activation. (operating-system)[sudoers-file]: Permit the laminar user to run auth database migrations as the genenetwork user. Signed-off-by: Arun Isaac <arunisaac@systemreboot.net>
2023-04-12Run tissue on an IP socket.Arun Isaac
tissue fails to clean up its Unix socket when deployed with shepherd. So, when tissue crashes, it fails to restart. Until this issue is resolved, run tissue on an IP socket. * genenetwork-development.scm (%tissue-port): New variable. (tissue-reverse-proxy-server-block): Forward tissue requests to IP socket. (operating-system)[services]: Run tissue on an IP socket.
2023-04-12Disable libgit2 repository ownership validation.Arun Isaac
* genenetwork-development.scm: Import guile-git from (gnu packages guile), libgit2-1.3 from (gnu packages version-control), and (guix utils). Prefix tissue import from (gnu packages web) with guix:. (libgit2-1.3, guile-git-for-tissue, tissue): New variables. (operating-system)[services]: Use local variant of tissue package.
2023-04-11Use tissue package from Guix upstream.Arun Isaac
The tissue package is now in Guix upstream and longer in the guix-forge channel. * genenetwork-development.scm: Import tissue from (gnu packages web).
2023-04-11Provide MySQL URI to the xapian indexing script.Frederick Muriuki Muriithi
* genenetwork-development.scm (build-xapian-index-gexp): Provide MySQL URI to the xapian indexing script.
2023-02-09Share the mysqld socket directory into the container.Arun Isaac
We ought to share the mysqld socket directory into the container, not the socket itself. If we only shared the socket, when the mysql server is restarted and a new socket is created, the bind mounting into the container would break and the container would need to be restarted. * genenetwork-development-deploy.sh: Share /var/run/mysqld, not /var/run/mysqld/mysqld.sock. * genenetwork-development.scm (genenetwork-shepherd-services): Share /run/mysqld, not /run/mysqld/mysqld.sock.
2023-01-21GN3: pylint - Check more files and directoriesFrederick Muriuki Muriithi
2023-01-18Add data directory to genenetwork configuration.Arun Isaac
* genenetwork-development.scm (<genenetwork-configuration>)[data-directory]: New field. (genenetwork3-cd-gexp): Set DATA_DIR in settings file. (genenetwork-shepherd-services): Map data directory into container. (operating-system)[services]: Set data-directory of genenetwork service to /export/data/genenetwork. * genenetwork-development-deploy.sh: Expose entire data directory /export/data/genenetwork instead of the contained genotype files directory.
2023-01-18Move xapian directory.Arun Isaac
* genenetwork-development.scm (%xapian-directory): Move from /export/data/genenetwork/xapian to /export/data/genenetwork-xapian. * genenetwork-development-deploy.sh (container_script): Share /export/data/genenetwork-xapian instead of /export/data/genenetwork/xapian.
2023-01-08Clean up xapian build directory if previous build failed to.Arun Isaac
* genenetwork-development.scm (build-xapian-index-gexp): Clean up xapian build directory if previous build failed to.
2023-01-05Create custom 502 error page for CD.Arun Isaac
This 502 error page is more informative than a generic "Bad Gateway" error message. It tells the user that the CD is down but not the entire container itself. * genenetwork-development.scm: Import guile-lib from (gnu packages guile-xyz). (%ci-domain): New variable. (cd-error-pages-gexp): New function. (development-server-reverse-proxy-server-block): Use (cd-error-pages-gexp) to construct error pages. (laminar-reverse-proxy-server-block): Use %ci-domain.
2023-01-02Register containers as garbage collector roots.Arun Isaac
We don't want the Guix garbage collector accidentally clearing away store items required by the container. So, register them as garbage collector roots. * genenetwork-development-deploy.sh, production-deploy.sh, public-sparql-deploy.sh, virtuoso-deploy.sh: Register containers as garbage collector roots.
2022-12-22Build and install xapian index nightly.Arun Isaac
* genenetwork-development.scm: Import (gnu services mcron). (%xapian-directory): New variable. (build-xapian-index-gexp): New function. (genenetwork-projects): Add genenetwork3-build-xapian-index CI job to genenetwork3 project. (operating-system)[sudoers-file]: Allow laminar user to start and stop genenetwork3. [services]: Add mcron service to trigger nightly xapian build. Use %xapian-directory instead of its literal value. chown %xapian-directory to the laminar user.
2022-12-22Separate out sudo into a variable.Arun Isaac
* genenetwork-development.scm (genenetwork-projects): Separate out "/run/setuid-programs/sudo" into ... (sudo): ... new variable.
2022-12-22gn3: tests: Update the tests invocationFrederick Muriuki Muriithi
* genenetwork-development.scm: use pytest rather than unittest to run the tests.
2022-12-10Move guix container state to /export2.Arun Isaac
We prefer /export2 to house really large data. * genenetwork-development-deploy.sh, production-deploy.sh, public-sparql-deploy.sh, virtuoso-deploy.sh: Move guix container state to /export2.
2022-11-10Configure SPARQL endpoint of genenetwork3 server.Arun Isaac
* genenetwork-development.scm (<genenetwork-configuration>)[sparql-endpoint]: New field. (genenetwork3-cd-gexp): Set SPARQL_ENDPOINT in gn3.conf. (%virtuoso-sparql-port): New variable. (operating-system)[services]: Configure sparql-endpoint of genenetwork service.
2022-11-07Add public facing SPARQL container.Arun Isaac
* public-sparql-deploy.sh, public-sparql.scm: New files.
2022-11-07Move CD virtuoso ports to 90*.Arun Isaac
In the interest of good organization, we keep all CD ports at 90*. Production and other services can use ports outside this range. * genenetwork-development.scm (operating-system)[services]: Move virtuoso ports from 8891, 8892 to 9081, 9082.
2022-11-04Import GeneRIF by calling the unified dump script.Arun Isaac
* genenetwork-development.scm (dump-genenetwork-database): Import GeneRIF by calling the unified dump script.
2022-10-31Find GeneRIF data in root of export directory.Arun Isaac
* genenetwork-development.scm (dump-genenetwork-database): Find GeneRIF data in root of export directory.
2022-10-31Use guile-zlib in dump-genenetwork-database CI job.Arun Isaac
* genenetwork-development.scm (dump-genenetwork-database): Use guile-zlib.
2022-10-31Ungexp gn2 and gn3 repositories correctly.Arun Isaac
* genenetwork-development.scm (genenetwork2-tests): Ungexp gn2-repository and gn3-repository correctly.
2022-10-31Import (guix build utils) in after scripts of CI jobs.Arun Isaac
* genenetwork-development.scm (genenetwork-projects): Import (guix build utils) in after scripts of CI jobs.
2022-10-30Import GeneRIF into RDF.Arun Isaac
* genenetwork-development.scm (dump-genenetwork-database): Import GeneRIF into RDF.
2022-10-30Unite genenetwork2 and genenetwork3 development servers.Arun Isaac
We unite the genenetwork2 and genenetwork3 development servers into a single Guix service. This unifies many of the configuration settings and simplifies the configuration. Changes to the genenetwork3 repository also immediately reflect in the genenetwork2 continuous deployment without guix-bioinformatics having to be updated. * genenetwork-development.scm (<development-server-configuration>): Delete type. (<genenetwork-configuration>): New type. (development-server-redeploy, development-server-activation, genenetwork2-project, genenetwork2-shepherd-service, genenetwork3-project, genenetwork3-shepherd-service): Delete functions. (genenetwork2-tests): Accept <genenetwork-configuration> object instead of <forge-project> object. (%genotype-files, %xapian-db-path, %default-genenetwork2-configuration, genenetwork2-service-type, %default-genenetwork3-configuration, genenetwork3-service-type): Delete variables. (genenetwork-projects, genenetwork2-cd-gexp, genenetwork3-cd-gexp, genenetwork-shepherd-services): New functions. (operating-system)[services]: Replace genenetwork2 and genenetwork3 services with the unified genenetwork service. * genenetwork/development-helper.scm (genenetwork2-runner-gexp, genenetwork3-runner-gexp): Delete functions.
2022-10-28Expose xapian index to genenetwork3 instead of genenetwork2.Arun Isaac
* genenetwork-development.scm (genenetwork2-shepherd-service): Do not expose xapian index. (genenetwork3-shepherd-service): Expose xapian index. * genenetwork/development-helper.scm (genenetwork2-runner-gexp): Do not accept xapian index argument. (genenetwork3-runner-gexp): Accept config file argument.
2022-10-25Link to gemtext documentation on securing virtuoso.Arun Isaac
* README.org (Virtuoso container for tux01 production): Remove user management commands. (Secure virtuoso authentication): New section.
2022-10-20Use GitHub URI for dump-genenetwork-database repo.Arun Isaac
* genenetwork-development.scm (dump-genenetwork-database-project)[repository]: Use GitHub URI.
2022-10-18Trigger genenetwork2 using laminar, not shepherd.Arun Isaac
This was a typo. * genenetwork-development.scm (genenetwork3-project)[ci-jobs]: Trigger genenetwork2 using laminar, not shepherd.
2022-10-17Use <forge-ip-socket> object to specify webhook port.Arun Isaac
* genenetwork-development.scm: Import (forge socket). (operating-system)[services]: Use <forge-ip-socket> object to specify webhook port.
2022-10-17Use least-authority-wrapper.Arun Isaac
* genenetwork-development.scm: Import (gnu build linux-container), (gnu system file-systems) and (guix least-authority). (genenetwork2-shepherd-service, genenetwork3-shepherd-service): Deprecate make-forkexec-constructor/container in favor of least-authority-wrapper.
2022-10-14Trigger genenetwork2 tests after genenetwork3 tests.Arun Isaac
Since genenetwork2 depends on genenetwork3, updates to genenetwork3 should trigger the genenetwork2 tests. * genenetwork-development.scm (genenetwork3-project): Trigger genenetwork2 tests after genenetwork3 tests.
2022-10-14Comment about triggering of Mechanical Rob.Arun Isaac
* genenetwork-development.scm (genenetwork2-project): Comment about triggering of Mechanical Rob after genenetwork2 tests.
2022-10-14Convert port number to string on host.Arun Isaac
* genenetwork-development.scm (genenetwork2-shepherd-service): Convert port number to string on the host, not on the build side.
2022-10-14Pull in latest genenetwork3 when running genenetwork2 tests.Arun Isaac
So far, whenever genenetwork2 code needed the latest updates from genenetwork3, the guix-bioinformatics channel had to be updated and the development container had to be rebuilt. No more. From now on, the latest genenetwork3 code will be pulled in at all times. The development container need only be rebuilt when the dependencies of genenetwork2 or genenetwork3 change. We may revisit this decision once genenetwork2 and genenetwork3 are better decoupled and we have semantically versioned releases. * genenetwork-development.scm (genenetwork2-tests): Pull in latest genenetwork3 when running genenetwork2 tests.
2022-10-12Add virtuoso container for tux01 production.Arun Isaac
* virtuoso.scm, virtuoso-deploy.sh: New files. * README.org (Virtuoso container for tux01 production): New section.
2022-10-06Enable rust backtrace in genenetwork2 CD.Frederick Muriuki Muriithi
* genenetwork/development-helper.scm (genenetwork2-runner-gexp): Set RUST_BACKTRACE environment variable. Signed-off-by: Arun Isaac <arunisaac@systemreboot.net>
2022-09-29Expose root of data directory instead of its sub-directories.Arun Isaac
* genenetwork-development-deploy.sh (container_script): Expose root of data directory instead of its sub-directories.
2022-09-29Run genenetwork2 CD on gunicorn.Arun Isaac
* genenetwork/development-helper.scm (genenetwork2-runner-gexp): Run genenetwork2 CD on gunicorn.
2022-09-26Set up xapian search index for genenetwork2 CD.Arun Isaac
* genenetwork-development.scm (%xapian-db-path): New variable. (genenetwork2-shepherd-service)[start]: Share %xapian-db-path with the container. (%default-genenetwork2-configuration)[runner]: Pass %xapian-db-path to CD runner. * genenetwork/development-helper.scm (genenetwork2-runner-gexp): Accept xapian-db-path argument and set XAPIAN_DB_PATH environment variable. * genenetwork-development-deploy.sh: Expose /export/data/genenetwork/xapian in the container.
2022-09-21Use pytest, rather than unittest, to run tests.Frederick Muriuki Muriithi
* genenetwork-development.scm (genenetwork2-project)[ci-jobs]: Use pytest instead of unittest. Signed-off-by: Arun Isaac <arunisaac@systemreboot.net>
2022-09-15Copy laminar HTML template from git source.Arun Isaac
The upstream laminar package now uses git-fetch instead of url-fetch. Hence, we must adapt our laminar template G-expression. * genenetwork-development.scm (laminar-template-gexp): Copy /src/resources/index.html from git source instead of extracting it from a tarball.
2022-09-15Link to CD from the navigation menu.Arun Isaac
* genenetwork-development.scm (laminar-template-gexp): Link to continuous deployment from the navigation menu.
2022-09-08Set genenetwork3 API endpoint used by genenetwork2 to /api3.Arun Isaac
* genenetwork/development-helper.scm (genenetwork2-runner-gexp): Set GN_SERVER_URL to /api3.
2022-09-08Expose continuously deployed genenetwork at cd.genenetwork.org.Arun Isaac
* genenetwork-development.scm (%genenetwork2-port): New variable. (genenetwork2-project): Use %genenetwork2-port. (development-server-reverse-proxy-server-block): New function. (operating-system)[services]: Use %genenetwork2-port. Add reverse proxy block for cd.genenetwork.org.
2022-09-08Use %genenetwork3-port in genenetwork2 tests.Arun Isaac
* genenetwork-development.scm (genenetwork2-tests): Use %genenetwork3-port.
2022-09-06Expose webhook at ci.genenetwork.org.Arun Isaac
* genenetwork-development.scm (%webhook-port): New variable. (laminar-server-reverse-proxy-server-block): Expose webhook under /hooks/. (operating-system)[services]: Use %webhook-port.
generated by cgit v1.2.3 (git 2.44.0) at 2024-11-15 20:35:26 +0000