diff options
| author | Pjotr <wrk@bosa.lan> | 2026-11-12 14:45:16 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-11-12 14:45:16 +0000 |
| commit | d507d3220910f528d95932f7772f80a902ec6fe9 (patch) | |
| tree | bed56d83e8d621b6e9d88aa0a685297bf73fd6d8 /docs | |
| parent | 60420b6e076ea16109d54789cfcd89262a01ae57 (diff) | |
| download | genenetwork4-d507d3220910f528d95932f7772f80a902ec6fe9.tar.gz | |
gemma/ravanan: resolve the same inputs as the direct runner; refuse what it cannot do
Two gaps, both invisible until a real request ran against a cluster. The endpoint read only genotype_file, snps_file and trait_values from the request; group, genotype_version and samples were accepted by the shared request type and silently dropped. A /compute/gemma-shaped request (group + samples) therefore arrived as empty paths and empty values and died with "I/O error: No such file or directory (os error 2)" from reading "". Running it exposed the second gap in the same breath: the gate sized the request from trait_values.len(), so a samples-only request was sized 0 and took no concurrency slot at all. resolve_inputs is now one function in routes/gemma.rs, called by both handlers: upload materialised into the pair the engine reads, basket resolved to named values, group prepared with the parent/F1 and empty-genotype rules, named samples aligned by name onto whatever individuals the file has, positional values checked against the panel size. That block was ~250 lines of duplicated rules inside compute_gemma, which is how it drifted in the first place; the direct runner now calls the same function and keeps only its covariates, normalization and gate. The ravanan reply also carries the input notes now (which genotype version, how many values aligned, what was dropped), so a run says what it mapped as well as where the peak is. Refused rather than ignored, because the CWL workflow has no step for either: covariates (422 covariates_not_supported) and normalize (422 normalization_not_supported), each naming /compute/gemma as the endpoint that can do it. A new test asserts both refusals happen before any job is submitted, so it needs no ravanan, ccwl or cluster. Verified against the Slurm container from gn-machines: group: BXD became 40 Slurm jobs (20 kinship + 20 GWA, scattered over the snps file's chromosomes), all completed, 21 055 rows in ~3.5 min. README and docs/ravanan-slurm.md record the four requirements that run turned up, three of them not obvious: ccwl on PATH, a ravanan build with the --slurm-api-endpoint fix, the *server's* HOME writable by the job user inside the cluster (gemma-wrapper locks $HOME/.<hash>.lck there), and GUIX_PACKAGE_PATH for the manifest.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/ravanan-slurm.md | 33 |
1 files changed, 33 insertions, 0 deletions
diff --git a/docs/ravanan-slurm.md b/docs/ravanan-slurm.md index 353e06b..69e1bc1 100644 --- a/docs/ravanan-slurm.md +++ b/docs/ravanan-slurm.md @@ -105,6 +105,39 @@ trusted: deployment's data directories are, but a pheno file written into a *different* mount gives `Invalid cross-device link`. +## Requirements found by running it (2026-11-12) + +Verified against `gn-machines/specials/slurm-container.scm` started through +`run-guix-system-container` (daemon on `127.0.0.1:8989`, fingerprint +`74c85932`): `POST /compute/gemma/ravanan` with `group: BXD` produced 40 +Slurm jobs -- 20 `kinship` + 20 `gwa`, one pair per chromosome -- all +completed, 21 055 rows back, in about 3.5 minutes. + +Four things had to be right, and three of them were not obvious: + +- **`ccwl` and `ravanan` on the server's `PATH`.** Without `ccwl` the + endpoint answers `502 failed to run ccwl: No such file or directory` + before doing anything. +- **A `ravanan` with the `--slurm-api-endpoint` fix.** The 0.2.0 build in + the store predates `ravanan: Parse --slurm-api-endpoint into a URI` + and dies with `struct-vtable: Wrong type argument: "http://127.0.0.1:6820"` + (it uses the string where a `uri` record is expected); a non-default + endpoint therefore needs that build. Using the built-in default + (`http://localhost:6820`, i.e. omitting the switch) is the workaround on + an older ravanan. +- **The server's `HOME` must be writable by the job user inside the + cluster.** ravanan forwards the submitting process's `HOME` into every + job (as `sbatch` would), and `gemma-wrapper` locks + `$HOME/.<hash>.lck`; inside the container there is no `/home/wrk`, so + every kinship job failed with `can not release ... .lck because it is not + owned by me` / `EACCES`. Starting the API with + `HOME=<a directory on the shared mount>` fixed it. This is a + *server-side* setting, not a node one. +- **`GUIX_PACKAGE_PATH`**, as above: ravanan resolves the manifest against + the ambient Guix, so without it the manifest cannot see + `(gn packages gemma)` and the failure surfaces as an unrelated channel + error (`rust-pyo3-…: unbound variable`). + ## Slurm-side requirements The endpoint only speaks HTTP to slurmrestd, so the cluster has to be |
