diff options
| author | Pjotr <wrk@bosa.lan> | 2026-10-24 09:33:40 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-10-24 09:33:40 +0000 |
| commit | a7ffb1cfecbdb8ac3fcc6286d7a09f7285c95d67 (patch) | |
| tree | 1cb865784d901fad979aed32694b77474019468d /README.md | |
| parent | cda325b6334ef36e3f49501cfd77d68a7bd6a295 (diff) | |
| download | genenetwork4-a7ffb1cfecbdb8ac3fcc6286d7a09f7285c95d67.tar.gz | |
gemma: Make ravanan's batch system configurable (optional Slurm).
POST /compute/gemma/ravanan used to run every CWL step locally, because compute_ravanan never passed ravanan a batch system. Add a RavananBatch setting: - RAVANAN_BATCH_SYSTEM=single-machine (default) or slurm-api, failing closed on anything else, exactly like GN4_DB_MODE; - slurm-api adds --batch-system=slurm-api, --slurm-api-endpoint, --slurm-jwt (a *file*, which is what ravanan reads) and, when set, --slurm-partition/--slurm-nice; a slurm selection without an endpoint or a JWT file refuses to start rather than 502-ing every request; - --scratch defaults to ravanan_cache_dir for slurm, so the store and the job scratch space live on the same shared filesystem (with slurm that directory has to be visible to the compute nodes); - the response's ravanan note records which backend ran. The switch building is RavananBatch::ravanan_args so it is unit testable; compute_ravanan takes the backend and the scratch directory, and the route passes them from config. docs/ravanan-slurm.md documents the keys, the shared-directory requirement, and the slurm-side auth wiring (AuthType=auth/munge + AuthAltTypes=auth/jwt with SLURM_JWT in slurmrestd's environment) that makes REST-submitted jobs actually run.
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/README.md b/README.md index 421e56f..a2dfbf6 100644 --- a/README.md +++ b/README.md @@ -1033,6 +1033,16 @@ invocation time. `ravanan_cmd`, `ccwl_cmd`, `gemma_ravanan_workflow` files, run logs, the compiled workflow, and ravanan's own `--store`) are configurable the same way as the rest of `src/config.rs`'s layered config. + +ravanan's batch system is configurable too: by default each CWL step is +a local child process, and `RAVANAN_BATCH_SYSTEM=slurm-api` submits +every step to a Slurm cluster through slurmrestd instead (endpoint, JWT +file, partition, nice and scratch directory via `RAVANAN_SLURM_*` / +`RAVANAN_SCRATCH`). With slurm, `ravanan_cache_dir` must be on a +filesystem the compute nodes can see, since ravanan's `--store` and +scratch live there. See [docs/ravanan-slurm.md](docs/ravanan-slurm.md) +for the keys, the fail-closed rules and the slurm-side auth +requirements. `GET /compute/gemma/status/{run_id}` is shared between both endpoints and checks whichever cache directory the `run_id` actually logged to. |
