about summary refs log tree commit diff
path: root/README.md
diff options
context:
space:
mode:
authorPjotr <wrk@bosa.lan>2026-10-24 09:33:40 +0000
committerPjotr <wrk@bosa.lan>2026-10-24 09:33:40 +0000
commita7ffb1cfecbdb8ac3fcc6286d7a09f7285c95d67 (patch)
tree1cb865784d901fad979aed32694b77474019468d /README.md
parentcda325b6334ef36e3f49501cfd77d68a7bd6a295 (diff)
downloadgenenetwork4-a7ffb1cfecbdb8ac3fcc6286d7a09f7285c95d67.tar.gz
gemma: Make ravanan's batch system configurable (optional Slurm).
POST /compute/gemma/ravanan used to run every CWL step locally, because
compute_ravanan never passed ravanan a batch system.  Add a
RavananBatch setting:

- RAVANAN_BATCH_SYSTEM=single-machine (default) or slurm-api, failing
  closed on anything else, exactly like GN4_DB_MODE;
- slurm-api adds --batch-system=slurm-api, --slurm-api-endpoint,
  --slurm-jwt (a *file*, which is what ravanan reads) and, when set,
  --slurm-partition/--slurm-nice; a slurm selection without an endpoint
  or a JWT file refuses to start rather than 502-ing every request;
- --scratch defaults to ravanan_cache_dir for slurm, so the store and
  the job scratch space live on the same shared filesystem (with slurm
  that directory has to be visible to the compute nodes);
- the response's ravanan note records which backend ran.

The switch building is RavananBatch::ravanan_args so it is unit
testable; compute_ravanan takes the backend and the scratch directory,
and the route passes them from config.  docs/ravanan-slurm.md documents
the keys, the shared-directory requirement, and the slurm-side auth
wiring (AuthType=auth/munge + AuthAltTypes=auth/jwt with SLURM_JWT in
slurmrestd's environment) that makes REST-submitted jobs actually run.
Diffstat (limited to 'README.md')
-rw-r--r--README.md10
1 files changed, 10 insertions, 0 deletions
diff --git a/README.md b/README.md
index 421e56f..a2dfbf6 100644
--- a/README.md
+++ b/README.md
@@ -1033,6 +1033,16 @@ invocation time. `ravanan_cmd`, `ccwl_cmd`, `gemma_ravanan_workflow`
 files, run logs, the compiled workflow, and ravanan's own `--store`)
 are configurable the same way as the rest of `src/config.rs`'s layered
 config.
+
+ravanan's batch system is configurable too: by default each CWL step is
+a local child process, and `RAVANAN_BATCH_SYSTEM=slurm-api` submits
+every step to a Slurm cluster through slurmrestd instead (endpoint, JWT
+file, partition, nice and scratch directory via `RAVANAN_SLURM_*` /
+`RAVANAN_SCRATCH`). With slurm, `ravanan_cache_dir` must be on a
+filesystem the compute nodes can see, since ravanan's `--store` and
+scratch live there. See [docs/ravanan-slurm.md](docs/ravanan-slurm.md)
+for the keys, the fail-closed rules and the slurm-side auth
+requirements.
 `GET /compute/gemma/status/{run_id}` is shared between both endpoints
 and checks whichever cache directory the `run_id` actually logged to.