diff options
| author | Pjotr <wrk@bosa.lan> | 2026-10-22 09:02:53 +0000 |
|---|---|---|
| committer | Pjotr <wrk@bosa.lan> | 2026-10-22 09:02:53 +0000 |
| commit | 4c692148e6de551ef7c83677b5f1e5ca11e05df7 (patch) | |
| tree | e573ca630ad74810e6e92035936905777486faef | |
| parent | 07298ef6b812cd07a7c505fec12cc765cfaf319b (diff) | |
| download | hmf-guix-cwl-main.tar.gz | |
docs: Record the GPU investigation and its two blockers. main
The host does have an RTX 5080 (driver 610.43.02), but it cannot be used from here and would not work with the packaged DJL stack anyway: - the sandbox has no nvidia device nodes (/dev is a private tmpfs with only null/zero/full/random/urandom/tty/fuse), CapEff=0 so mknod is denied, and there is no libcuda.so* anywhere; only the kernel module is visible through /proc/driver/nvidia; - DJL's newest CUDA native is pytorch-native-cu124 (PyTorch 2.5.1), and the libtorch_cuda.so in our cu121 bundle carries sm_35..sm_90 kernels only - no sm_100/sm_120, which is what Blackwell needs (CUDA 12.8+). The plausible route is written down for whoever has the device: DJL 0.39 (pytorch-jni 2.7.1; all 17 DJL classes v-chord/v-cuppa import are unchanged) plus a system libtorch 2.7.1+cu128 via PYTORCH_LIBRARY_PATH. README's v-chord paragraph now states the same facts instead of "wait for a cu128 runtime".
| -rw-r--r-- | README.md | 16 | ||||
| -rw-r--r-- | session.md | 32 |
2 files changed, 42 insertions, 6 deletions
diff --git a/README.md b/README.md index 4b01029..a759e92 100644 --- a/README.md +++ b/README.md @@ -281,11 +281,17 @@ The scores are meaningless biologically (the circos PNG is a stub), but every stage of the DJL pipeline exercises: `libtorch` dlopen, TorchScript deserialisation, forward pass, output translation. -The GPU leg needs a DJL runtime with kernels for the installed card. -CUDA 12.1 (the newest DJL native published for PyTorch 2.2.2) has no -`sm_120` images, so RTX 50-series (Blackwell) cards fail with *"no kernel -image is available for execution on the device"*; either wait for a cu128 -runtime or point `PYTORCH_LIBRARY_PATH` at a system libtorch. +The GPU leg needs a DJL runtime with kernels for the installed card, and +no DJL release has them for RTX 50-series (Blackwell, `sm_120`): the +newest native DJL publishes is `pytorch-native-cu124` 2.5.1, i.e. CUDA +12.4, whose `libtorch_cuda.so` carries `sm_35` … `sm_90` kernels only +(checked against the 2.2.2/cu121 bundle here). Blackwell needs CUDA 12.8+ +(PyTorch 2.7+), so the card fails with *"no kernel image is available for +execution on the device"*. The plausible route is a newer DJL +(`pytorch-jni` 2.7.1 exists for DJL 0.34–0.39, and every DJL class v-chord +uses is unchanged in 0.39) plus a system libtorch 2.7.1+cu128 pointed at +by `PYTORCH_LIBRARY_PATH` — that has to be tried on a host where the GPU +device is actually visible. ## Related diff --git a/session.md b/session.md index d18fe00..f4522d7 100644 --- a/session.md +++ b/session.md @@ -199,8 +199,38 @@ ravanan (`cwl-directory-and-array-support`): | item | state | |------------------------------------------|-------| +| GPU (RTX 5080) from this sandbox | impossible: `/dev` is a private tmpfs with no `nvidia*` nodes, `CapEff=0` so `mknod` is denied, and no `libcuda.so*` exists anywhere (`/`, `/gnu/store`); only the kernel module is visible via `/proc/driver/nvidia` | +| GPU with the packaged DJL stack | impossible even on the host: newest DJL native is `pytorch-native-cu124` 2.5.1 (CUDA 12.4 -> `sm_90`), the card is `sm_120`; our cu121 `libtorch_cuda.so` lists `sm_35`…`sm_90` only | +| GPU, plausible route (host only) | DJL 0.39 (`pytorch-jni` 2.7.1, all 17 DJL classes used are unchanged) + system libtorch 2.7.1+cu128 via `PYTORCH_LIBRARY_PATH`; needs the device visible and a ~2.5 GB download | | `workflows/wigits.cwl` execution | not run end to end: needs an Ensembl cache, GC profile, BAF loci VCF, LILAC resources and indexed BAMs; `qc.cwl` and `dir-handoff.cwl` are the executable proofs | | ravanan cwltest e2e / conformance suites | not buildable here: `cwltest` -> `python-cheroot` fails two SSL assertions on this OpenSSL | -| v-chord GPU leg | needs a DJL runtime with `sm_120` kernels (cu128) for RTX 50-series cards | | ravanan branch | `cwl-directory-and-array-support`, 6 commits local; not merged, not sent upstream | +## 11. GPU investigation + +The host has an NVIDIA GeForce RTX 5080 (driver 610.43.02, open kernel +module), visible through `/proc/driver/nvidia`. Two independent blockers +prevent using it: + +| blocker | evidence | +|------------------------------------------------|----------| +| no device access from the sandbox | `/dev` is a private tmpfs carrying only `null`, `zero`, `full`, `random`, `urandom`, `tty`, `fuse`; `CapEff=0` so `mknod` fails with `EPERM`; no `libcuda.so*` under `/`, `/gnu/store` or `/usr/lib` | +| the card needs CUDA 12.8+, DJL ships CUDA 12.4 | newest DJL artifact is `pytorch-native-cu124` 2.5.1; `strings libtorch_cuda.so` from the 2.2.2/cu121 bundle lists `sm_35`…`sm_90` and no `sm_100`/`sm_120`, so a Blackwell card fails with "no kernel image is available for execution on the device" | + +What would be needed to go further, in order of cost: + +1. run `test/test-vchord.sh` on the host, where `/dev/nvidia*` and + `/usr/lib/x86_64-linux-gnu/libcuda.so.1` exist — still fails on the + second blocker; +2. bump the channel's DJL stack to 0.39 (the API is compatible: all 17 + DJL classes v-chord and v-cuppa import still exist), install + `pytorch-jni` 2.7.1-0.39.0, and point `PYTORCH_LIBRARY_PATH` at a + libtorch 2.7.1+cu128 build from download.pytorch.org (~2.5 GB). That + is the first combination that could execute on this card, and it needs + a machine with the device visible; +3. or bind-mount `/dev/nvidia*` and the host's `libcuda.so.1` into the + sandbox — only the host owner can do that, and it does not help until + step 2 is done. + +Until then the CPU leg of `test/test-vchord.sh` and `v-chord-e2e` in the +CWL suite remain the GPU-less proofs. |
