guix-hmf
Guix packages for the Hartwig Medical Foundation
WiGiTS tools: every Java module of the hmftools monorepo (AMBER,
PURPLE, SAGE, LINX, ...) plus the shared hmf-common library, the Java
dependency tree they need, and CWL descriptions of the pipeline for
ravanan.
Everything that can reasonably be compiled here is compiled here: the
hmftools modules are built from a pinned upstream source snapshot, and
the third-party libraries are built from their Maven Central source
archives. Only artifacts that are native payloads, or that cannot be
built in this environment, are installed as published jars (see
Not built from source).
Layout
hmf/packages/hmftools.scm hmftools modules + maven-* release-name variants
hmf/packages/java-libraries.scm third-party Java dependencies
test/test-vchord.sh end-to-end DJL/PyTorch smoke test
cwl/ CWL tools, workflows and tests (ravanan)
The channel is used directly from a checkout:
guix build -L . java-purple
guix shell -L . java-purple java-amber java-cobalt -- purple --version
or as a (local-file ...) / git channel entry in ~/.config/guix/channels.scm.
Source pin and version tracking
The hmftools monorepo builds as amber 4.4 / purple 4.5 / sage 5.1 / ...
but has published no tag for those versions (the newest release tag
is still amber-v4.3). %hmf-source therefore pins the master commit
those version properties come from:
| field | value |
|---|---|
| commit | 6531166b6664981725de58703224b6d8b044336e (2026-10-14) |
| sha256 | 1nbr2csac8naz5x21wrza4qlhkykzmk6avn977333ysxn3di9h7g |
Each package's version is taken from that tree's pom.xml, so the
version strings describe what is actually compiled. When the pin moves,
bump the versions together with it (they are all read from the same
schema-free <module>.version properties).
%hmf-source-4.3 keeps the old amber-v4.3 snapshot for the two
modules upstream has since removed (fastq-tools, health-checker);
hmftools-module takes #:source for exactly this case.
Packages
hmftools modules
All built from source by hmftools-module, one package per module, each
installing share/java/<module>.jar and a bin/<module> launcher:
| package | version | note |
|---|---|---|
java-amber |
4.4 | |
java-bam-tools |
1.6.2 | |
java-chord |
2.1.2 | |
java-cider |
1.2 | |
java-cobalt |
3.1 | |
java-compar |
1.6 | |
java-crest |
1.0.0 | |
java-cuppa |
2.5 | |
java-esvee |
2.1 | |
java-fastq-tools |
1.0 | built from the 4.3 tree (dropped upstream) |
java-gene-utils |
1.3 | |
java-health-checker |
3.6 | built from the 4.3 tree (dropped upstream) |
java-hmf-common |
4.4 | library, no launcher |
java-hmf-common-bwa |
4.4 | library, keeps common.bwa |
java-hmf-id-generator |
3.1 | |
java-isofox |
2.1 | |
java-lilac |
2.1 | |
java-linx |
2.4 | |
java-neo |
1.2 | |
java-orange |
5.1 | |
java-orange-datamodel |
5.1 | library, no launcher |
java-paddle |
1.1 | |
java-panel-builder |
1.2 | |
java-patient-db |
3.0 | library, no launcher |
java-pave |
1.9 | |
java-pave-reverse |
1.9 | |
java-peach |
2.0 | |
java-purple |
4.5 | |
java-qsee |
1.0 | |
java-redux |
2.1 | |
java-sage |
5.1 | |
java-sigs |
1.2.1 | |
java-sv-tools |
1.0 | |
java-taur |
1.0 | |
java-teal |
1.4 | |
java-v-chord |
1.0.1 | |
java-v-cuppa |
0.1 | |
java-virus-interpreter |
1.7.1 | |
java-wisp |
1.4 |
maven-amber and maven-bam-tools are the same builds under the
names upstream publishes (the amber_v<version>.jar /
bam-tools_v<version>.jar release artifacts). They used to be those
jars, installed verbatim; they are now compiled from source and keep the
release artifacts' visible behaviour: the names, the bamtools alias for
bam-tools, and the release launcher that lifts -Xm*, -D* and -XX*
arguments out of argv into JVM options (#:launcher-style 'release).
Plain module launchers are thin exec java -cp ... MAIN "$@" scripts
(#:launcher-style 'thin).
Launchers start with an absolute #!<bash-minimal>/bin/bash: /bin/sh
does not exist on a Guix System. Modules without an entry point
(java-orange-datamodel) pass #:main-class #f and get no launcher.
Third-party dependencies
hmf/packages/java-libraries.scm holds 78 packages:
| category | count | what |
|---|---|---|
| compiled here | 57 | 47 via maven-central-source-jar plus this channel's own from-source packages |
| published jars | 14 | decided case by case, see Not built from source |
| DJL native bundles | 7 | platform jars that are the libtorch binaries |
| total | 78 |
Two constructors:
maven-central-source-jar— the default. FetchesARTIFACT-VERSION-sources.jarfrom Maven Central and compiles it with javac on openjdk17, installing the same<artifact>-<version>.jarname the binary package used, so consumers are unaffected.maven-central-jar— copies the published jar verbatim (see the exceptions below).
maven-central-source-jar handles the things Maven normally does:
| concern | handling |
|---|---|
module-info.java |
deleted; ant has no module path and javac errors on it |
multi-release META-INF/versions/** |
pruned; ant's flat javac would compile duplicate classes |
resources (META-INF/services, *.properties) |
copied into the class tree before jarring |
META-INF/maven/<group>/<artifact>/pom.properties |
synthesised (DJL reads versions back from it at runtime) |
| generated resources | #:extra-resources, e.g. DJL's pytorch-engine.properties |
| optional integrations | #:prune, a list of regexps (Avalon logging, OSGi annotations, c3p0, Lucene full-text, PDFBox pubkey crypto) |
| JDK level | #:jdk openjdk17; several artifacts need instanceof patterns or java.util.concurrent.Flow |
Things that bit us and are worth remembering:
- BouncyCastle split. iText 7.1.5 compiles against the 1.53 line
(
DEROutputStreamwas still public); PDFBox 2.0.30 and 3.0.6 need the 1.68 line (ASN1Encodable.encodeTo). Both are packaged (java-bcprov-jdk15on,-1.68and matchingbcpkix); they only meet in PDFBox's unused certificate-encryption path. - jOOQ 3.15.5 is pinned to
r2dbc-spi 0.9.0.M1, not the GA — that is whatjooq-parentdeclares, and the GA'sRow/connection API does not match. - jOOQ codegen needs
java-jaxb-{api,runtime},istack,txw2,jakarta-activationand H2 (as theDDLDatabaseengine) on a hand-built classpath;java-patient-dbglobs each dependency'sshare/javarather than hard-coding jar names. - H2's sources jar is incomplete:
org/h2/util/JSR310Utils.javais generated at build time upstream.java-h2databasefetches the git tag and compilesh2/src/maintogether withh2/src/java8/src. - boxable 1.8.2 uses
CSVFormat.Builder.get(), which is newer than commons-csv 1.9 and drags in commons-io ≥ 2.16; only itsdatatablehelper package is pruned, ORANGE builds tables throughBaseTable. - hmftools resources carry Maven filter placeholders
(
${project.version},${timestamp});hmftools-modulesubstitutes them at bundle time orVersionInfothrowsDateTimeParseExceptionat startup. hmf-common's resources matter:ant-build-systemonly compilessrc/main/java, sojava-hmf-commonjarssrc/main/resourcesitself. Without itRefGenomeCoordinatesNPEs and BAM Tools, AMBER, COBALT, PURPLE and SAGE all die withExceptionInInitializerError.
Not built from source
| group | packages, and why it stays a published jar |
|---|---|
| Kotlin toolchain | java-kotlin-* (8), java-kotlinx-coroutines-* (2) — sources are .kt; needs a Kotlin compiler bootstrapped from these jars |
| native payloads | java-djl-pytorch-jni, java-gatk-bwamem-jni, java-djl-pytorch-native-* (7) — the jar is the native code (libdjl_torch.so, libtorch, BWA-MEM) |
| empty sources | java-selenium-java — aggregator pom, its sources jar is empty |
| deep dependency tree | java-tablesaw-core — needs fastutil, RoaringBitmap, univocity, icu4j, classgraph, smile-core, jamm |
| cannot compile on JDK 17 | java-postgresql — waffle 1.9 needs the JAAS java.security.acl API that JDK 17 removed, and scram 2.1 is not published on Central; jOOQ only needs its public API at compile time |
| shaded fat jar | java-immutables-value — upstream ships a shade-plugin jar with relocated guava, consumed only as an annotation processor |
The Kotlin group is java-kotlin-stdlib, -stdlib-jdk7,
-stdlib-jdk8, java-kotlin-reflect, java-kotlin-compiler,
-compiler-embeddable, -daemon-embeddable, plus
java-kotlinx-coroutines-core and -core-jvm.
CWL workflows (ravanan)
cwl/ describes the WiGiTS pipeline in CWL and runs it with ravanan,
taking every tool binary from this channel's packages via a Guix
manifest. Stage hand-offs are Directory values, as the modules'
-<stage>_dir flags expect; running that shape needs ravanan built from
its checkout with the six patches in cwl/README.md (Directory values,
array staging, unlisted outputs). See cwl/README.md
for the graph and the full test matrix. In short:
export HMF_GUIX_CHANNEL=$PWD
RAVANAN=/path/to/ravanan HMFTOOLS_DIR=/path/to/hmftools \
./cwl/tests/run-tests.sh # add VCHORD=1 for the PyTorch case
| case | what it proves (all cases pass) |
|---|---|
version-<module> (15) |
the manifest exposes that module: <module> -version runs inside a ravanan job |
peach-haplotypes |
a real PEACH run on the hmftools haplotype fixtures, producing five output TSVs |
bam-metrics |
a real two-step workflow: samtools index, then BAM Tools' metrics |
array-stage |
an InitialWorkDirRequirement listing a File[] input stages every element |
dir-handoff |
one step's Directory output feeds the next step's Directory input |
v-chord-e2e (VCHORD=1) |
libtorch under ravanan: the PyTorch HRD classifier on the TSO500 model |
ravanan must be built from its checkout, not from the 0.2.0 release:
the release crashes on any tool without an arguments field, and
Directory-based workflows need the patches listed in cwl/README.md
(branch cwl-directory-and-array-support). Its own unit suites pass
here (58/58); the cwltest-based e2e and conformance suites cannot be
built because cwltest's dependency python-cheroot fails two SSL
assertions on this OpenSSL.
Verification
# every package in the channel
guix build -L . --no-offload --keep-going \
$(grep -oP '^\(define-public \Kjava-[a-z0-9.-]+' hmf/packages/hmftools.scm) \
maven-amber maven-bam-tools
# ditto for the libraries
guix build -L . --no-offload --keep-going \
$(grep -oP '^\(define-public \Kjava-[a-z0-9.-]+' hmf/packages/java-libraries.scm)
Expected: exit 0. Smoke test that every launcher resolves its main class:
for s in /gnu/store/*-java-*/bin/*; do "$s" 2>&1 | head -3; done
Upstream test suite (Maven, outside Guix)
Guix has no Maven. Using a Maven binary and the store's JDK:
export JAVA_HOME=$(guix build openjdk@17:jdk)
export PATH=/path/to/apache-maven/bin:$JAVA_HOME/bin:$PATH
cd hmftools
mvn -B install -DskipTests # hmf-common's test-jar
mvn -B -Dmaven.build.cache.enabled=false test # 3062 tests, 0 failures
install -DskipTests first: amber's tests need hmf-common's test-jar,
which only package/install produces. The build-cache extension is
enabled upstream, so a second mvn test silently reuses cached results
unless -Dmaven.build.cache.enabled=false is passed.
v-chord end-to-end (DJL / libtorch)
./test/test-vchord.sh
Downloads the upstream TSO500 TorchScript model, fabricates a minimal
PURPLE output directory, and runs java-v-chord on CPU (and on GPU if
nvidia-smi is present). Expected CPU result for the fabricated
COLO829T sample:
breast 0.7854 ovarian 0.8577 pancreatic 0.8223 prostate 0.8063 other 0.8236
The scores are meaningless biologically (the circos PNG is a stub), but
every stage of the DJL pipeline exercises: libtorch dlopen, TorchScript
deserialisation, forward pass, output translation.
The GPU leg needs a DJL runtime with kernels for the installed card, and
no DJL release has them for RTX 50-series (Blackwell, sm_120): the
newest native DJL publishes is pytorch-native-cu124 2.5.1, i.e. CUDA
12.4, whose libtorch_cuda.so carries sm_35 … sm_90 kernels only
(checked against the 2.2.2/cu121 bundle here). Blackwell needs CUDA 12.8+
(PyTorch 2.7+), so the card fails with "no kernel image is available for
execution on the device". The plausible route is a newer DJL
(pytorch-jni 2.7.1 exists for DJL 0.34–0.39, and every DJL class v-chord
uses is unchanged in 0.39) plus a system libtorch 2.7.1+cu128 pointed at
by PYTORCH_LIBRARY_PATH — that has to be tried on a host where the GPU
device is actually visible.
Related
cwl/README.md— CWL/ravanan side of this repo.skills/guix-containers/SKILL.md,skills/guix-packaging/SKILL.mdin the ai-root checkout — container/VM and packaging workflows used with this channel.
