about summary refs log tree commit diff
path: root/scripts
diff options
context:
space:
mode:
authorPjotr <wrk@bosa.lan>2026-11-11 12:52:36 +0000
committerPjotr <wrk@bosa.lan>2026-11-11 12:52:36 +0000
commitc9db07f6d372b0b8812e5f6698257a9df94dc1ef (patch)
tree98b8e918ff8d7e2d6e97773bc6326fb4f7513721 /scripts
parente616b84fc86b176a20170a61ad205979af8bffc1 (diff)
downloadgenenetwork4-c9db07f6d372b0b8812e5f6698257a9df94dc1ef.tar.gz
docs: what BulkLMM models, what its cores do, and how to package it
docs/bulklmm.md records the three things that are easy to get wrong when a
BulkLMM result is quoted:

  * where it stands against the other engines -- Haley-Knott with no
    kinship (R/qtl), LOCO LMM whose permutations are refused (GEMMA),
    LOCO LMM on HMM probabilities (R/qtl2), and this: a global-kinship
    LMM whose permutation thresholds are rotation-based and therefore
    cheap -- the only engine here that can say "genome-wide significant
    at alpha 0.05" against a null that respects relatedness;
  * what it is not: global kinship rather than LOCO, so its p-values are
    not GEMMA's; h2_null says how much relatedness it found (near zero
    here, which is why it reproduces R/qtl's 9.59 where GEMMA differs);
    one degree of freedom, so multiparent scans stay with /compute/qtl2;
    and no standard errors in 1.2.2's scan(), so the rows are
    marker, chr, pos, lod, beta, h2;
  * the cores, measured: Threads.@threads is in bulkscan.jl and
    bulkscan_helpers.jl, never in scan.jl, so a single-trait scan is
    serial by design and the visible parallelism was BLAS (96 threads
    10.5 s against 4 threads 7.7 s). What the driver does about it --
    exact marker blocks, cached matrix and kinship, capped BLAS, and the
    breakdown printed into the reply -- with the numbers (20.9 s at one
    block against 18.3 s at four, same LOD 9.589938).

Also the packaging picture, since it is not obvious from the outside:
guix-bioinformatics has a working julia-bin and a julia-build-system, but
only three of BulkLMM's closure are packaged, so a pure-Guix build is
35-45 recipes; the pragmatic route is julia-bin plus a depot built once
from the pinned Manifest. scripts/prepare-julia-depot.sh is that step,
written down, and the depot stays small and GPU-free (79 packages,
139 MB) with JULIA_PKG_OFFLINE=true at request time.

The plot test learns that both thresholds are drawn, not one.
Diffstat (limited to 'scripts')
-rwxr-xr-xscripts/prepare-julia-depot.sh35
1 files changed, 35 insertions, 0 deletions
diff --git a/scripts/prepare-julia-depot.sh b/scripts/prepare-julia-depot.sh
new file mode 100755
index 0000000..0e7b2d9
--- /dev/null
+++ b/scripts/prepare-julia-depot.sh
@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+# Prepare the Julia depot BulkLMM needs, once, deliberately.
+#
+# The depot is small on purpose (79 packages, ~140 MB: CSV, DataFrames,
+# Distributions, Optim and their dependencies -- no GPU/CUDA stack). Requests
+# run with JULIA_PKG_OFFLINE=true, so a scan never downloads anything; this
+# script is the only thing that does, and only when run by hand.
+#
+#   scripts/prepare-julia-depot.sh /path/to/depot /path/to/project
+#
+# The project must carry the pinned Manifest.toml (see docs/bulklmm.md); the
+# deployment's copy lives in factory/gn4-julia-env.
+set -euo pipefail
+
+DEPOT="${1:-$HOME/.julia}"
+PROJECT="${2:-$(cd "$(dirname "$0")/.." && pwd)/../gn4-julia-env}"
+JULIA="${JULIA:-julia}"
+
+[ -f "$PROJECT/Manifest.toml" ] || {
+    echo "$PROJECT: no Manifest.toml -- the depot must be built from a pinned project" >&2
+    exit 1
+}
+
+echo "depot:   $DEPOT"
+echo "project: $PROJECT"
+echo "julia:   $("$JULIA" --version)"
+
+JULIA_DEPOT_PATH="$DEPOT" "$JULIA" --startup-file=no --project="$PROJECT" -e '
+    using Pkg
+    Pkg.instantiate()
+    Pkg.precompile()
+    println("depot ready: ", length(readdir(joinpath(DEPOT, "packages"))), " packages")
+'
+echo
+echo "Point the API at it:  BULKLMM_DEPOT=$DEPOT BULKLMM_ENV=$PROJECT"