From c9db07f6d372b0b8812e5f6698257a9df94dc1ef Mon Sep 17 00:00:00 2001 From: Pjotr Date: Wed, 11 Nov 2026 12:52:36 +0000 Subject: docs: what BulkLMM models, what its cores do, and how to package it docs/bulklmm.md records the three things that are easy to get wrong when a BulkLMM result is quoted: * where it stands against the other engines -- Haley-Knott with no kinship (R/qtl), LOCO LMM whose permutations are refused (GEMMA), LOCO LMM on HMM probabilities (R/qtl2), and this: a global-kinship LMM whose permutation thresholds are rotation-based and therefore cheap -- the only engine here that can say "genome-wide significant at alpha 0.05" against a null that respects relatedness; * what it is not: global kinship rather than LOCO, so its p-values are not GEMMA's; h2_null says how much relatedness it found (near zero here, which is why it reproduces R/qtl's 9.59 where GEMMA differs); one degree of freedom, so multiparent scans stay with /compute/qtl2; and no standard errors in 1.2.2's scan(), so the rows are marker, chr, pos, lod, beta, h2; * the cores, measured: Threads.@threads is in bulkscan.jl and bulkscan_helpers.jl, never in scan.jl, so a single-trait scan is serial by design and the visible parallelism was BLAS (96 threads 10.5 s against 4 threads 7.7 s). What the driver does about it -- exact marker blocks, cached matrix and kinship, capped BLAS, and the breakdown printed into the reply -- with the numbers (20.9 s at one block against 18.3 s at four, same LOD 9.589938). Also the packaging picture, since it is not obvious from the outside: guix-bioinformatics has a working julia-bin and a julia-build-system, but only three of BulkLMM's closure are packaged, so a pure-Guix build is 35-45 recipes; the pragmatic route is julia-bin plus a depot built once from the pinned Manifest. scripts/prepare-julia-depot.sh is that step, written down, and the depot stays small and GPU-free (79 packages, 139 MB) with JULIA_PKG_OFFLINE=true at request time. The plot test learns that both thresholds are drawn, not one. --- scripts/prepare-julia-depot.sh | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100755 scripts/prepare-julia-depot.sh (limited to 'scripts') diff --git a/scripts/prepare-julia-depot.sh b/scripts/prepare-julia-depot.sh new file mode 100755 index 0000000..0e7b2d9 --- /dev/null +++ b/scripts/prepare-julia-depot.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# Prepare the Julia depot BulkLMM needs, once, deliberately. +# +# The depot is small on purpose (79 packages, ~140 MB: CSV, DataFrames, +# Distributions, Optim and their dependencies -- no GPU/CUDA stack). Requests +# run with JULIA_PKG_OFFLINE=true, so a scan never downloads anything; this +# script is the only thing that does, and only when run by hand. +# +# scripts/prepare-julia-depot.sh /path/to/depot /path/to/project +# +# The project must carry the pinned Manifest.toml (see docs/bulklmm.md); the +# deployment's copy lives in factory/gn4-julia-env. +set -euo pipefail + +DEPOT="${1:-$HOME/.julia}" +PROJECT="${2:-$(cd "$(dirname "$0")/.." && pwd)/../gn4-julia-env}" +JULIA="${JULIA:-julia}" + +[ -f "$PROJECT/Manifest.toml" ] || { + echo "$PROJECT: no Manifest.toml -- the depot must be built from a pinned project" >&2 + exit 1 +} + +echo "depot: $DEPOT" +echo "project: $PROJECT" +echo "julia: $("$JULIA" --version)" + +JULIA_DEPOT_PATH="$DEPOT" "$JULIA" --startup-file=no --project="$PROJECT" -e ' + using Pkg + Pkg.instantiate() + Pkg.precompile() + println("depot ready: ", length(readdir(joinpath(DEPOT, "packages"))), " packages") +' +echo +echo "Point the API at it: BULKLMM_DEPOT=$DEPOT BULKLMM_ENV=$PROJECT" -- cgit 1.4.1