# BNW release notes Changes since `e9a320d0` (*Update home.php*, the last pre-Guix upstream commit), summarized for users and deployers. 34 commits, 2021-10-11 → 2026-10-03. ## Deployment / Guix packaging * **Guix system container and VM** (`7f460113`): BNW deploys as a Guix system container (`gn-machines/genenetwork/containers/bnw-container.scm`) and VM. The package definition lives in `guix-bioinformatics/gn/packages/bnw.scm`, pinned to commits of this repository and fetched from git.genenetwork.org. * **Container test cycle** (`7b6a8ccf`, `f76f79b8`): `make` builds the container from the pinned origin, starts it through the run-guix-system-container daemon, runs the HTTP test suite, and stops it again; `make run` leaves it up, `make clean` removes artifacts. Checkout paths are overridable (`base`, `GUIX_BIOINF`, ...). * **Anubis AI-bot firewall support** (in gn-machines): the web UI is challenge-protected, while the GeneNetwork workflow endpoints, the static assets and the .htm tutorial pages are exempt — the full modeling flow runs without ever hitting a challenge. * **Pinned container SSH keys** (`10ae4902`, `74f922e3`): host and admin keys live outside the repository in the building user's `$HOME`, so every container presents the same SSH host key. ## GeneNetwork (GN2) integration * **Single-POST handshake, XSS fixed** (`8380798b`, `75761501`): GN2 posts `My_Gennenet` (commas = columns, semicolons = rows) and the conversion now happens **server-side** in PHP, returning the prepared-data page in one request. The previous release echoed the raw payload into a JavaScript string (`genenet_str = ""`), an XSS hole on public deployments — that echo is gone. * **Accept the GN2 field spelling** (`3616c964`): GN2's template posts `My_Genenet` (one 'n'), while BNW historically read `My_Gennenet`. Both spellings are accepted now. * **Sanitize trait names with spaces** (`18dcbf7b`): GN2 trait display names often contain spaces ("Age at death"); BNW's file format does not allow them (the octave parsers split on whitespace), so spaced names corrupted the header and structure learning died with "A(0,_): subscripts must be ... integers" in mk_adj_mat. The GN2 handshake now replaces whitespace inside name-row names with underscores before the separator conversion. * GN2 may also supply its own session key via `$_POST["My_key"]` and later deep-link to `executionprogress_default.php?My_key=`. ## UX: feedback during structure learning * **Status page instead of a blank browser** (`1de3cbe0`): the `execute_bn_gom.php` runner used to output nothing for the whole run (~12 s+). It now immediately shows a status page — "Structure learning is running. Forwarding to the network diagram when done…" — flushed before the work starts. * **Click-here fallback** (`1de3cbe0`): the status page links straight to the network diagram in case the automatic forward is blocked. * **Pipeline failures are shown, not blank** (`e18ba415`): when `run.sh`/`run_octave` fails, the error (command, exit code, stdout, stderr) is rendered on the status page with a link back to the upload page, instead of an uncaught RuntimeException killing the page mid-flight. ## Bug fixes (PHP 8 compatibility and robustness) * Guard `valid_keyval()`/`valid_input()` against null input (PHP 8.1 deprecation) (`4825da8e`); guard fopen() results before fwrite/fprintf/fclose (`9cae7ea1`); guard arithmetic on non-numeric file/shell values (`a1f64bda`). * Fix undefined `$TextinFile`/`$keyval` in bn_after_upload_gom.php (`9f820106`) and an orphaned fwrite() that crashed runtime_check.php on PHP 8 (`55909cd3`). * Misc fixes: missing key check (`47e8ac23`), path fix (`6254c4b4`), variable moved to first use (`02c5eef4`). * Shell scripts made executable (`f7d65f6a`) — the php pages invoke them by name. ## Build system * **pkgconf finds libRmath** (`ea3796d3`): `build.sh` no longer hard-codes paths; under Guix the package passes the rmath-standalone library path explicitly. * **`runTerminalCommand()` wrapper** (`d1e617f3`, `a08f24c6`): a thin proc_open wrapper (with argument escaping) replaces scattered shell_exec/system calls and raises RuntimeExceptions on failure. ## Vendored third-party code removed * **Vendored javascript libraries and prebuilt binaries removed from the repository** (`00e5b05f`): the js libs (jquery, d3, cytoscape, FileSaver, font-awesome, ...) come from Guix packages and are symlinked in at package build time; the prebuilt .o/.dll/.so binaries are recompiled from source by `build.sh`. 111 files / ~36k lines of vendored code dropped. ## Tests * `test/bnw_http_tests.py`: 26 checks covering the full user workflow (static pages → key validation → upload → octave preparation → C structure learning → network rendering), all six workflow.php examples, the GN2 handshake (server-side conversion, no JS echo, template spelling through the legacy `/sourcecodes/` action path), and the asset downloads. * `test/check_container_tools.sh`: verifies every unix tool BNW calls externally exists in the container environment. ## Internal * Ignore-list hygiene for build artifacts (`0a39376c`, `3c3f6c0d`). * Test-path documentation: no hardcoded factory paths; checkout roots and per-checkout overrides are environment variables (`4dd78e7b`). * `make` no longer exports a local-source hook; the tests always build the origin pinned in guix-bioinformatics (`dd8dc307`) — push to gn-bnw main (or update the pin) before testing source changes. * **Self-locating binaries in run.sh** (`f9d5c411`): `run.sh` cd'ed to `/var/lib/genenet/bnw` and ran `./network_score` etc. relative to that directory, depending on the activation having copied the binaries there — on production that copy was incomplete and every structure-learning run died with "./network_score: No such file or directory". run.sh now locates the binaries next to itself in the package store item (`$(dirname $0)/../var_lib_genenet_bnw`), so the flow works regardless of what `/var/lib/genenet/bnw` contains. * **`/version.php` deployment-validation endpoint** (via the Guix package): serves JSON with the package version and the exact gn-bnw commit the deployment was built from (`{"name":"bnw","version":"1.0.0","commit":"..."}`). After a redeploy, `curl https:///version.php` shows what is running — no more inferring the version from error messages. ## Input validation and informative failures (this release) The GN2 entry point (`bn_genenet.php`) now validates the posted payload before any file is written or octave runs, and fails with a plain-English problem list instead of a deep octave crash. Found by an exhaustive small-payload search (`test/fuzz_bnw.py`, 75 cases over sizes, data types, special characters, missing data and separators): - header-only payload (traits with no sample values loaded in GN2) -> "No sample data rows were received ..." (was: "Could not open continuous_input.txt" deep in octave) - ragged rows, single variable, single sample, non-numeric cells, empty payload, all-NA data -> each with a specific message - a variable whose every entry is NA -> rejected (prep crashed on it) - every row carrying an NA -> rejected with an explanation (after prep's row removal nothing remains: "fscanf: invalid stream number") - duplicate variable names (same trait from two data sets) -> renamed with a note instead of crashing - CRLF line endings and redundant row separators (`;` plus newline) are normalized instead of corrupting the data files - `runtime_check.php` (`exe_time`) no longer fatals with "Unsupported operand types: int + string" when prep did not produce its tables - the structure-learning failure page now appends plain-English hints for the recurring octave failure modes (unlaid-out/cyclic networks, memory, missing prep files) Validated end-to-end on production: the 75-case input-space sweep produces zero crashes and zero blank pages; the header-only case that prompted this work (traits with no sample values loaded in GeneNetwork) returns the explanation above. One genuine limitation surfaced: with exactly two variables the scorer can prefer a cyclic network that the layout engine cannot draw (`sources(0): subscripts must be ...`); the failure page explains this — the deeper fix lives in the C scoring code. `test/bnw_http_tests.py` is now 39 checks (t22–t33 added for the validation layer); `test/fuzz_bnw.py` is the parameter-space fuzzer.