1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
|
# BNW release notes
Changes since `e9a320d0` (*Update home.php*, the last pre-Guix upstream
commit), summarized for users and deployers. 34 commits, 2021-10-11 →
2026-10-03.
## Deployment / Guix packaging
* **Guix system container and VM** (`7f460113`): BNW deploys as a Guix
system container (`gn-machines/genenetwork/containers/bnw-container.scm`)
and VM. The package definition lives in
`guix-bioinformatics/gn/packages/bnw.scm`, pinned to commits of this
repository and fetched from git.genenetwork.org.
* **Container test cycle** (`7b6a8ccf`, `f76f79b8`): `make` builds the
container from the pinned origin, starts it through the
run-guix-system-container daemon, runs the HTTP test suite, and stops
it again; `make run` leaves it up, `make clean` removes artifacts.
Checkout paths are overridable (`base`, `GUIX_BIOINF`, ...).
* **Anubis AI-bot firewall support** (in gn-machines): the web UI is
challenge-protected, while the GeneNetwork workflow endpoints, the
static assets and the .htm tutorial pages are exempt — the full
modeling flow runs without ever hitting a challenge.
* **Pinned container SSH keys** (`10ae4902`, `74f922e3`): host and admin
keys live outside the repository in the building user's `$HOME`, so
every container presents the same SSH host key.
## GeneNetwork (GN2) integration
* **Single-POST handshake, XSS fixed** (`8380798b`, `75761501`): GN2
posts `My_Gennenet` (commas = columns, semicolons = rows) and the
conversion now happens **server-side** in PHP, returning the
prepared-data page in one request. The previous release echoed the
raw payload into a JavaScript string (`genenet_str = "<?php echo ... ?>"`),
an XSS hole on public deployments — that echo is gone.
* **Accept the GN2 field spelling** (`3616c964`): GN2's template posts
`My_Genenet` (one 'n'), while BNW historically read `My_Gennenet`.
Both spellings are accepted now.
* **Sanitize trait names with spaces** (`18dcbf7b`): GN2 trait display
names often contain spaces ("Age at death"); BNW's file format does
not allow them (the octave parsers split on whitespace), so spaced
names corrupted the header and structure learning died with
"A(0,_): subscripts must be ... integers" in mk_adj_mat. The GN2
handshake now replaces whitespace inside name-row names with
underscores before the separator conversion.
* GN2 may also supply its own session key via `$_POST["My_key"]` and
later deep-link to `executionprogress_default.php?My_key=<key>`.
## UX: feedback during structure learning
* **Status page instead of a blank browser** (`1de3cbe0`): the
`execute_bn_gom.php` runner used to output nothing for the whole
run (~12 s+). It now immediately shows a status page — "Structure
learning is running. Forwarding to the network diagram when
done…" — flushed before the work starts.
* **Click-here fallback** (`1de3cbe0`): the status page links straight
to the network diagram in case the automatic forward is blocked.
* **Pipeline failures are shown, not blank** (`e18ba415`): when
`run.sh`/`run_octave` fails, the error (command, exit code, stdout,
stderr) is rendered on the status page with a link back to the
upload page, instead of an uncaught RuntimeException killing the
page mid-flight.
## Bug fixes (PHP 8 compatibility and robustness)
* Guard `valid_keyval()`/`valid_input()` against null input (PHP 8.1
deprecation) (`4825da8e`); guard fopen() results before
fwrite/fprintf/fclose (`9cae7ea1`); guard arithmetic on non-numeric
file/shell values (`a1f64bda`).
* Fix undefined `$TextinFile`/`$keyval` in bn_after_upload_gom.php
(`9f820106`) and an orphaned fwrite() that crashed runtime_check.php
on PHP 8 (`55909cd3`).
* Misc fixes: missing key check (`47e8ac23`), path fix (`6254c4b4`),
variable moved to first use (`02c5eef4`).
* Shell scripts made executable (`f7d65f6a`) — the php pages invoke
them by name.
## Build system
* **pkgconf finds libRmath** (`ea3796d3`): `build.sh` no longer
hard-codes paths; under Guix the package passes the
rmath-standalone library path explicitly.
* **`runTerminalCommand()` wrapper** (`d1e617f3`, `a08f24c6`): a thin
proc_open wrapper (with argument escaping) replaces scattered
shell_exec/system calls and raises RuntimeExceptions on failure.
## Vendored third-party code removed
* **Vendored javascript libraries and prebuilt binaries removed from
the repository** (`00e5b05f`): the js libs (jquery, d3, cytoscape,
FileSaver, font-awesome, ...) come from Guix packages and are
symlinked in at package build time; the prebuilt .o/.dll/.so
binaries are recompiled from source by `build.sh`. 111 files /
~36k lines of vendored code dropped.
## Tests
* `test/bnw_http_tests.py`: 26 checks covering the full user workflow
(static pages → key validation → upload → octave preparation → C
structure learning → network rendering), all six workflow.php
examples, the GN2 handshake (server-side conversion, no JS echo,
template spelling through the legacy `/sourcecodes/` action path),
and the asset downloads.
* `test/check_container_tools.sh`: verifies every unix tool BNW calls
externally exists in the container environment.
## Internal
* Ignore-list hygiene for build artifacts (`0a39376c`, `3c3f6c0d`).
* Test-path documentation: no hardcoded factory paths; checkout roots
and per-checkout overrides are environment variables (`4dd78e7b`).
* `make` no longer exports a local-source hook; the tests always build
the origin pinned in guix-bioinformatics (`dd8dc307`) — push to
gn-bnw main (or update the pin) before testing source changes.
* **Self-locating binaries in run.sh** (`f9d5c411`): `run.sh` cd'ed
to `/var/lib/genenet/bnw` and ran `./network_score` etc. relative to
that directory, depending on the activation having copied the
binaries there — on production that copy was incomplete and every
structure-learning run died with "./network_score: No such file or
directory". run.sh now locates the binaries next to itself in the
package store item
(`$(dirname $0)/../var_lib_genenet_bnw`), so the flow works
regardless of what `/var/lib/genenet/bnw` contains.
* **`/version.php` deployment-validation endpoint** (via the Guix
package): serves JSON with the package version and the exact
gn-bnw commit the deployment was built from
(`{"name":"bnw","version":"1.0.0","commit":"..."}`). After a
redeploy, `curl https://<host>/version.php` shows what is running
— no more inferring the version from error messages.
## Input validation and informative failures (this release)
The GN2 entry point (`bn_genenet.php`) now validates the posted payload
before any file is written or octave runs, and fails with a plain-English
problem list instead of a deep octave crash. Found by an exhaustive
small-payload search (`test/fuzz_bnw.py`, 75 cases over sizes, data
types, special characters, missing data and separators):
- header-only payload (traits with no sample values loaded in GN2)
-> "No sample data rows were received ..." (was: "Could not open
<key>continuous_input.txt" deep in octave)
- ragged rows, single variable, single sample, non-numeric cells,
empty payload, all-NA data -> each with a specific message
- a variable whose every entry is NA -> rejected (prep crashed on it)
- every row carrying an NA -> rejected with an explanation (after
prep's row removal nothing remains: "fscanf: invalid stream number")
- duplicate variable names (same trait from two data sets) -> renamed
with a note instead of crashing
- CRLF line endings and redundant row separators (`;` plus newline)
are normalized instead of corrupting the data files
- `runtime_check.php` (`exe_time`) no longer fatals with "Unsupported
operand types: int + string" when prep did not produce its tables
- the structure-learning failure page now appends plain-English hints
for the recurring octave failure modes (unlaid-out/cyclic networks,
memory, missing prep files)
Validated end-to-end on production: the 75-case input-space sweep
produces zero crashes and zero blank pages; the header-only case that
prompted this work (traits with no sample values loaded in GeneNetwork)
returns the explanation above. One genuine limitation surfaced: with
exactly two variables the scorer can prefer a cyclic network that the
layout engine cannot draw (`sources(0): subscripts must be ...`); the
failure page explains this — the deeper fix lives in the C scoring code.
`test/bnw_http_tests.py` is now 39 checks (t22–t33 added for the
validation layer); `test/fuzz_bnw.py` is the parameter-space fuzzer.
|