about summary refs log tree commit diff
path: root/info_files
AgeCommit message (Collapse)Author
6 daysbin/bnw: headless network learning, and parity with the web application HEAD mainPjotr
BNW's computation does not need the web application: run.sh drives three C programs over files in a per-run directory, and everything above that is file assembly and rendering. bin/bnw does the same pipeline -- network_score, the k-best search, structure -- in a private working directory, and reports one row per candidate edge with its posterior and whether it survived the threshold. The reference for all of it is the running bnw-container, not a memory of it: its six continuous traits example was run through the web application (bn_genenet.php -> execute_bn_gom.php) and the resulting files kept as fixtures in test/golden/. test/parity_check.py compares our continuous_input.txt and structure_input.txt against those, and the driver's own stdout against the file. Today: byte-identical, 5 edges. Four things that run settled, each of which had been guessed wrong or not at all: * continuous_input.txt is header, then a TYPE LINE (1 = continuous, n = n-level discrete), then data. network_score reads the type before the data, and the web application's file is one line longer than the matrix because of it. Without the line the first data row shifts and network_score segfaults. * structure exits with an uninitialised status while writing a correct file. The web application ignores the status; the driver now judges it by its output and only fails if the file is missing. * get_kbest_nets.cc used SGI's <ext/hash_set>, which modern libstdc++ no longer has. It now uses std::unordered_set with an explicit hasher (specialising std::hash for a pointer type is not allowed), and build.sh compiles the C with -std=gnu89 because the sources predate C99. * our run leaves postProbEachEdge.txt with per-edge posteriors; the web application's flow leaves none behind, so the posteriors are information the endpoint can report that the web app never showed. Exit codes are part of the contract GN4 will code against: 0 ok, 2 bad input, 3 the computation failed, 77 the binaries are not installed (a conventional skip, so a test suite can run where BNW is not). Build output is gitignored. AI plan and status updated in info_files/gn4-integration-plan.md; README gained a "Headless driver" section.
6 daysPlan: a headless driver here, a call-out in GN4, and the parameters measuredPjotr
Two corrections from the owner, both structural, so the plan is rewritten around them rather than patched. The split: BNW's knowledge stays in this repository. Phase 1 becomes a headless driver with a contract GN4 can code against -- `bnw learn --input matrix.tsv --maxparents 4 --k 1 --tiers 3 --threshold 0.5 [--ban] [--white] [--format json|tsv] [--svg] [--workdir] [--keep]`, stdout as one edge per line plus a header naming the parameters and version, exit 2 for bad input and 3 for a failed computation -- with a private work directory per invocation instead of /var/lib/genenet/bnw, the parameters as arguments instead of <key>*.txt files, and no PHP, web server or /var/www. Acceptance is unchanged: byte-identical structure_input.txt and postProbEachEdge.txt against the web app's own answer. GN4's side is then a call-out and nothing more: BNW_CMD in config, a matrix in a temp file, one subprocess, edges parsed out. No knowledge of run.sh, of the C programs or of the <key> layout enters GN4, so they can change without a GN4 release. The guix.scm dependency lands last, once the driver works -- gn-bioinformatics/gn/packages/bnw.scm already pins this repository. The parameters are now measured rather than guessed. GN2 sends none: send_to_bnw.py builds only the payload (traits' shared samples, comma-joined rows, semicolon-terminated, My_Genenet) and the user picks parameters in BNW's own settings page afterwards. BNW's defaults are in sourcecodes/create_tiers_gom_part1.php, which writes parent.txt, k.txt and thr.txt from a | -joined form value: maxparents 4 (with an "All" option capped at variables-1), k 1 (no model averaging), tiers 3, edge-posterior threshold 0.5. A single-call endpoint has to expose all four, because the UI step that would set them is gone. Licence question settled by the owner: we own the code, so shipping the binaries in GN4's profile needs no third-party permission. The repository still has no LICENSE/COPYING and the C sources carry no header -- worth adding before GN4 distributes it.
6 daysPlan for embedding BNW in GeneNetwork4, and an ai-memory store herePjotr
BNW's computation is three C programs driven by run.sh over files in a per-run directory: network_score (local scores, links libRmath), k-best/src/data2netk_poster.sh (posterior per candidate edge) and structure (adjacency at a threshold). The PHP layer above them only builds those files and renders the result, so an endpoint does not need PHP and does not need the web app: it needs a private directory instead of the hardcoded /var/lib/genenet/bnw, the same three binaries, and the two files they leave behind -- structure_input.txt and postProbEachEdge.txt -- as its output. info_files/gn4-integration-plan.md sets out the phases against the engines GN4 has already embedded (bulklmm, qtl2, ravanan): golden files from the existing container first; a driver that runs the C programs without PHP and is byte-comparable to them; POST /compute/bnw taking a dataset+probes, group+ traits or a basket, gated by variables x samples rather than individuals; POST /plot/bnw rendering the SVG from the posted edges; then the part GeneNetwork actually wants -- module scores returned in the shape of trait values, so they plug straight into /compute/rqtl and /compute/gemma -- and tests that compare against the golden files. Risks named in order: combinatorics at 30 nodes, the global directory, whether the k-best search is deterministic (settle it before promising golden-file parity), discrete vs continuous networks (continuous first), and libRmath/ Python/Octave creep into the GN4 profile. Also initialises an ai-memory store for this repository (168 commits, 170 symbols) and ignores it in .gitignore, so the reasoning behind the endpoint accrues where the code lives.
2021-03-04Rename server_change_notes.txt to old_server_change_notes.txtziejd2
2021-03-04Update server_change_notes.txtziejd2
2021-03-02Update BNW_file_overview.txtziejd2
2021-03-02Update BNW_data_flow.txtziejd2
2021-02-23Update BNW_to_do.txtziejd2
2019-06-27Update server_change_notes.txtziejd2
2019-06-27Update BNW_to_do.txtziejd2
2019-06-27Update server_change_notes.txtziejd2
2019-06-27Update BNW_file_overview.txtziejd2
2019-06-27Update BNW_file_overview.txtziejd2
2019-06-27Update BNW_data_flow.txtziejd2
2019-06-27Update BNW_data_flow.txtziejd2
2019-06-27Version 1.22ziejd2
Adding new visualization options
2019-01-31Update server_change_notes.txtziejd2
2019-01-31Update BNW_to_do.txtziejd2
2019-01-31Added info_files folderziejd2