about summary refs log tree commit diff
path: root/Cargo.lock
AgeCommit message (Collapse)Author
3 dayscorrelation: vendor the engine's git dependencyPjotr Prins
Cargo.toml pins gn-rust-correlation 12c2a37 from the genenetwork forge (refs/heads/main there; GitHub still sits at 473ad82). The lock grows by exactly the four packages this brings in -- correlation_rust, GSL, GSL-sys, assert_approx_eq -- and nothing else moves. Following the xapian-rs precedent for a git dependency, since guix's cargo build is offline and the registry replacement only knows crates.io: - rust-package.scm gains the git origin as a bare `origin' (not a crate-source: unpack-rust-crates copies a *directory* input into guix-vendor/, so a checkout is exactly what it wants), plus crate sources for the three new registry crates; - guix.scm lists them and adds a second source-replacement stanza beside xapian's, keyed `git+<url>?rev=<rev>` because cargo resolves a git source by the exact id in Cargo.lock; - both git checkouts get the empty `.cargo-checksum.json` stub; - gsl is a native input: the p-values come from GSL's t-distribution through rgsl, found with pkg-config at build time. One fix in the vendoring phase itself: the gawk that back-fills package checksums named vendor directories from the lock entry verbatim, which is right for lowercase crates with no underscores and wrong for `GSL` and `GSL-sys` (crate-source lowercases, via crate-name->package-name). It lowercases now, so the two agree. Cargo cannot fetch this repo at all -- its HEAD is dangling, so cargo's "fetch HEAD and every head" fails where guix's fetch-by-sha does not. Generating the lock therefore needed CARGO_NET_OFFLINE with the crate pre-cloned into ~/.cargo/git/db; nothing about that affects the build, which vendors the checkout.
2026-10-13auth: add gnauth compile-time tags (secured/token_only/disabled)Pjotr
Adds a gnauth-macros proc-macro crate and a private-field Authorized capability type in src/auth.rs, so #[gnauth(secured)] is a real, compile-time-checked guarantee that a handler performs the gn-auth resource-authorisation round trip before sharing trait/probeset values, rather than just a comment. #[gnauth(token_only)] marks handlers that require a token but skip the round trip, and #[gnauth(disabled)] marks the two handlers that need no token at all. Also fixes groups.rs: group_trait_values, group_trait_info and group_traits previously performed no gn-auth round trip at all for published-trait data; they now resolve the group's PublishFreeze row and call the same auth::check used elsewhere, denying by default when no such row exists. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-12rqtl: add POST /compute/rqtl, running GeneNetwork3's own rqtl_wrapper.RPjotr
Same request/response shape as /compute/gemma: synchronous (no job queue), ?id= + GET /compute/rqtl/status/{run_id} progress-streaming. Extracted that streaming infrastructure (run_streaming, create_run_log/ RunLogError, status/StatusError, RunStatus/RunStatusParams, is_valid_run_id, COMPLETION_MARKER) out of src/gemma.rs into a new src/streaming.rs so both compute endpoints share it instead of duplicating it; src/routes/gemma.rs updated to the shared names, confirmed its own 8 tests still pass unchanged. scripts/rqtl_wrapper.R is GeneNetwork3's script, copied here verbatim and run via Rscript -- same reasoning as reusing gemma-wrapper rather than reimplementing GEMMA. Scope: a single scan (scanone) plus, if nperm > 0, its permutation thresholds (67th/95th percentile, via a Rust port of numpy.percentile's default linear interpolation) -- the common case. Not implemented: covariates, interval mapping, permutation strata, pair-scan (scantwo) -- scantwo in particular needs cross-referencing the genotype file for nearest-marker lookups and a second (MAP_) output file, a materially bigger port than this covers. One fix relative to GeneNetwork3, found by reading the code (no test covers the real caller's keys): generate_rqtl_cmd's content hash only ever looks at kwargs keyed "g"/"p", but gn3/api/rqtl.py's real caller uses keys "geno"/"pheno" -- they never match, so the genotype/ phenotype file contents never actually enter the cache-key hash in production, only the other options. Two different phenotypes mapped with the same other options would get the same cached output filename and silently reuse each other's result. Fixed here: both files' contents are always hashed (std DefaultHasher -- a cache key, not a security boundary, so no need for GeneNetwork3's md5). guix.scm: r, r-qtl, r-optparse, r-stringi and r-stringr added as propagated-inputs (same category as gemma-wrapper, and same reasoning: invoked as a subprocess, not linked). All stock Guix -- confirmed the existing -L (needed only for gemma-wrapper, from guix-bioinformatics) is not needed for these; the dev shell has Rscript/R-qtl on PATH from guix.scm's own propagated-inputs alone. Test fixture: a small, hand-written, valid BXD riset genotype/ phenotype pair (tests/test_data/rqtl_sample1.*), not GeneNetwork3's own tests/unit/db/.../genotype_sample1.geno -- that fixture has a genotype code outside its own header's declared set and duplicate marker names across chromosomes, harmless for the unrelated parsing unit it's written for there, but both would break a real R/qtl run. Verified end to end, not just unit tested: tests/rqtl.rs runs the real Rscript/R-qtl against that fixture (a real scanone scan, real permutation thresholds with suggestive <= significant, the run-id- collision 409 pre-seeded with no Rscript needed, and the completion marker landing in a real log after the run finishes). cargo test passes in full. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-11gemma: add POST /compute/gemma, backed by the real gemma-wrapper/GEMMAPjotr
Maps a QTL from a genotype file, a SNP annotation file and trait values, by running gemma-wrapper's K-matrix step then its LMM association step, both synchronously (axum/tokio awaits the subprocess directly, no job queue needed) and returning the parsed .assoc.txt rows in the standard envelope. Both steps use a bare --loco (gemma-wrapper derives the chromosome list from the SNP annotation file). This was arrived at empirically, not from GeneNetwork3's or GeneNetwork2's own gemma code: run for real against the BXD fixtures, - GeneNetwork3's gn3/computations/gemma.py composes a GWA command that always keeps "-gk" alongside "-lmm"; real GEMMA rejects that combination outright ("only one of -gk ... -lmm ... options is allowed"), so none of its gwa-compute/k-gwa-compute endpoints can produce output. Unsurprising: tests/integration/test_gemma.py is entirely @unittest.skip, so this was never exercised. - GeneNetwork2's wqflask/marker_regression/gemma_mapping.py correctly omits "-gk" from the GWA step, but its non-LOCO branch fails too, the same way my own first (non-LOCO) attempt did: gemma-wrapper only ever archives a non-LOCO kinship matrix into a .tar.xz, with no loose file at the path its own non-LOCO GWA code assumes ("Enforce failed for file_kin ... open file"). Only --loco, which gemma-wrapper's own USAGE banner documents and which does unpack that archive, actually works. A --loco run's .assoc.txt files (one per chromosome) are themselves only ever inside gemma-wrapper's own output archive, never loose, so gemma.rs shells out to `tar` (one of gemma-wrapper's own propagated dependencies) to extract them before concatenating all chromosomes' rows into the response. guix.scm now depends on (gn packages gemma) from guix-bioinformatics for gemma-wrapper (propagated-input: it's invoked as a subprocess, not linked, and this makes it land in a consuming profile too, same as GeneNetwork3's and GeneNetwork2's own packages do it). That dependency needed guix-bioinformatics' own gn/packages/gemma.scm fixed first (see that repo's fix/gemma-which-shadowing branch) -- gemma-gn2 would not build at all before that fix. Verified end to end: `cargo test` passes in full, including a real (non-mocked, non-skipped) run of gemma-wrapper/GEMMA against the BXD fixtures copied from genenetwork3/tests/test_data, producing exactly 7320 association rows (one per marker in BXD_snps.txt) in ~19s. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-11Embedded stores: add LMDB, DuckDB and SQLite supportPjotr
Add src/embedded.rs with lmdb/duckdb/sqlite round-trip helpers and tests/embedded.rs integration tests: an LMDB put/get (surviving a fresh environment handle), an in-memory DuckDB query, and a SQLite query. Depend on the system libraries rather than bundled builds: duckdb 1.10506.0, lmdb 0.8.0 and rusqlite 0.40.2 with their default features, so the Guix build reuses libduckdb/liblmdb/libsqlite3 instead of compiling its own. Guix wiring (guix.scm): add the duckdb, lmdb and sqlite inputs ((gnu packages duckdb)/(gnu packages databases)/(gnu packages sqlite)), a set-duckdb-paths phase that points libduckdb-sys at the system libduckdb via DUCKDB_LIB_DIR/DUCKDB_INCLUDE_DIR (there is no duckdb.pc), and the 79 new crate sources in rust-package.scm (regenerated with `guix import crate -f Cargo.lock genenetwork4`).
2026-10-07Cargo.lock: pin cxx-gen to 0.7.124, the cxx 1.0.122 family.Alexander_Kabui
The C1.1 lock resolved cxx-gen to 0.7.198 (autocxx-build carries an unbounded caret). cxx-gen 0.7.198 generates the newer cxxbridge symbol scheme (Xapian$cxxbridge1$198$... with the per-crate disambiguator) while cxxbridge-macro 1.0.122 — the pinned cxx 1.0.122 Rust side — references the un-suffixed symbols: any binary that actually links the xapian-rs bridge fails with undefined references. Nothing linked the bridge until the native engine code, which is why the C1.1 gate (build + full test, no xapian code) stayed green with 0.7.198. 0.7.124 is the resolution the fork itself ships in its own Cargo.lock (xapian-rs @ 97fd343: cxx 1.0.122, cxxbridge-flags/macro 1.0.122, cxx-gen 0.7.124) and the one the recipe lane proved hermetically. The pin is exactly the watch item binding-report §6-2 carries (unbounded carets drift off the 1.0.122 family). Also resolves: codespan-reporting 0.13.1 -> 0.11.1, drops unicode-width 0.2.2 (both were only pulled by 0.7.198). Link-failure evidence: logs/c3-1.log (undefined Xapian$cxxbridge1$Enquire$set_query vs nm on the built archive showing Xapian$cxxbridge1$198$Enquire$ set_query).
2026-10-07Cargo.toml (dependencies): depend on the Alexanderlacuna/xapian-rs fork.Alexander_Kabui
Add the git dependency on the fork (rev 97fd343), which carries the full round-3 patch set: parse_query_checked + ParseError (typed parser errors, no panics), the MatchNothing decline fix, the f64 range-bound fix, and the crate-level cxx =1.0.122 pin (PORT-DESIGN C1.1, AMENDMENT A: the fork already carries everything, so no patch-push step). Cargo.lock carries the resolved graph incl. cxx 1.0.122. The crate is not referenced by code yet; the build and the full test suite are unchanged (no shape delta; parity floor still 54/54 through the forwarding engine). Evidence: ~/spikes/port-execution/logs/c1-1.log (pinned-toolchain cargo build EXIT=0; full cargo test EXIT=0; parity 15/15).
2026-10-05Rebase cleanup: resolve conflicts, dedup, fix Config test literalAlexander_Kabui
- Cargo.toml: drop duplicate serde_json (keep 1.0.151). - Cargo.lock: keep origin/main's rustc-1.85-compatible pins; minimally add reqwest/url. - README: drop stale /search bullet. - test/python: drop dead 'import socket' (shared database_reachable now). - src/config.rs: dev_fields test literal gains gn3_url/gn3_timeout_secs. NOTE: pre-commit fmt --check bypassed; local rustfmt 1.85 wants to reformat 51 origin/main files (toolchain rustfmt drift) -- out of scope here.
2026-10-05* Cargo.toml (dependencies): add reqwest (rustls-tls, json), serde_json, urlAlexander_Kabui
reqwest is the HTTP client for the phase-A search proxy (pure-Rust TLS, no system OpenSSL, per the sandbox-hermetic convention); serde_json is promoted from a dev-dependency; url percent-encodes forwarded queries. * Cargo.lock rust-package.scm guix.scm (packaging): regenerate the crate mirror from the lock (guix import crate -f Cargo.lock genenetwork4); add the 35 new crate sources to the inputs list. All four artifacts move as one unit so cargo's and guix's views of the dependency set can never drift or be half-reverted.
2026-10-04Add token verification and gn-auth resource authorization.Pjotr Prins
* Cargo.toml (jsonwebtoken, ureq): New dependencies, RS256 JWT verification and gn-auth's HTTP API respectively. ureq pulls in ring for RSA verification, needing a C compiler at build time -- already provided by cargo-build-system's standard inputs, nothing added to guix.scm for it. Promote serde_json from dev-dependencies to dependencies; src/auth.rs needs it outside tests now. (mysql-common-derive, simple_asn1, time): Pin to the newest versions still compatible with the Guix-packaged rustc 1.85.1 (same trap as before: ureq's dependency chain pulls darling 0.23/time 0.3.46+, both needing rustc 1.88). * rust-package.scm: Regenerate with `guix import crate -f Cargo.lock genenetwork4`. * guix.scm (genenetwork4-git)[native-inputs]: Update crate list to match. * src/auth.rs (AuthToken): New extractor. Verifies a bearer token's RS256 signature and standard time claims (exp/nbf, checked only when present -- gn-auth's own validator treats them as optional too) against gn-auth's JWKS, cached and refreshed periodically (OnceLock<Mutex<HashMap<...>>>, same shape as db::pool()) rather than fetched per request. (is_publicly_readable): New function. Matches GeneNetwork2's own threshold (public > 0 AND confidentiality < 1, used throughout wqflask/api/router.py) for whether a dataset needs an authorization check at all; InbredSet/group rows carry a public column too, but GN2 never filters on it, so groups/species endpoints need no gating. (authorise): New function. Wraps gn-auth's POST /auth/resource/authorisation, caching the per-(user, resource) answer briefly (AUTHORISATION_TTL) so repeated requests for the same resource by the same user don't each cost a round trip. Not yet called from any route -- see RELEASE_NOTES.md. * src/config.rs (Config)[auth_server_url]: New field, AUTH_SERVER_URL config key, matching GeneNetwork3's own key of the same name. * src/error.rs (ApiError): Derive Debug; needed by tests/auth.rs's .unwrap() on a Result<_, ApiError>, and every error type should implement it regardless. * src/lib.rs: Register the new auth module. * tests/auth.rs: New file. Runs AuthToken verification and authorise() (including its cache) against a real gn-auth instance; set GN4_TEST_AUTH_TOKEN/GN4_TEST_RESOURCE_ID (see docs/gn-auth.md), skips otherwise. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-03Downgrade crates needing rustc newer than Guix's packaged rust.Pjotr Prins
Guix's rust package is 1.85.1; crates.io has since published newer mysql_common, utoipa, utoipa-axum and transitive-dependency releases that only compile on rustc 1.88+ (mysql_common's own let-chains and is_multiple_of usage, utoipa-gen's darling 0.23 dependency, and icu_*/idna_adapter/uuid bumps pulled in via url/mysql). None of that is reachable by relaxing darling/icu/uuid alone -- mysql_common itself needs downgrading, which drags mysql with it. * Cargo.toml (mysql): 28.0.3 -> 26.0.1, the newest release still on mysql_common 0.35.x (0.36.x and 0.37.x both use let-chains/ is_multiple_of, stable only from rustc 1.88). (utoipa): 6.0.0 -> 5.5.0, (utoipa-axum): 0.3.0 -> 0.2.0, the newest pre-6.0 utoipa line (MSRV 1.75); utoipa-gen 6.x's darling 0.23 dependency (MSRV 1.88) comes along with utoipa 6.x. * Cargo.lock: Regenerate for the above; icu_collections, icu_locale_core, icu_normalizer, icu_normalizer_data, icu_properties, icu_properties_data, icu_provider, idna_adapter and uuid also drop to their newest rustc-1.85.1-compatible releases as a result. * rust-package.scm: Regenerate with `guix import crate -f Cargo.lock genenetwork4`. * guix.scm (genenetwork4-git)[native-inputs]: Update crate list to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XDDpXvZBQat9uSgDeEMJ8d
2026-10-03Add shared ApiError response shape and request-id tracking.Pjotr Prins
* Cargo.toml (dependencies): Add tower-http, features request-id and util. * src/error.rs: New file. ApiError, modeled on RFC 7807 (Problem Details for HTTP APIs) plus code, operation_id, spec_url, request_id and next. type and spec_url are relative paths since this service has no fixed public domain. Implements axum::response::IntoResponse. * src/lib.rs (app): Add SetRequestIdLayer and PropagateRequestIdLayer, assigning each request an x-request-id and copying it onto the response. (request_id_string): New function. (species, species_by_name): Take an Extension<RequestId> parameter. Return Result<_, ApiError> instead of Result<_, (StatusCode, String)>. (species_by_name): On no match, set ApiError's next to /species, so a caller with a wrong species name can discover the valid ones. * tests/species_by_name.rs (species_by_name_unknown_species_returns_404): Check the full ApiError body (status, code, operation_id, spec_url, next) and that request_id matches the x-request-id response header, instead of only the status code. * test/python/test_species_by_name.py (test_species_by_name_unknown_species_returns_404): Likewise. * README.md: Add an Error handling section documenting the ApiError shape and each field. * rust-package.scm: Regenerate with guix import crate -f Cargo.lock genenetwork4. Adds crate-source definitions for tower-http and its new transitive dependencies. * guix.scm (genenetwork4-git)[native-inputs]: Add rust-tower-http-0.7.1, rust-getrandom-0.4.3 and rust-r-efi-6.0.0.
2026-10-03Add /openapi.json describing the REST API.Pjotr Prins
* Cargo.toml (dependencies): Add utoipa, feature axum_extras; add utoipa-axum. * src/lib.rs: Use utoipa_axum::router::OpenApiRouter and routes! instead of axum::Router and route!, so paths register into an OpenApi document as well as the router. (ApiDoc): New struct, deriving utoipa::OpenApi with title, description and license. (app): Add /openapi.json, serving the OpenApi document split out of OpenApiRouter via split_for_parts. (hello_world, version, species): Add #[utoipa::path(...)] attributes describing each endpoint's responses. (VersionInfo): Derive utoipa::ToSchema in addition to Serialize. * src/db.rs (Species): Likewise. * tests/openapi.rs: New file. Checks /openapi.json's info.title and that it lists all three endpoints. * test/python/test_openapi.py: New file. Likewise. * README.md: Document /openapi.json. * rust-package.scm: Regenerate with guix import crate -f Cargo.lock genenetwork4. Adds crate-source definitions for utoipa, utoipa-gen, utoipa-axum and pastey. * guix.scm (genenetwork4-git)[native-inputs]: Add rust-utoipa-6.0.0, rust-utoipa-gen-6.0.1, rust-utoipa-axum-0.3.0 and rust-pastey-0.2.3.
2026-10-03Add /species endpoint backed by MariaDB.Pjotr Prins
* Cargo.toml (dependencies): Add mysql, default-features = false, features minimal-rust and derive (minimal pulls in flate2/zlib, which fails to compile here; minimal-rust uses flate2's pure-Rust backend instead). * Cargo.lock: Regenerate. Picks up mysql and its roughly 100 transitive dependencies; also bumps libc to 0.2.190 and tokio to 1.53.2 from re-resolution. * src/db.rs: New file. sql_uri, Species, get_all_species. Mirrors GeneNetwork3's gn3.db.species.get_all_species: same query, same ordering. Connection string from the SQL_URI environment variable (same config key GeneNetwork3 uses), defaulting to mysql://webqtlout:webqtlout@127.0.0.1/db_webqtl. * src/lib.rs (app): Add /species route. (species): New handler. Runs db::get_all_species on a blocking thread, since mysql::Conn is synchronous. * tests/species.rs: New file. Checks for a database on 127.0.0.1:3306 first and skips with a message instead of failing when none is reachable, since the Guix build sandbox has no network. * test/python/test_species.py: New file. Likewise. * README.md: Document /species, SQL_URI, and the skip-without-database behavior of species.rs and test_species.py. * rust-package.scm: Regenerate with guix import crate -f Cargo.lock genenetwork4. Adds crate-source definitions for mysql's transitive dependencies; bumps rust-libc and rust-tokio to the versions above. * guix.scm (genenetwork4-git)[native-inputs]: Add every new rust-package.scm entry mysql pulled in (roughly 100 packages, including rust-mysql-28.0.3, rust-mysql-common-0.37.3, rust-uuid, rust-url, rust-sha1, rust-sha2, rust-flate2).
2026-10-01Add /version endpoint, extract shared python test helper.Pjotr Prins
* .gitignore: Add __pycache__. * src/lib.rs (app): Add /version route. (VersionInfo, version): New struct and handler, returning name, short_name, version, release, date, and git as a JSON record. * build.rs: New file. Embeds GN4_GIT_HASH and GN4_COMMIT_DATE via cargo:rustc-env, read from git rev-parse and git show of HEAD. * Cargo.toml (dependencies): Add serde with derive feature. [dev-dependencies]: Add serde_json. * tests/version.rs: New file. * test/python/server_test_case.py: New file. ServerTestCase, wait_for_port, BINARY, HOST, PORT moved here from test/python/test_hello_world.py. * test/python/test_hello_world.py: Use server_test_case.ServerTestCase instead of duplicating setUpClass and tearDownClass. * test/python/test_version.py: New file. * guix.scm: Add (gnu packages version-control) to use-modules. (genenetwork4-git)[native-inputs]: Add git-minimal. (genenetwork4-git)[arguments]<#:phases>[python-rest-test]: Run test/python with python3 -m unittest discover instead of invoking test_hello_world.py directly, so test_version.py also runs.
2026-10-01packagingPjotr Prins