gn-transform-databases/transform, branch master
Data transformers for GeneNetwork (particularly to RDF)
http://git.genenetwork.org/gn-transform-databases/atom?h=master
2026-03-04T15:16:59+00:00
Revert "transform: add keyset pagination and streaming emit path"
2026-03-04T15:16:59+00:00
Munyoki Kilyungi
2026-03-04T15:16:59+00:00
urn:sha1:a5c1958d5b1abfaaa4ae2a2749ce7f8b0ed3055c
This reverts commit ba347362d64f9e634fad7f1aa8ab0a7753ff7a81. Causes
and endless loop. See:
https://ci.genenetwork.org/jobs/transform-genenetwork-database/99
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
transform: add keyset pagination and streaming emit path
2026-03-02T15:30:01+00:00
Munyoki Kilyungi
2026-03-02T15:19:27+00:00
urn:sha1:ba347362d64f9e634fad7f1aa8ab0a7753ff7a81
* transform/special-forms.scm (define-transformer): Add keyset
arguments and cursor return path; add SQL helpers for keyset
ordering/filtering; avoid po-alist allocation by streaming short
turtle emission.
(with-documentation): Add keyset pagination loop using per-input
cursor state and configurable chunk key field.
(emit-short-turtle): Emit triples in a single linear pass.
* examples/probesets-experiment-metadata.scm: Enable keyset pagination
and set composite chunk key fields for ProbeSetFreeze/ProbeSet join.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
Make url char separator optional in "string->identifier".
2026-02-16T10:33:25+00:00
Munyoki Kilyungi
2026-02-16T10:33:25+00:00
urn:sha1:d8345bc41d9c5567021e328734a16a1c8998f8d8
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
Add uuid function.
2026-02-16T10:33:12+00:00
Munyoki Kilyungi
2026-02-16T10:33:12+00:00
urn:sha1:89b70e6d35dd1044c4025616693d20fd4217df50
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
path-without-extension: utility function.
2026-02-10T07:18:54+00:00
Munyoki Kilyungi
2026-02-10T07:18:54+00:00
urn:sha1:8ed23ab93ea5f82866dc8e97224c2aafbe54ed11
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
Give "with-documentation" & "define-transformer" chunking ability.
2026-02-10T06:41:51+00:00
Munyoki Kilyungi
2026-02-10T06:41:51+00:00
urn:sha1:9dd449a2615fd18968c2dc84142f809241b498ad
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
Fix bug with when terminating a short turtle pair.
2026-02-09T14:49:01+00:00
Munyoki Kilyungi
2026-02-09T14:48:39+00:00
urn:sha1:576ace2d3155411c42d483d0091efb3ca3386b17
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
Don't create triple when string is blank.
2026-02-09T11:49:54+00:00
Munyoki Kilyungi
2026-02-09T11:49:54+00:00
urn:sha1:99d31d2be7bfc2e900db8fbd2c835ef7931360ba
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
Use shorter triple-forms for ttl files where possible.
2026-02-09T08:00:43+00:00
Munyoki Kilyungi
2026-02-09T08:00:43+00:00
urn:sha1:0e3a7305f45d70bb4173b6cd1e8d543957b6899a
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
Move blank-p to strings.scm.
2026-02-06T02:32:10+00:00
Munyoki Kilyungi
2026-02-06T02:32:10+00:00
urn:sha1:418568557b2b0d2ca977e5fe1594ed280b4f92f8
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>