| Age | Commit message (Expand) | Author |
|---|
| 2024-08-01 | Update case-attribute editing to use new authorisation. | Frederick Muriuki Muriithi |
| 2024-08-01 | Add `gn3.oauth2` module to handle authorisations•••Add the `gn3.oauth2` which will handle the authorisations.
It is intended to eventually replace the `gn3.auth` module which is a
left over from when the authorisation server was part of GN3.
| Frederick Muriuki Muriithi |
| 2024-08-01 | Stem group field regardless of case.•••* gn3/api/search.py (parse_boolean_prefixed_field): New function.
(parse_query): Use parse_boolean_prefixed_field for the group field.
| Arun Isaac |
| 2024-08-01 | Stem all the time.•••With the STEM_SOME, xapian does not stem query words that start with a
capital letter. Hence, we switch to the STEM_ALL strategy.
* gn3/api/search.py (parse_query): Set stemming strategy to STEM_ALL.
| Arun Isaac |
| 2024-07-23 | chore: add logging info | John Nduli |
| 2024-07-23 | chore: mypy and pylint fixes | John Nduli |
| 2024-07-23 | fix: resolve duplicate errors when updating data | John Nduli |
| 2024-07-23 | refactor: clean query for insert | John Nduli |
| 2024-07-23 | refactor: clean up insert query | John Nduli |
| 2024-07-23 | refactor: reorganize update_rif script to be more pythonic | John Nduli |
| 2024-07-23 | chore: fix pylint errors | John Nduli |
| 2024-07-23 | refactor: rename addRIf to update_rif_table.py | John Nduli |
| 2024-07-12 | Rename hash_rdf_graph -> md5hash_ttl_dir.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-07-12 | Use correct ttl-dir path when generating checksums.•••* scripts/index-genenetwork (is_data_modified): Provide directory
instead of specific ttl file.
(create_xapian_index): Ditto.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-07-12 | fix: remove .py extension for addRif to prevent pylint checks | John Nduli |
| 2024-07-12 | refactor: fix mypy and pylint errors | John Nduli |
| 2024-07-12 | feat: copy addRif script from genenetwork1•••original: https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/maintainance/addRif.py
Included some changes to make it python3 compatible
| John Nduli |
| 2024-07-08 | Pass output directory to R/qtl script instead of pulling it from the•••environment
Also fixes issue where the control marker keyword was wrong
| zsloan |
| 2024-07-05 | fix: return query error message from xapian | John Nduli |
| 2024-07-03 | Return a "-1" if the turtle directory does not exist.•••* scripts/index-genenetwork (hash_rdf_graph): Remove check for the
turtle directory.
(is_data_modified): Ditto.
(create_xapian_index): Ditto.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-07-03 | Generate a checksum for all the ttl files.•••* scripts/index-genenetwork (hash_generif_graph): Rename to
hash_rdf_graph. Generate a checksum of all the turtle files inside
the ttl directory that's the basis for the GN virtuoso graph.
(create_xapian_index): Rename hash_generif_graph -> hash_rdf_graph.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-07-03 | Add type-hints to hash_generif_graph.•••* scripts/index-genenetwork (hash_generif_graph): Add proper type
hints.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-07-03 | Refactor how the generif md5 sum is calculated and stored in XAPIAN.•••* scripts/index-genenetwork (hash_generif_graph): Build the generif
checksum by directly building it from the file.
(is_data_modified): Update how generif-checksums are verified.
(create_xapian_index): Update how generif-checksums are stored in
XAPIAN.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-07-03 | Use correct cache for RIF/Wiki entries.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-07-03 | feat: drop intermediate folders when running parallel xapian compact | John Nduli |
| 2024-07-03 | feat: add support for parallel xapian compact | John Nduli |
| 2024-07-03 | feat: index rif and wiki without positions | John Nduli |
| 2024-07-03 | feat: drop common words when building rdf caches | John Nduli |
| 2024-07-03 | feat: set 67 parallel processes to run in prod | John Nduli |
| 2024-07-03 | fix: remove namespaces since child processes copy the rdf caches | John Nduli |
| 2024-07-03 | fix: use correct prefix and index key; group wiki cache query | John Nduli |
| 2024-07-03 | feat: add wikidata prefix to search api | John Nduli |
| 2024-07-03 | feat: add wikidata indexing | John Nduli |
| 2024-07-03 | feat: add global wikicache | John Nduli |
| 2024-07-03 | feat: add sparql query to get wikidata | John Nduli |
| 2024-06-26 | Increase max number of results to 50000 for Xapian search•••This change needs to be accompanied by a change in GN2! If it's lower
than the GN2 MAX_SEARCH_RESULTS value, searches will throw an error.
| zsloan |
| 2024-06-24 | Use dataset Name instead of FullName for indexing•••The Name is generally used as the identifier, while the FullName can container spaces which can cause problems
| zsloan |
| 2024-06-18 | Revert "Set the file path for the logger."•••This reverts commit b21102bc4ad3678173e7c94d3e66333ec7c1d40a.
| Munyoki Kilyungi |
| 2024-06-18 | refactor: drop global variables | John Nduli |
| 2024-06-17 | Check table names in Xapian; if not, default to "-1".•••Without this check, there will always be an error when this script is
run with the "is-data-modified" flag should there be no database in
the XAPIAN_DIRECTORY.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-17 | Fetch distinct comments.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-14 | fix: typehints in index-genenetwork script | John Nduli |
| 2024-06-14 | fix: fix incorrect parameters in index_query function | John Nduli |
| 2024-06-12 | Move the generated xapian files to the correct directory.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-12 | Set the file path for the logger.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-12 | Change the date format for the logger.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-12 | Log how long it takes to run the indexing script.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-12 | Check for a running process by viewing the build dir's contents.•••In the CI build, the actual build is run in the
xapian_directory/build, which is seen as the xapian_directory in this
script. The CI handles clean up WRT removing files related to the
build process.
* scripts/index-genenetwork (create_xapian_index): Create the xapian
directory if it doesn't exist. If the xapian directory has files,
exit. Create the temporary directory inside the xapian_directory.
Remove "build_directory.rmdir()"
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-12 | Return 0 if data changes, else exit with 1.•••* scripts/index-genenetwork (is_data_modified): Replace click.echo
with the respective sys.exit call.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
| 2024-06-12 | Explicitly pass sparql_uri to script.•••Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
| Munyoki Kilyungi |
