aboutsummaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)Author
2024-07-03feat: add sparql query to get wikidataJohn Nduli
2024-06-24Use dataset Name instead of FullName for indexingzsloan
The Name is generally used as the identifier, while the FullName can container spaces which can cause problems
2024-06-18Revert "Set the file path for the logger."Munyoki Kilyungi
This reverts commit b21102bc4ad3678173e7c94d3e66333ec7c1d40a.
2024-06-18refactor: drop global variablesJohn Nduli
2024-06-17Check table names in Xapian; if not, default to "-1".Munyoki Kilyungi
Without this check, there will always be an error when this script is run with the "is-data-modified" flag should there be no database in the XAPIAN_DIRECTORY. Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-17Fetch distinct comments.Munyoki Kilyungi
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-14fix: typehints in index-genenetwork scriptJohn Nduli
2024-06-14fix: fix incorrect parameters in index_query functionJohn Nduli
2024-06-12Move the generated xapian files to the correct directory.Munyoki Kilyungi
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Set the file path for the logger.Munyoki Kilyungi
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Change the date format for the logger.Munyoki Kilyungi
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Log how long it takes to run the indexing script.Munyoki Kilyungi
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Check for a running process by viewing the build dir's contents.Munyoki Kilyungi
In the CI build, the actual build is run in the xapian_directory/build, which is seen as the xapian_directory in this script. The CI handles clean up WRT removing files related to the build process. * scripts/index-genenetwork (create_xapian_index): Create the xapian directory if it doesn't exist. If the xapian directory has files, exit. Create the temporary directory inside the xapian_directory. Remove "build_directory.rmdir()" Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Return 0 if data changes, else exit with 1.Munyoki Kilyungi
* scripts/index-genenetwork (is_data_modified): Replace click.echo with the respective sys.exit call. Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Explicitly pass sparql_uri to script.Munyoki Kilyungi
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Rework how the indexes are built.Munyoki Kilyungi
Right now, the checks are done in Guix's build expression. This moves that work to the index-genenetwork script.
2024-06-12Add method to check the validity of the tables+RDF checksums.Munyoki Kilyungi
* scripts/index-genenetwork (verify_checksums): New function. Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-12Generate a SHA256 checksum for the generif graph.Munyoki Kilyungi
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-01Use global cache to store generif metadata.Munyoki Kilyungi
This global caches has 3,528 entries and there's no expectation for it to grow significantly. Since child processes inherit the parent’s memory, we can pass the global cache to them, reducing fetch times from 0.001s to 0.00001s, significantly boosting performance when indexing the entire database and enriching results with RDF metadata. Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-06-01Add geneRIF to gene index.Munyoki Kilyungi
* scripts/index-genenetwork: Import Template, lru_cache, SPARQLWrapper, JSON (get_rif_metadata): New function. (index_rif_comments): New function. (index_genes): Add rif comments to probeset index. Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2024-03-18pep8 formattingAlexander_Kabui
2024-03-18pep8 formattingAlexander_Kabui
2024-03-15add pubmed metadataAlexander_Kabui
2024-03-13Add documentation on what the script is for and does.Frederick Muriuki Muriithi
2024-03-13Remove debug prints in the scriptFrederick Muriuki Muriithi
2024-03-12Commit migration script for CaseAttribute* tablesFrederick Muriuki Muriithi
2023-12-05Move script to gn-authFrederick Muriuki Muriithi
The script is not used in GN3.
2023-11-02Make scripts directory a package to eliminate path issues.Frederick Muriuki Muriithi
2023-10-27Make scripts/ a module since it referenced in argparse_actions.py.Munyoki Kilyungi
* scripts/__init__.py: New file. Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
2023-10-10Remove authentication from GN3Frederick Muriuki Muriithi
Authentication should be handled by the auth server (gn-auth) and thus, this commit removes code handling user authentication from the GN3 system.
2023-08-02Use correct variableFrederick Muriuki Muriithi
After copying the calls to assign appropriate roles to the admin in the script, I forgot to update the variable name. This commit fixes that.
2023-08-02Fix module name.Frederick Muriuki Muriithi
2023-08-02Assign `group-leader` role to the admin for automatic group.Frederick Muriuki Muriithi
2023-06-02auth: Increase limit to 100K recordsFrederick Muriuki Muriithi
2023-06-02auth: Add delay between DB insertsFrederick Muriuki Muriithi
Add delays to avoid overwhelming the DB server.
2023-06-02Auth: Bug: Display all admins before taking choiceFrederick Muriuki Muriithi
2023-05-31scripts: Write table checksums into index.Arun Isaac
* scripts/index-genenetwork (main): Write table checksums into index.
2023-05-31scripts: Introduce SQLTableClause.Arun Isaac
* scripts/index-genenetwork (SQLTableClause): New variable. (genes_query, phenotypes_query): Express tables using SQLTableClause. (serialize_sql): Serialize SQLTableClause.
2023-05-31scripts: Fold long lines.Arun Isaac
* scripts/index-genenetwork (write_document, index_query): Fold long lines.
2023-05-31scripts: Ensure only one indexing job may run at a time.Arun Isaac
* scripts/index-genenetwork (main): Ensure no other indexing job is running.
2023-05-30Fix linting errors and failing testsFrederick Muriuki Muriithi
* scripts/register_sys_admin.py: fix linting error * tests/unit/auth/fixtures/oauth2_client_fixtures.py: hash client secret in database. The code expects it hashed in the database.
2023-05-30Enable registering sys-admin on CLIFrederick Muriuki Muriithi
To ease registration of system-admin user, provide a CLI script to register the user and mark them as sys admin in one go.
2023-05-26Document CLI Utility CommandsFrederick Muriuki Muriithi
Document some CLI utility commands useful for development and for supporting the operation of the GN3 app.
2023-05-25Add some error checks.Frederick Muriuki Muriithi
2023-05-25Script to assign existing data to publicly-visible resourcesFrederick Muriuki Muriithi
A script to assign existing data not assigned to any group to publicly-visible resources.
2023-05-22Make directory at "path" and all intermediate ones.Frederick Muriuki Muriithi
Make the directory at the given path, and any intermediate ones to avoid errors in the indexing code when the directory, or its parent(s) do not exist.
2023-04-19oauth2: Link the phenotype traits to user groups.Frederick Muriuki Muriithi
2023-04-19Fix minor linting and typing issuesFrederick Muriuki Muriithi
2023-04-19auth: Setup selected traits correctlyFrederick Muriuki Muriithi
Fix bugs with setting up of the selected traits for use while filtering the search results.
2023-04-18auth: Consistently JSON encode values.Frederick Muriuki Muriithi
Consistently encode all values for the top-level keys stored in redis to avoid issues with json encode/decode
generated by cgit v1.2.3 (git 2.44.0) at 2024-11-29 09:01:53 +0000