aboutsummaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Expand)Author
2025-07-14Handle case where authorisation is already set up.Frederick Muriuki Muriithi
2025-07-14Restore original names and instead ignore specific linter errors.Frederick Muriuki Muriithi
2025-07-14Fix linting issues.Frederick Muriuki Muriithi
2025-07-14Provide timeouts for requests calls.Frederick Muriuki Muriithi
2025-07-14Remove unused script•••The bulk edit feature is no longer supported in the uploader. Frederick Muriuki Muriithi
2025-07-14Fix imports•••Remove unused imports, update old imports to newer forms and add missing imports. Frederick Muriuki Muriithi
2025-07-02Compute the means while doing the cross-refences.Frederick Muriuki Muriithi
2025-07-01Ignore any optional files that are not provided.Frederick Muriuki Muriithi
2025-06-26Add cross-reference IDs to job as metadata.Frederick Muriuki Muriithi
2025-06-26BugFix: Logical bug - files have already been "untransposed".•••The `read_csv_file_headers` function can read headings from non-transposed or transposed file, depending on the value of a flag. The bug was passing the value of the `*_transposed` flag from the control file, even after any transposed files have been "untransposed". This commit passes an explicit `False` value, and adds a comment to note the fix. Frederick Muriuki Muriithi
2025-06-26Convert to tuple to allow indexing.Frederick Muriuki Muriithi
2025-06-26Replace first field header with standard "id" header.Frederick Muriuki Muriithi
2025-06-26Pass in the field separator, and comment char while reading file•••Pass in the field separator character to split the fields correctly. Also pass in the comment character to safely ignore any comment lines. Frederick Muriuki Muriithi
2025-06-17Allow users to provide transposed files.Frederick Muriuki Muriithi
2025-06-10Pass on the publication to the next job.Frederick Muriuki Muriithi
2025-06-10Change message level.Frederick Muriuki Muriithi
2025-06-09Fix minor bug.Frederick Muriuki Muriithi
2025-06-09Run the authorisation update requests.Frederick Muriuki Muriithi
2025-06-09Compute the data_link_ids from the request results.Frederick Muriuki Muriithi
2025-06-09Fix bug in data for creating a new resource.Frederick Muriuki Muriithi
2025-06-09Compute request data for …/data/link/phenotypes endpoint.Frederick Muriuki Muriithi
2025-06-09Add debug logging to track process.Frederick Muriuki Muriithi
2025-06-09Fix return: Don't use `RETURNING *`.•••Fetching the results of the insert query with `RETURNING *` does not work as expected with MySQLdb. This commit, thus, returns the explicitly computed values instead. Frederick Muriuki Muriithi
2025-06-05Break up long lines.Frederick Muriuki Muriithi
2025-06-05`update_auth()` scaffolding•••Add untested scaffolding for how the `update_auth()` function will work. Include notes to help with figuring the feature out. Frederick Muriuki Muriithi
2025-06-05Only update authorisation after all data is entered.•••Pull the auth update function out of the database connection contextmanager to prevent any problems with an auth update from causing a rollback of all the data. We can always manually update the authorisation later, therefore, we do not want a failure in that causing the (potentially) time-consuming data entry process from being rolled back. Frederick Muriuki Muriithi
2025-06-03Add more logging to track progress.Frederick Muriuki Muriithi
2025-06-03Save data with `LOAD DATA INFILE …` query•••To help speed up the saving of the data (for really huge files) into the database, use the `LOAD DATA INFILE …` command if available, and if not fallback to the one using raw queries. Frederick Muriuki Muriithi
2025-06-03Extract building of data items into helper function.Frederick Muriuki Muriithi
2025-06-03Memory saving: Don't reload saved data•••We already have all the data in memory, so we do not need to reload it and keep yet another copy. Frederick Muriuki Muriithi
2025-06-02Compute data in files lazily.Frederick Muriuki Muriithi
2025-06-02Process NA-strings in the worker script.Frederick Muriuki Muriithi
2025-06-02Update placeholder function•••We'll probably provide a token to use to update the authorisation for the new data. The commit also explicitly raises a `NotImplement` exception for now. Frederick Muriuki Muriithi
2025-06-02Add missing import.Frederick Muriuki Muriithi
2025-06-02Fix bug: Compute data correctly. Replace N/A strings with NoneType.Frederick Muriuki Muriithi
2025-06-02Add some debugging statements.Frederick Muriuki Muriithi
2025-06-02Pass the phenotypes name-to-id map to functions.Frederick Muriuki Muriithi
2025-06-02Compute the map from a phenotype's name to its ID.Frederick Muriuki Muriithi
2025-05-30Refactor: Extract common pattern into separate function•••Extract the common pattern into a separate, more generalized function and pass the new function the data it requires to perform its tasks for the different file types. Frederick Muriuki Muriithi
2025-05-30Add typing to load_data function and invoke it.Frederick Muriuki Muriithi
2025-05-30Save standard errors and counts.Frederick Muriuki Muriithi
2025-05-30Get the species, population and dataset.Frederick Muriuki Muriithi
2025-05-30Add more tables to lock list.Frederick Muriuki Muriithi
2025-05-30Fix query: Add missing field and parameter.Frederick Muriuki Muriithi
2025-05-30Fix function names.Frederick Muriuki Muriithi
2025-05-30Fix processing of numerical data to save to the database.Frederick Muriuki Muriithi
2025-05-30Fix file paths: append the path to the extraction directory.Frederick Muriuki Muriithi
2025-05-30Refactor out common `__row_to_dataitems__` function.Frederick Muriuki Muriithi
2025-05-28Bug: Fix actual data value, not result set.Frederick Muriuki Muriithi
2025-05-28Bug: Add missing commas.Frederick Muriuki Muriithi