| Age | Commit message (Expand) | Author |
|---|
| 2024-12-03 | Display cell value...Display the cell value to enable the user compare the error message to
the actual value, in order to figure out how to fix the error(s).
| Frederick Muriuki Muriithi |
| 2024-12-03 | Improve UI: Move if check out of table...Create a new section for the errors display, and only display the
table if there is at least one error.
| Frederick Muriuki Muriithi |
| 2024-12-03 | Enable limiting of number of errors to fetch. | Frederick Muriuki Muriithi |
| 2024-12-03 | Save 'pheno' errors in redis as they are found. | Frederick Muriuki Muriithi |
| 2024-12-03 | Add some phenocovar metadata. | Frederick Muriuki Muriithi |
| 2024-12-03 | Fetch a jobs errors from its errors lists. | Frederick Muriuki Muriithi |
| 2024-12-03 | Save phenocovar errors in redis as they are found. | Frederick Muriuki Muriithi |
| 2024-12-03 | Pass the redis connection and fully qualified job id...Pass the redis connection on to the function used to build main since
it might need to use a connection to redis.
Also pass the computed fully qualified job id rather than recomputing
it every time.
Update dependent functions to take the new arguments.
| Frederick Muriuki Muriithi |
| 2024-12-03 | Cleanup linting and type (mypy) errors. | Frederick Muriuki Muriithi |
| 2024-12-03 | Wrap everything in try-catch to handle errors gracefully....Put everything in the build_main function within a try-catch block to
ensure we capture all exceptions that might occur in different scripts
and log them out.
This helps with debugging errors in the asynchronous scripts.
| Frederick Muriuki Muriithi |
| 2024-12-02 | Add emphasis to important point. | Frederick Muriuki Muriithi |
| 2024-12-02 | Fix typo. | Frederick Muriuki Muriithi |
| 2024-12-02 | Send logs to STDOUT too. | Frederick Muriuki Muriithi |
| 2024-12-02 | Launch job and redirect to job status page. | Frederick Muriuki Muriithi |
| 2024-12-02 | Add UI to view status of the job as it runs. | Frederick Muriuki Muriithi |
| 2024-12-02 | Return 0 on completion of QC processing. | Frederick Muriuki Muriithi |
| 2024-12-02 | Change from 'pending' to 'started' on entry to script....Update the status of the job from 'pending' to 'started' at the
entry-point of the script.
| Frederick Muriuki Muriithi |
| 2024-11-22 | Remove local code that does db connection....Reuse the code in gn-libs for connecting to the database, rather than
using the replicated local code.
| Frederick Muriuki Muriithi |
| 2024-11-22 | Use gn-libs code for db connection....Use the code in gn-libs to connect to the database, rather than a
local module.
| Frederick Muriuki Muriithi |
| 2024-11-22 | Add gn-libs as a dependency. | Frederick Muriuki Muriithi |
| 2024-10-24 | Reduce number of local variables created in function....Use the values in the argparse.Namespace object directly to avoid
creating many local variables.
| Frederick Muriuki Muriithi |
| 2024-10-24 | Add `jobid` to extraction directory...Add the job id to help "group" files that belong to the same job.
| Frederick Muriuki Muriithi |
| 2024-10-24 | Check pheno, phenose, and phenonum files with logging...Add logging to the `qc_pheno_file` function so that we get the
messages pushed to redis for every file that is being checked for
errors.
| Frederick Muriuki Muriithi |
| 2024-10-24 | Check phenocovar files with logging...Build a function-scope logger using the new
`scripts.redis_logger.RedisMessageListHandler` log handler to log-out
the messages for each phenocovar file being processed in a more
thread-safe way.
| Frederick Muriuki Muriithi |
| 2024-10-24 | Fix linting and typing errors. | Frederick Muriuki Muriithi |
| 2024-10-24 | Add a `--loglevel` argument to select loglevel in scripts...To enable selection of log levels within scripts, this commit adds the
`--loglevel` argument to the list of arguments that can be passed to
scripts.
| Frederick Muriuki Muriithi |
| 2024-10-24 | Parse entire namespace object rather than individual values. | Frederick Muriuki Muriithi |
| 2024-10-24 | Move logger creation to `build_main` function...Since the module-level loggers are built mostly the same, move the
creation of the logger to the more general function to reduce
repetition.
| Frederick Muriuki Muriithi |
| 2024-10-24 | Add handler to send log messages to specific Redis list...Class `RedisMessageListHandler` builds a log handler that pushes any
messages logged out to the redis list specified by its
`fullyqualifiedkey` argument.
| Frederick Muriuki Muriithi |
| 2024-10-24 | Update Redis prefix settings. | Frederick Muriuki Muriithi |
| 2024-10-24 | Don't compute settings in configuration file. | Frederick Muriuki Muriithi |
| 2024-10-23 | Add documentation on background jobs. | Frederick Muriuki Muriithi |
| 2024-10-22 | Document emergent general QC structure for CSV files. | Frederick Muriuki Muriithi |
| 2024-10-22 | Refactor `qc_pheno_file` and reuse it for different file types....The QC/QA steps taken by the `qc_pheno_file` function are very
similar across the "pheno", "phenose" and "phenonum" files. This
commit makes the `qc_pheno_file` function a higher-order function and
we pass the file-type specific check(s) as a callable (function) to be
used for the QC/QA process.
| Frederick Muriuki Muriithi |
| 2024-10-22 | Check for errors in `pheno` files. | Frederick Muriuki Muriithi |
| 2024-10-21 | Check `phenocovar` files for errors. | Frederick Muriuki Muriithi |
| 2024-10-21 | Add utilities for reading text files and csv files...The function `read_text_file(…)` will read the raw text file and
return the content line by line.
The function `read_csv_file(…)` builds on `read_text_file(…)` to read
a file as a character-separated-values file. The field separator, and
comment_char characters can be provided to customise how the CSV file
is read.
| Frederick Muriuki Muriithi |
| 2024-10-17 | UI: Create background job when user uploads file. | Frederick Muriuki Muriithi |
| 2024-10-17 | Cleanup: Delete all extracted files after processing. | Frederick Muriuki Muriithi |
| 2024-10-17 | Leave TODO notes for what needs to be done. | Frederick Muriuki Muriithi |
| 2024-10-17 | Fetch samples from database...Fetch the samples from the database. These will be used to verify
that the samples in the phenotype files already exist in the database
and are valid.
| Frederick Muriuki Muriithi |
| 2024-10-17 | Undo transpose for any transposed files...To reduce the complexity involved in the processing of the files, we
undo any transposition of the CSV files for those files that are
marked as transposed.
| Frederick Muriuki Muriithi |
| 2024-10-17 | Pass new arguments to QC function. | Frederick Muriuki Muriithi |
| 2024-10-17 | Add `speciesid` and `populationid` arguments to the script. | Frederick Muriuki Muriithi |
| 2024-10-17 | Add the working directory argument to the script....Add a `--working-dir` argument to allow passing a directory where the
script process the files within. If not provided, it defaults to a
directory within the systems temporary directory.
| Frederick Muriuki Muriithi |
| 2024-10-17 | Extract the R/qtl2 bundle for processing....To enable processing of the files individually, this commit will
enable the extraction of the files into a known working directory in
which all further processing will take place.
| Frederick Muriuki Muriithi |
| 2024-10-17 | Extract common functions. | Frederick Muriuki Muriithi |
| 2024-10-17 | Save errors for each file in lists. Parallelise error checking....* Save the errors for each file in a redis list for that file.
* Make error checking parallel, i.e. ensure every file of a particular
type is checked completely independent of other files of the same
type.
| Frederick Muriuki Muriithi |
| 2024-10-17 | Rewrite the QC code for R/qtl2 | Frederick Muriuki Muriithi |
| 2024-10-17 | Ignore comment lines. | Frederick Muriuki Muriithi |
