| Age | Commit message (Expand) | Author |
|---|
| 2021-09-15 | Reorganise modules...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* The heatmap generation does not fall cleanly within the computations or db
modules. This commit moves it to the higher level gn3 module.
| Frederick Muriuki Muriithi |
| 2021-09-08 | Ease search for traits and chromosomes...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Return a dict of values rather than list for the traits and chromosomes to
ease searching through the data.
| Muriithi Frederick Muriuki |
| 2021-09-08 | Parse Chr value as int where possible...* To ease sorting of data by numerical order down the line, sort the "Chr"
values by numerical order.
| Muriithi Frederick Muriuki |
| 2021-09-08 | Remove extraneous text to ease sorting...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Change the id from 'T<n>' to simply '<n>' to ease sorting of the trait
results by numerical order rather than string order.
| Muriithi Frederick Muriuki |
| 2021-09-08 | Fix the traits order computations for clustering...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Fix ordering function
* tests/unit/computations/test_heatmap.py: update test
The order of the traits is important for the clustering algorithm, since the
clustering seems to use the distance of one trait from another to determine
how to order them.
This commit also gets rid of the xoffset argument that is not important to
the ordering, and was used in the older GN1 to determine how to draw the
clustering lines.
| Muriithi Frederick Muriuki |
| 2021-09-06 | Provide function to organise parsed QTLReaper results...* gn3/computations/qtlreaper.py: Provide a function to organise the results by
trait for easier use down the line.
* tests/unit/computations/test_qtlreaper.py: provide a test to ensure that the
organising function works as expected.
| Muriithi Frederick Muriuki |
| 2021-09-06 | Leave "Chr" value as string when parsing...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* The "Chr" value seems to be mostly a name of some sort, despite it being,
seemingly an number. This commit parses the "Chr" value as a string.
It also updates the tests to expec a string, rather than a number for "Chr"
values.
| Muriithi Frederick Muriuki |
| 2021-09-06 | Handle type-coercion exceptions...* gn3/computations/qtlreaper.py: handle exceptions
Sometimes, the values being parsed are plain strings and cannot be cast to
the float types. This commit handles that by casting only those values that
can be cast to float, and returning the others as strings.
| Muriithi Frederick Muriuki |
| 2021-09-06 | Find nearest marker...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Migrate the `web.webqtl.heatmap.Heatmap.getNearestMarker` function in GN1 to
GN3.
| Muriithi Frederick Muriuki |
| 2021-08-31 | Fix linting errors, minor bugs and reorganise code...* Fix some linting errors and some minor bugs caught by the linter.
Move the `random_string` function to separate module for use in multiple
places in the code.
| Muriithi Frederick Muriuki |
| 2021-08-31 | Update `heatmap_data` function: remove extraneous data...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: update function
* gn3/db/traits.py: new function
Remove extraneous data and arguments from the function.
- Load the genotype file
- Generate traits file
- Provide both raw traits data, and exported traits data in return
| Muriithi Frederick Muriuki |
| 2021-08-31 | Parse QTLReaper outputs...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/qtlreaper.py: pass output files
* tests/unit/computations/data/qtlreaper/main_output_sample.txt: sample test
data
* tests/unit/computations/data/qtlreaper/permu_output_sample.txt: sample test
data
* tests/unit/computations/test_qtlreaper.py: add tests
Add code to parse the QTLReaper output data files.
| Muriithi Frederick Muriuki |
| 2021-08-31 | Fix bugs with `run_reaper` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/qtlreaper.py: Fix some bugs
* qtlfilesexport.py: Test out running rust-qtlreaper
Test out the qtlreaper interface code and fix some bugs caught in the
process.
| Muriithi Frederick Muriuki |
| 2021-08-30 | Fix issues with traits file format...* README.md: update header: Traits ==> Trait
* gn3/computations/qtlreaper.py: update header: Traits ==> Trait
* qtlfilesexport.py: Choose only BXD strains
Rename the first column header from "Traits" to "Trait" to correspond with
what `rust-qtlreaper` expects.
Choose only the BXD strains for the proof-of-concept example - this helped
bring out the fact that the traits file SHOULD NOT contain a strain column
for a strain that does not exist in the genotype file in consideration.
If the traits file has a strain column which does not exist in the genotype
file, then `rust-qtlreaper` fails with a panic, since, from what I can tell,
it tries to get a value from the genotype file for the non-existent strain,
which results to a `None` type. Subsequent attempts at running an operation
on the `None` type lead to the panic.
| Muriithi Frederick Muriuki |
| 2021-08-30 | Remove empty line...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Remove empty line at the end of the traits file
| Muriithi Frederick Muriuki |
| 2021-08-30 | Fix some linting errors and minor bugs. | Muriithi Frederick Muriuki |
| 2021-08-30 | Implement module for interfacing with rust-qtlreaper...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: move `generate_traits_file` function to new
module
* gn3/computations/qtlreaper.py: new module to interface with the
`rust-qtlreaper` utility.
* gn3/settings.py: Provide setting for the path to the `rust-qtlreaper`
utility
* qtlfilesexport.py: Move `random_string` function to new module. Update to
use functions in new module.
Provide a module with functions to be used to interface with
`rust-qtlreaper`. This module essentially contains all the functions that
are needed to build the files needed for, and to run the qtlreaper utility.
| Muriithi Frederick Muriuki |
| 2021-08-27 | Provide intermediate data in final results...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Seeing as not every requirement/feature has been migrated over at this time,
this commit just provides all the intermediate data representations in the
final return of the function for later use down the line.
| Muriithi Frederick Muriuki |
| 2021-08-27 | Export trait data to file...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Provide a function to export the given strains and traits data into a traits
file for use with `rust-qtlreaper`.
| Muriithi Frederick Muriuki |
| 2021-08-27 | Rework strains and trait values retrieval...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Rework the strains and values retrieval function to more closely correspond
to the working of the original code in GN1
| Muriithi Frederick Muriuki |
| 2021-08-20 | Merge branch 'main' of github.com:genenetwork/genenetwork3 into heatmap_gener... | Muriithi Frederick Muriuki |
| 2021-08-20 | Fix typing issue(s) caught by mypy...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Use `Sequence` type not `Iterator` type
| Muriithi Frederick Muriuki |
| 2021-08-20 | Retrieve the strains with valid values...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: add function to get strains with values
* tests/unit/computations/test_heatmap.py: new tests
Add function to get the strains whose values are not `None` from the
`trait_data` object passed in.
This migrates
https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/Heatmap.py#L215-221
into a separate function that can handle that and be tested independently of
any other code.
| Muriithi Frederick Muriuki |
| 2021-08-20 | Add tests for ordering and implement function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: implement new ordering function
* tests/unit/computations/test_heatmap.py: add new tests
Implement the ordering function to migrate the setup of the `neworder`
variable from GN1 to GN3.
This migration is incomplete, since there is dependence on the return from
the `web.webqtl.heatmap.Heatmap.draw` function in form of the `d_1` variable
in some of the paths.
The thing is, this `d_1` variable, and the `xoffset` variable seem to be
used for laying out things on the drawn heatmap, and might actually end up
not being needed for the new system using plotly, which has other ways of
laying out things on the drawing.
For now though, this commit "shims" the presence of these values until when
the use of these variables is confirmed as present or absent in the new GN3
system.
| Muriithi Frederick Muriuki |
| 2021-08-20 | Minor correlation fixes (#36)...* fix key error for (*tissue_cor) tissue correlation
* update tests for tissue correlation
* rename speed_compute to fast_compute
* pep8 formatting | Alexander Kabui |
| 2021-08-18 | Test the clustering...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Fix clustering bugs
* tests/unit/computations/test_heatmap.py: Add new tests. Fix linting issues.
Test and fix the clustering function.
| Muriithi Frederick Muriuki |
| 2021-08-18 | Fix obvious linting errors...* Fix linting errors that do not change the function of the code.
| Muriithi Frederick Muriuki |
| 2021-08-18 | Make child sequence a list...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Since the `slink` function assigns values to the `listcopy` variable and its
children, this commit ensures that the sequence is a list to allow for the
assignment.
If the child-sequence is a tuple, that would lead to an exception.
| Muriithi Frederick Muriuki |
| 2021-08-18 | Add tests and fix errors caught with tests...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: fix errors
* tests/unit/computations/test_heatmap.py: new tests
Add new tests with the expected source data format, and expected results.
Fix all errors that were caught by running the tests
| Muriithi Frederick Muriuki |
| 2021-08-17 | Test the clustering...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Fix clustering bugs
* tests/unit/computations/test_heatmap.py: Add new tests. Fix linting issues.
Test and fix the clustering function.
| Muriithi Frederick Muriuki |
| 2021-08-17 | Fix obvious linting errors...* Fix linting errors that do not change the function of the code.
| Muriithi Frederick Muriuki |
| 2021-08-17 | Make child sequence a list...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Since the `slink` function assigns values to the `listcopy` variable and its
children, this commit ensures that the sequence is a list to allow for the
assignment.
If the child-sequence is a tuple, that would lead to an exception.
| Muriithi Frederick Muriuki |
| 2021-08-17 | Add tests and fix errors caught with tests...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: fix errors
* tests/unit/computations/test_heatmap.py: new tests
Add new tests with the expected source data format, and expected results.
Fix all errors that were caught by running the tests
| Muriithi Frederick Muriuki |
| 2021-08-16 | Merge pull request #32 from genenetwork/heatmap_decompose_db_retrieval...Heatmap decompose db retrieval | BonfaceKilz |
| 2021-08-12 | Fix linting errors...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Fix some errors caught by the linter.
| Muriithi Frederick Muriuki |
| 2021-08-12 | Import missing definitions...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Import some missing definitions.
| Muriithi Frederick Muriuki |
| 2021-08-11 | use normal function for correlation (#34)...* use normal function for correlation + rename functions
* update test for sample correlation
* use normal function for tissue correlation + rename functions | Alexander Kabui |
| 2021-08-09 | Build up the heatmap data...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Add code to compute and organise the data that will be used to draw the
final heatmap.
This varies significantly in how it works from the original, but it still
tries to retain the general flow of data.
| Muriithi Frederick Muriuki |
| 2021-08-04 | Fix issues caught by pylint...* gn3/computations/slink.py: remove unused imports
* gn3/db/traits.py: remove unnecessary `else` clauses
* tests/unit/db/test_traits.py: add docstrings for functions
| Muriithi Frederick Muriuki |
| 2021-07-29 | Add partial type annotations for slink module...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Add some type annotations for the `nearest` function.
* Leave some comments regarding the issues experienced when trying to add some
typing annotations to the function to help with future endeavours of the
same.
| Muriithi Frederick Muriuki |
| 2021-07-29 | Add partial type annotations for slink module...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Add some type annotations for the `nearest` function.
* Leave some comments regarding the issues experienced when trying to add some
typing annotations to the function to help with future endeavours of the
same.
| Muriithi Frederick Muriuki |
| 2021-07-26 | Merge branch 'main' of github.com:genenetwork/genenetwork3 | Muriithi Frederick Muriuki |
| 2021-07-26 | Fix issues caught by pylint...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Fix a myriad of issues caught by pylint to ensure the code passes all tests.
| Muriithi Frederick Muriuki |
| 2021-07-26 | Check if corr_coefficient is NaN, since apparently it's stored as NaN instead... | zsloan |
| 2021-07-23 | Add data examples for `slink`. Implement function....Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Copy the function, mostly verbatim from
genenetwork1. See:
https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/slink.py#L107-L138
* tests/unit/computations/test_slink.py: Add a test with some example data to
test that the implementation gives the same results as that in genenetwork1
| Muriithi Frederick Muriuki |
| 2021-07-23 | Iterate through all valid pairs...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Fix the iteration construct.
Given two lists of member coordinates, such as [0, 1] and [3, 5], the
initial code would iterate over the pairs [0, 3] and [1, 5].
This commit fixes the iteration constructs such that the new code iterates
over the pairs [0, 3], [0, 5], [1, 3] and [1, 5].
| Muriithi Frederick Muriuki |
| 2021-07-23 | Extract function to flatten list of lists...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Extract the `__flatten_list_of_lists` function
since it is used in more than one place.
| Muriithi Frederick Muriuki |
| 2021-07-23 | Fix issue caught in `nearest` while testing `slink`...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* While running tests for slink, to try and understand what it is doing in
order to write the appropriate tests for it, an issue arose that pointed a
blindspot in the former understanding of now `nearest` should work.
This commit fixes the issue found in both the expected data, and the code.
| Muriithi Frederick Muriuki |
| 2021-07-23 | New function (`slink`): return [] on exception...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Add minimum code to pass new test
* tests/unit/computations/test_slink.py: new test
Add test to ensure that the new `slink` function return an empty list in
case and exception is raised.
Add the new `slink` function with minimum amount of code needed to pass the
test.
| Muriithi Frederick Muriuki |
| 2021-07-23 | Add dummy module documentation...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Add initial dummy module documentation in form of
a python docstring
Update the names of what would be private methods/function to start with a
double-underscore (__) so that they do not show up in the default python
documentation.
| Muriithi Frederick Muriuki |
