| Age | Commit message (Expand) | Author |
|---|
| 2021-11-04 | Create blackbox tests for some functions migrated from R...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: new stub
functions (partial_correlation_matrix, partial_correlation_recursive)
*
tests/unit/computations/partial_correlations_test_data/pcor_mat_blackbox_test.csv:
blackbox sample data and results for variance-covariance matrix method
*
tests/unit/computations/partial_correlations_test_data/pcor_rec_blackbox_test.csv:
blackbox sample data and results for recursive method
* tests/unit/computations/test_partial_correlations.py: Tests for new function
Provide some blackbox testing sample data for checking the operation of the
functions migrated from R.
| Frederick Muriuki Muriithi |
| 2021-11-01 | Merge branch 'main' of github.com:genenetwork/genenetwork3 into partial-corre... | Frederick Muriuki Muriithi |
| 2021-11-01 | Retrieve indices of the selected samples...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: New
function (good_dataset_samples_indexes).
* tests/unit/computations/test_partial_correlations.py: Tests for new
function (good_dataset_samples_indexes)
Get the indices of the selected samples. This is a partial migration of the
`web.webqtl.correlation.PartialCorrDBPage.getPartialCorrelationsFast`
function in GN1.
| Frederick Muriuki Muriithi |
| 2021-10-29 | Feature/biweight reimplementation (#47)...* add biweight reimplementation with pingouin
* delete biweight scripts and tests
* add python-pingouin to guix file
* delete biweight paths
* mypy fix:pingouin mising imports
* pep8 formatting && pylint fixes | Alexander Kabui |
| 2021-10-29 | Specify ten (10) decimal places...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: specify 10 decimal places
* tests/unit/computations/test_partial_correlations.py: update examples
Slight differences in python implementations, possibly hardware and
operating systems could cause the value of float (double) values to be
different in the less significant parts of the decimal places.
This commit limits the usable part of the decimals to the first 10 decimal
places for now.
| Frederick Muriuki Muriithi |
| 2021-10-29 | Fix some linting errors...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
| Frederick Muriuki Muriithi |
| 2021-10-29 | Implement `tissue_correlation` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: New function (tissue_correlation)
* tests/unit/test_partial_correlations.py ->
tests/unit/computations/test_partial_correlations.py: Move module. Implement
tests for new function
Migrate the `cal_tissue_corr` function embedded in the
`web.webqtl.correlation.correlationFunction.batchCalTissueCorr` function in
GN1 and implement tests to ensure it works correctly.
| Frederick Muriuki Muriithi |
| 2021-10-25 | fix unittests | Alexander Kabui |
| 2021-09-27 | modify unittests | Alexander Kabui |
| 2021-09-27 | add tests for calling wgcna_script | Alexander Kabui |
| 2021-09-27 | fix merge conflicts | Alexander Kabui |
| 2021-09-27 | Update terminology: `strain` to `sample`...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Update the terminology used: use `sample` in place of `strain` according to
Zachary's direction at
https://github.com/genenetwork/genenetwork3/pull/37#issuecomment-926043306
| Frederick Muriuki Muriithi |
| 2021-09-25 | minor fixes for unittests | Alexander Kabui |
| 2021-09-23 | Refactor: Move common sample data to separate file...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Move common sample test data into a separate file where it can be imported
from, to prevent pylint error R0801 which proved tricky to silence in any
other way.
| Frederick Muriuki Muriithi |
| 2021-09-22 | add init tests for call to script | Alexander Kabui |
| 2021-09-17 | Fix a number of linting issues...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
| Frederick Muriuki Muriithi |
| 2021-09-16 | pylint fixes | Alexander Kabui |
| 2021-09-16 | init tests for wgcna | Alexander Kabui |
| 2021-09-15 | initial test file for wgcna | Alexander Kabui |
| 2021-09-15 | Reorganise modules...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* The heatmap generation does not fall cleanly within the computations or db
modules. This commit moves it to the higher level gn3 module.
| Frederick Muriuki Muriithi |
| 2021-09-15 | Fix format of arguments and expected values...* tests/unit/computations/test_heatmap.py: ordering is not longer provided as
a list of tuples; the ordering values are just a list of numbers now. This
commit updates the test to take this into consideration.
* tests/unit/computations/test_qtlreaper.py: the 'Chr' value if numeric, is
represented by an actual number, not a string. This commit updates the code
to take this into consideration.
| Frederick Muriuki Muriithi |
| 2021-09-08 | Ease search for traits and chromosomes...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Return a dict of values rather than list for the traits and chromosomes to
ease searching through the data.
| Muriithi Frederick Muriuki |
| 2021-09-08 | Fix the traits order computations for clustering...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Fix ordering function
* tests/unit/computations/test_heatmap.py: update test
The order of the traits is important for the clustering algorithm, since the
clustering seems to use the distance of one trait from another to determine
how to order them.
This commit also gets rid of the xoffset argument that is not important to
the ordering, and was used in the older GN1 to determine how to draw the
clustering lines.
| Muriithi Frederick Muriuki |
| 2021-09-06 | Provide function to organise parsed QTLReaper results...* gn3/computations/qtlreaper.py: Provide a function to organise the results by
trait for easier use down the line.
* tests/unit/computations/test_qtlreaper.py: provide a test to ensure that the
organising function works as expected.
| Muriithi Frederick Muriuki |
| 2021-09-06 | Leave "Chr" value as string when parsing...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* The "Chr" value seems to be mostly a name of some sort, despite it being,
seemingly an number. This commit parses the "Chr" value as a string.
It also updates the tests to expec a string, rather than a number for "Chr"
values.
| Muriithi Frederick Muriuki |
| 2021-08-31 | Fix linting errors, minor bugs and reorganise code...* Fix some linting errors and some minor bugs caught by the linter.
Move the `random_string` function to separate module for use in multiple
places in the code.
| Muriithi Frederick Muriuki |
| 2021-08-31 | Fix test...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* The number of the arguments to the function changed, and so the tests for
the function needed to be updated.
| Muriithi Frederick Muriuki |
| 2021-08-31 | Parse QTLReaper outputs...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/qtlreaper.py: pass output files
* tests/unit/computations/data/qtlreaper/main_output_sample.txt: sample test
data
* tests/unit/computations/data/qtlreaper/permu_output_sample.txt: sample test
data
* tests/unit/computations/test_qtlreaper.py: add tests
Add code to parse the QTLReaper output data files.
| Muriithi Frederick Muriuki |
| 2021-08-20 | Merge branch 'main' of github.com:genenetwork/genenetwork3 into heatmap_gener... | Muriithi Frederick Muriuki |
| 2021-08-20 | Retrieve the strains with valid values...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: add function to get strains with values
* tests/unit/computations/test_heatmap.py: new tests
Add function to get the strains whose values are not `None` from the
`trait_data` object passed in.
This migrates
https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/Heatmap.py#L215-221
into a separate function that can handle that and be tested independently of
any other code.
| Muriithi Frederick Muriuki |
| 2021-08-20 | Add tests for ordering and implement function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: implement new ordering function
* tests/unit/computations/test_heatmap.py: add new tests
Implement the ordering function to migrate the setup of the `neworder`
variable from GN1 to GN3.
This migration is incomplete, since there is dependence on the return from
the `web.webqtl.heatmap.Heatmap.draw` function in form of the `d_1` variable
in some of the paths.
The thing is, this `d_1` variable, and the `xoffset` variable seem to be
used for laying out things on the drawn heatmap, and might actually end up
not being needed for the new system using plotly, which has other ways of
laying out things on the drawing.
For now though, this commit "shims" the presence of these values until when
the use of these variables is confirmed as present or absent in the new GN3
system.
| Muriithi Frederick Muriuki |
| 2021-08-20 | Minor correlation fixes (#36)...* fix key error for (*tissue_cor) tissue correlation
* update tests for tissue correlation
* rename speed_compute to fast_compute
* pep8 formatting | Alexander Kabui |
| 2021-08-18 | Test the clustering...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Fix clustering bugs
* tests/unit/computations/test_heatmap.py: Add new tests. Fix linting issues.
Test and fix the clustering function.
| Muriithi Frederick Muriuki |
| 2021-08-18 | Add tests and fix errors caught with tests...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: fix errors
* tests/unit/computations/test_heatmap.py: new tests
Add new tests with the expected source data format, and expected results.
Fix all errors that were caught by running the tests
| Muriithi Frederick Muriuki |
| 2021-08-17 | Test the clustering...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: Fix clustering bugs
* tests/unit/computations/test_heatmap.py: Add new tests. Fix linting issues.
Test and fix the clustering function.
| Muriithi Frederick Muriuki |
| 2021-08-17 | Add tests and fix errors caught with tests...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/heatmap.py: fix errors
* tests/unit/computations/test_heatmap.py: new tests
Add new tests with the expected source data format, and expected results.
Fix all errors that were caught by running the tests
| Muriithi Frederick Muriuki |
| 2021-08-11 | use normal function for correlation (#34)...* use normal function for correlation + rename functions
* update test for sample correlation
* use normal function for tissue correlation + rename functions | Alexander Kabui |
| 2021-07-26 | Fix issues caught by pylint...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* Fix a myriad of issues caught by pylint to ensure the code passes all tests.
| Muriithi Frederick Muriuki |
| 2021-07-23 | Add data examples for `slink`. Implement function....Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Copy the function, mostly verbatim from
genenetwork1. See:
https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/slink.py#L107-L138
* tests/unit/computations/test_slink.py: Add a test with some example data to
test that the implementation gives the same results as that in genenetwork1
| Muriithi Frederick Muriuki |
| 2021-07-23 | Add more test data...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
| Muriithi Frederick Muriuki |
| 2021-07-23 | Fix issue caught in `nearest` while testing `slink`...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* While running tests for slink, to try and understand what it is doing in
order to write the appropriate tests for it, an issue arose that pointed a
blindspot in the former understanding of now `nearest` should work.
This commit fixes the issue found in both the expected data, and the code.
| Muriithi Frederick Muriuki |
| 2021-07-23 | New function (`slink`): return [] on exception...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Add minimum code to pass new test
* tests/unit/computations/test_slink.py: new test
Add test to ensure that the new `slink` function return an empty list in
case and exception is raised.
Add the new `slink` function with minimum amount of code needed to pass the
test.
| Muriithi Frederick Muriuki |
| 2021-07-22 | Get shortest distance from two lists/tuples of member coordinates...* gn3/computations/slink.py: add code to ensure new test passes
* tests/unit/computations/test_slink.py: new test
This one is a little weird: from
https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/slink.py#L57-L63
It gets rid of the last coordinates in both the lists of the member
coordinates, and uses the remaining coordinates to find the shortest
members.
For example, given the following member coordinates:
- i=[0,1,2] and j=[5,7,9], it uses [0,1] and [5,7]
- i=[3,6,1] and j=[7,13], it uses [3,6] and [7]
to find the shortest distances.
I (fredmanglis) am not sure why it does it this way, since I'd have expected
it to use all the coordinates, however, since at this time we need to retain
bug-compatibility with the older code, I have done it as it is done in the
old code.
I also add a statement to raise an exception in the case where i and j are
not lists of integers, or integers
| Muriithi Frederick Muriuki |
| 2021-07-22 | Test for shortest distance between members in a list and coordinate...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: add code to pass new test
* tests/unit/computations/test_slink.py: new test
Given a list of members in a group, and a coordinate for a member in the
same group, find the distance of the closest member from the given
coordinate in the group.
| Muriithi Frederick Muriuki |
| 2021-07-22 | Check that given list and both coordinates, we get shortest distance...* gn3/computations/slink.py: Add code to compute the distance given the
coordinate of both members on the parent list/tuple
* tests/unit/computations/test_slink.py:
* Change the name of the tests to more closely correspond to the business
requirement the test is checking for
* Update the comments to indicate some more things that might need to be
done in the future
| Muriithi Frederick Muriuki |
| 2021-07-22 | Check that all distances are positive or zero...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: check that all distances between the 'somethings'
are all either zero or positive.
* tests/unit/computations/test_slink.py:
* Remove data with all distances positive or zero, since it would fail the
test
* Change the expected message to more closely correspond to the business
logic
| Muriithi Frederick Muriuki |
| 2021-07-22 | Check that distance from A to B is same as from B to A...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: check that the distance from child A to B is the
same as distance from child B to A. If not, throw an exception.
* tests/unit/computations/test_slink.py:
* Change the name of the test to more closely correspond to the business
logic being tested.
* Update the data in a separate test such that it does not error out due to
failing to fulfill the expectations of separate requirement.
- pass tests
- Rename test
- Fix errors: distances same both directions
| Muriithi Frederick Muriuki |
| 2021-07-22 | Check that child distance from itself is zero...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Check that a child's distance from itself is
zero. If not, throw an exception.
The children lists are a list of distances of "something" from other
"somethings". There is still some need to establish what those "somethings"
are, so that the test names can reflect the ideas that are actually being
tested for.
* tests/unit/computations/test_slink.py: Change the name of the test so that
it more closely corresponds to the business logic it is actually testing,
and not the mechanics of testing the idea.
| Muriithi Frederick Muriuki |
| 2021-07-21 | Extract tests from code in GN1...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* gn3/computations/slink.py: Add dummy `nearest' function
* tests/unit/computations/test_slink.py: Add some tests
This commit adds tests to try and reproduce the working of the `nearest'
function in:
https://github.com/genenetwork/genenetwork1/blob/master/web/webqtl/heatmap/slink.py
This commit might not yet have extracted all the expected behaviour of the
`nearest' function, and therefore, there is a possibility of a later commit
augmenting the work in this commit.
| Muriithi Frederick Muriuki |
| 2021-07-20 | Add test for code to move over from GN1...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/clustering.gmi
* .gitignore: ignore emacs temporary files
* gn3/computations/correlations2.py: add a dummy function
* tests/unit/computations/test_correlation.py: add unit tests for the function
As part of the move of the clustering and heatmap code over from GN1 to GN3,
this commit begins by providing some unit tests for the correlation function
used to ensure that the implementation that is built up here corresponds,
and produces the same results as the original.
This tests and the function might change in the new system, but for now, we
try and maintain bug-to-bug compatibility.
| Muriithi Frederick Muriuki |
