| Age | Commit message (Expand) | Author |
|---|
| 2021-11-22 | Add test to query builders...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Add some tests for the query builders to ensure that the queries are built
up correctly.
| Frederick Muriuki Muriithi |
| 2021-11-19 | Avoid rounding: compare floats approximately...Notes:
https://github.com/genenetwork/genenetwork3/pull/56#issuecomment-973798918
* As mentioned in the notes, rather than rounding to an arbitrary number of
decimal places, it is a much better practice to use approximate comparisons
of floats for the tests.
| Frederick Muriuki Muriithi |
| 2021-11-19 | Replace use of assert_allclose. Disable linter error...Notes:
https://github.com/genenetwork/genenetwork3/pull/56#issuecomment-973798918
* From the notes above, the assert_allclose is a better function for figuring
out what failed, unlike the allclose that simply just returns a True/False
value.
This commit restores the use of the assert_allclose function, and then
disables the linter error due to the fact that there is no use of the `self`
keyword.
| Frederick Muriuki Muriithi |
| 2021-11-18 | Fix some linting errors...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Fix some obvious linting errors and remove obsolete code
| Frederick Muriuki Muriithi |
| 2021-11-18 | Replace code migrated from R with pingouin functions...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Replace the code that was in the process of being migrated from R in
GeneNetwork1 with calls to pingouin functions that achieve the same thing.
Since the functions in this case are computing correlations and partial
correlations, rather than having home-rolled functions to do that, this
commit makes use of the tried and tested pingouin functions.
This avoids complicating our code with edge-case checks, and leverages the
performance optimisations done in pingouin.
| Frederick Muriuki Muriithi |
| 2021-11-15 | Fix bugs in recursive partial correlations...* gn3/computations/partial_correlations.py: Remove rounding. Fix computation
of remaining covariates
*
tests/unit/computations/partial_correlations_test_data/pcor_rec_blackbox_test.txt:
reduce the number of covariates to between one (1) and three (3)
* tests/unit/computations/test_partial_correlations.py: fix some minor bugs
It turns out that the computation complexity increases exponentially, with
the number of covariates. Therefore, to get a somewhat sensible test time,
while retaining a large-ish number of tests, this commit reduces the number
of covariates to between 1 and 3.
| Frederick Muriuki Muriithi |
| 2021-11-15 | Fix the columns in built data frame...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* When the z value is a Sequence of sequences of values, each of the internal
sequences should form a column of its own, and not a row, as it was
originally set up to do.
| Frederick Muriuki Muriithi |
| 2021-11-12 | Merge branch 'main' of github.com:genenetwork/genenetwork3 into partial-corre... | Frederick Muriuki Muriithi |
| 2021-11-12 | Pass in parser function for flexibility...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* To improve the usefulness of already existing code, provide the parser
function as an argument to the `parse_input_line` function.
This was found to be useful when writing code to compare the `pcor.test`
function in GN1 and the `pingouin.partial_corr` function.
The format of the data generated when getting results for the `pcor.test`
function shared a lot with that of the `pcor.rec` function, but it was
different in a few, places, and the differences were non-trivial, needing
different parsing processes.
In such a case, it was found necessary to just pass in the function to do
the actual parsing, rather than create code with the same form as the
existing one, save for the function being called.
| Frederick Muriuki Muriithi |
| 2021-11-11 | Reimplement correlations2.compute_correlation using pearsonr....correlations2.compute_correlation computes the Pearson correlation
coefficient. Outsource this computation to scipy.stats.pearsonr. When the
inputs are constant, the Pearson correlation coefficient does not exist and is
represented by NaN. Update the tests to reflect this.
* gn3/computations/correlations2.py: Remove import of sqrt from math.
(compute_correlation): Reimplement using scipy.stats.pearsonr.
* tests/unit/computations/test_correlation.py: Import math.
(TestCorrelation.test_compute_correlation): When inputs are constant, set
expected correlation coefficient to NaN.
| Arun Isaac |
| 2021-11-11 | Compare floats approximately....Floating point numbers should only be compared approximately. Different
implementations of functions might produce slightly different results.
* tests/unit/computations/test_correlation.py: Import assert_almost_equal from
numpy.testing.
(TestCorrelation.test_compute_correlation): Compare floats using
assert_almost_equal instead of assertEqual.
* tests/unit/test_heatmaps.py: Import assert_allclose from numpy.testing.
(TestHeatmap.test_cluster_traits): Use assert_allclose instead of assertEqual.
| Arun Isaac |
| 2021-11-11 | pep8 formatting;update unittests | Alexander Kabui |
| 2021-11-09 | Merge branch 'main' of github.com:genenetwork/genenetwork3 into partial-corre... | Frederick Muriuki Muriithi |
| 2021-11-09 | Implement remaining part of `partial_correlation_recursive` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: implement remaining portion of
`partial_correlation_recursive` function.
* tests/unit/computations/test_partial_correlations.py: add parsing for new
data format and update tests
| Frederick Muriuki Muriithi |
| 2021-11-09 | Update test data: have both vectors and matrices...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Delete the older test data that was only testing with `z` being a vector and
replace it with data that has `z` being a vector or matrix from data item to
data item.
| Frederick Muriuki Muriithi |
| 2021-11-09 | Add new data processing utility...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/data_helpers.py: New function (`partition_by`)
* tests/unit/test_data_helpers.py: Tests for new function
Add a function that approximates Clojure's `partition-by` function, to help
with processing the data in a more functional way.
| Frederick Muriuki Muriithi |
| 2021-11-04 | test_partial_correlations: skip failing tests...Fix these later. I need a passing test suite so as to update the gn2 docker
image.
| BonfaceKilz |
| 2021-11-04 | Create blackbox tests for some functions migrated from R...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: new stub
functions (partial_correlation_matrix, partial_correlation_recursive)
*
tests/unit/computations/partial_correlations_test_data/pcor_mat_blackbox_test.csv:
blackbox sample data and results for variance-covariance matrix method
*
tests/unit/computations/partial_correlations_test_data/pcor_rec_blackbox_test.csv:
blackbox sample data and results for recursive method
* tests/unit/computations/test_partial_correlations.py: Tests for new function
Provide some blackbox testing sample data for checking the operation of the
functions migrated from R.
| Frederick Muriuki Muriithi |
| 2021-11-04 | Retrieve indices of the selected samples...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: New
function (good_dataset_samples_indexes).
* tests/unit/computations/test_partial_correlations.py: Tests for new
function (good_dataset_samples_indexes)
Get the indices of the selected samples. This is a partial migration of the
`web.webqtl.correlation.PartialCorrDBPage.getPartialCorrelationsFast`
function in GN1.
| Frederick Muriuki Muriithi |
| 2021-11-04 | Parse single line from CSV file...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/data_helpers.py: New function (parse_csv_line)
* tests/unit/test_data_helpers.py: Add tests for new function (parse_csv_line)
Add a function to parse a single line from a CSV file.
| Frederick Muriuki Muriithi |
| 2021-11-04 | Specify ten (10) decimal places...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: specify 10 decimal places
* tests/unit/computations/test_partial_correlations.py: update examples
Slight differences in python implementations, possibly hardware and
operating systems could cause the value of float (double) values to be
different in the less significant parts of the decimal places.
This commit limits the usable part of the decimals to the first 10 decimal
places for now.
| Frederick Muriuki Muriithi |
| 2021-11-04 | Fix some linting errors...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
| Frederick Muriuki Muriithi |
| 2021-11-04 | Implement `tissue_correlation` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: New function (tissue_correlation)
* tests/unit/test_partial_correlations.py ->
tests/unit/computations/test_partial_correlations.py: Move module. Implement
tests for new function
Migrate the `cal_tissue_corr` function embedded in the
`web.webqtl.correlation.correlationFunction.batchCalTissueCorr` function in
GN1 and implement tests to ensure it works correctly.
| Frederick Muriuki Muriithi |
| 2021-11-04 | Partially implement `partial_correlation_recursive`...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: Implement one path for the
`gn3.computations.partial_correlations.partial_correlation_recursive`
function.
* gn3/settings.py: Add a setting for how many decimal places to round to
* tests/unit/computations/test_partial_correlations.py: Update test to take
the number of decimal places into consideration
Implement a single path (where the z value is a vector and not a matrix) for
the `partial_correlation_recursive` function.
| Frederick Muriuki Muriithi |
| 2021-11-04 | Implement `build_data_frame`...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: new function (`build_data_frame`)
* tests/unit/computations/test_partial_correlations.py: Add tests for new
function
Add a new function to build a pandas DataFrame object from the provided
values:
- x: a vector of floats (represented with a tuple of floats)
- y: a vector of floats (represented with a tuple of floats)
- z: a vector OR matrix of floats (represented with a tuple of floats or a
tuple of tuples of floats)
| Frederick Muriuki Muriithi |
| 2021-11-04 | Create blackbox tests for some functions migrated from R...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: new stub
functions (partial_correlation_matrix, partial_correlation_recursive)
*
tests/unit/computations/partial_correlations_test_data/pcor_mat_blackbox_test.csv:
blackbox sample data and results for variance-covariance matrix method
*
tests/unit/computations/partial_correlations_test_data/pcor_rec_blackbox_test.csv:
blackbox sample data and results for recursive method
* tests/unit/computations/test_partial_correlations.py: Tests for new function
Provide some blackbox testing sample data for checking the operation of the
functions migrated from R.
| Frederick Muriuki Muriithi |
| 2021-11-01 | Add test cases for gn3.authentication | BonfaceKilz |
| 2021-11-01 | Merge branch 'main' of github.com:genenetwork/genenetwork3 into partial-corre... | Frederick Muriuki Muriithi |
| 2021-11-01 | Retrieve indices of the selected samples...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: New
function (good_dataset_samples_indexes).
* tests/unit/computations/test_partial_correlations.py: Tests for new
function (good_dataset_samples_indexes)
Get the indices of the selected samples. This is a partial migration of the
`web.webqtl.correlation.PartialCorrDBPage.getPartialCorrelationsFast`
function in GN1.
| Frederick Muriuki Muriithi |
| 2021-11-01 | Parse single line from CSV file...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/data_helpers.py: New function (parse_csv_line)
* tests/unit/test_data_helpers.py: Add tests for new function (parse_csv_line)
Add a function to parse a single line from a CSV file.
| Frederick Muriuki Muriithi |
| 2021-10-29 | Feature/biweight reimplementation (#47)...* add biweight reimplementation with pingouin
* delete biweight scripts and tests
* add python-pingouin to guix file
* delete biweight paths
* mypy fix:pingouin mising imports
* pep8 formatting && pylint fixes | Alexander Kabui |
| 2021-10-29 | Specify ten (10) decimal places...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: specify 10 decimal places
* tests/unit/computations/test_partial_correlations.py: update examples
Slight differences in python implementations, possibly hardware and
operating systems could cause the value of float (double) values to be
different in the less significant parts of the decimal places.
This commit limits the usable part of the decimals to the first 10 decimal
places for now.
| Frederick Muriuki Muriithi |
| 2021-10-29 | Fix some linting errors...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
| Frederick Muriuki Muriithi |
| 2021-10-29 | Implement `tissue_correlation` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: New function (tissue_correlation)
* tests/unit/test_partial_correlations.py ->
tests/unit/computations/test_partial_correlations.py: Move module. Implement
tests for new function
Migrate the `cal_tissue_corr` function embedded in the
`web.webqtl.correlation.correlationFunction.batchCalTissueCorr` function in
GN1 and implement tests to ensure it works correctly.
| Frederick Muriuki Muriithi |
| 2021-10-25 | Implement `partition_all` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/data_helpers.py: new function (partition_all)
* tests/unit/test_data_helpers.py: tests for function
`gn3.data_helpers.partition_all`
As part of migrating some functions that access the database, this commit
extracts generic processes that can be accomplished on data, and implements
the `partition_all` function, that is equivalent to Clojure's
`partition-all` function.
| Frederick Muriuki Muriithi |
| 2021-10-25 | Merge branch 'main' of github.com:genenetwork/genenetwork3 into partial-corre... | Frederick Muriuki Muriithi |
| 2021-10-25 | fix unittests | Alexander Kabui |
| 2021-10-21 | Implement `find_identical_traits` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: implement function `find_identical_traits`
* tests/unit/test_partial_correlations.py: implement tests for function
`find_identical_traits`
Migrate `web.webqtl.correlation.correlationFunction.findIdenticalTraits`
function in GN1 to here, adding in tests to ensure the migration works in a
bug-compatible version with the original.
| Frederick Muriuki Muriithi |
| 2021-10-21 | Document tests better...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Document the issues better to help with understanding what each test checks
for.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Implement remaining `fix_samples` functionality...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: implement `fix_samples` function
* tests/unit/test_partial_correlations.py: implement tests for `fix_samples`
function
Implement the remaining partial migration for the
`web.webqtl.correlation.correlationFunction.fixStrain` function in GN1.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Implement `dictify_by_samples`...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: implement `dictify_by_samples` function
* tests/unit/test_partial_correlations.py: implement tests for
`dictify_by_samples` function
Implement the `dictify_by_samples` function as a partial migration of the
`web.webqtl.correlation.correlationFunction.fixStrains` function from GN1.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Disable pylint issue...* Disable minor pylint issue.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Implement `control_samples` function as is in GN1...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: Implement `control_samples` function
* tests/unit/test_partial_correlations.py: add tests for `control_samples`
function
Implement the function `control_samples` and make it mostly bug-compatible
with the `web/webqtl/correlation/correlationFunction.controlStrain` function
in GN1.
This implementation in GN3 does not do any calls to the database. It will
rely on other functions to provide the data from the database to it.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Move `export_informative` function to `gn3.db.traits` module...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/db/traits.py: Move `export_informative` function here
* gn3/partial_correlations.py: Remove `export_informative` function
* tests/unit/db/test_traits.py: Move `export_informative` function tests here
* tests/unit/test_partial_correlations.py: Remove `export_informative`
function tests
The `export_informative` function relates more to the traits than to the
partial correlations, and could find use in more than just the partial
correlations stuff. This commit moves the function to the more
traits-specific `gn3.db.traits` module.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Move 'export_trait_data' to 'gn3.db.traits' module...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/db/traits.py: Move function `export_trait_data` here
* gn3/heatmaps.py: Remove function `export_trait_data`
* tests/unit/db/test_traits.py: Move function `export_trait_data` tests here
* tests/unit/test_heatmaps.py: Remove function `export_trait_data` here
Function `export_trait_data` more closely corresponds to the traits and is
used in more than just the `gn3.heatmaps` module. This commit moves the
relevant code over to the `gn3.db.traits` module and also moves the tests to
the corresponding tests modules.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Migrate `export_informative` function...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: Implement a mostly, bug-compatible
`export_informative` function as part of migrating the partial correlations
feature over to GN3 from GN1
* tests/unit/test_partial_correlations.py: Implement tests to ensure the code
work in a similar manner as that one in GN1.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Implement remaining `fix_samples` functionality...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: implement `fix_samples` function
* tests/unit/test_partial_correlations.py: implement tests for `fix_samples`
function
Implement the remaining partial migration for the
`web.webqtl.correlation.correlationFunction.fixStrain` function in GN1.
| Frederick Muriuki Muriithi |
| 2021-10-19 | Implement `dictify_by_samples`...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: implement `dictify_by_samples` function
* tests/unit/test_partial_correlations.py: implement tests for
`dictify_by_samples` function
Implement the `dictify_by_samples` function as a partial migration of the
`web.webqtl.correlation.correlationFunction.fixStrains` function from GN1.
| Frederick Muriuki Muriithi |
| 2021-10-18 | Disable pylint issue...* Disable minor pylint issue.
| Frederick Muriuki Muriithi |
| 2021-10-18 | Implement `control_samples` function as is in GN1...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/partial_correlations.py: Implement `control_samples` function
* tests/unit/test_partial_correlations.py: add tests for `control_samples`
function
Implement the function `control_samples` and make it mostly bug-compatible
with the `web/webqtl/correlation/correlationFunction.controlStrain` function
in GN1.
This implementation in GN3 does not do any calls to the database. It will
rely on other functions to provide the data from the database to it.
| Frederick Muriuki Muriithi |
