| Age | Commit message (Collapse) | Author |
|
gn3/computations/gemma.py's generate_gemma_cmd always appended "-gk"
to every composed command, including its gwa-compute/k-gwa-compute
endpoints that also set -lmm. Confirmed by actually running the
composed command against real gemma-wrapper/GEMMA: real GEMMA rejects
that combination outright ("only one of -gk ... -lmm ... options is
allowed"), so none of these endpoints could ever produce output.
tests/integration/test_gemma.py was entirely @unittest.skip, so this
was never caught.
Checked that GeneNetwork2 never called these endpoints over HTTP (no
reference to api/gemma, k-compute, gwa-compute or k-gwa-compute
anywhere in its codebase) -- it runs gemma-wrapper itself directly
(wqflask/marker_regression/gemma_mapping.py), via its own
GEMMA_WRAPPER_COMMAND and os.system() calls, never through GN3's API.
So these endpoints had no real caller on either side.
Removed:
- gn3/api/gemma.py and its blueprint registration in gn3/app.py.
- generate_gemma_cmd and generate_pheno_txt_file from
gn3/computations/gemma.py (gn3/api/gemma.py was their only caller).
generate_hash_of_string stays: GeneNetwork2 imports it directly
(gemma_mapping.py, export_traits.py, views.py), and
gn3/computations/rqtl.py uses it too.
- compose_gemma_cmd from gn3/commands.py (generate_gemma_cmd was its
only caller), and the now-unused `List` import that went with it.
- tests/integration/test_gemma.py (tested only the removed blueprint).
- The generate_gemma_cmd/generate_pheno_txt_file tests from
tests/unit/computations/test_gemma.py and the compose_gemma_cmd
tests from tests/unit/test_commands.py; generate_hash_of_string's
and compose_rqtl_cmd's own tests stay.
- CACHEDIR and GEMMA_WRAPPER_CMD from gn3/settings.json: both were
read only by the removed gn3/api/gemma.py.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
|
|
|
|
|
Fix the code to avoid using global variables holding configuration
variables to avoid the failures caused by action-at-a-distance effects
due to change of a value elsewhere.
This way, all code consistently receives the same configuration values
passed on from callers.
We pass on the following configuration settings:
* TMPDIR
* SQL_URI
* TEXTDIR
|
|
|
|
|
|
|
|
|
|
* gn3/auth/authentication/oauth2/models/oauth2token.py
(introspect_token, check_permission): Delete "pylint:
disable=[no-self-use]". R0022: Useless option value for 'disable',
'no-self-use' was moved to an optional extension, see
https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/summary.html#removed-checkers.
* gn3/auth/authentication/oauth2/grants/authorisation_code_grant.py
(delete_authorization_code): Ditto.
* tests/unit/test_heatmaps.py (test_cluster_traits):
Delete "pylint: disable=R0201". R0022: Useless option value for
'disable', 'no-self-use' was moved to an optional extension, see
https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/summary.html#removed-checkers.
* tests/unit/computations/test_partial_correlations.py
(test_tissue_correlation): Ditto.
Signed-off-by: Munyoki Kilyungi <me@bonfacemunyoki.com>
|
|
* Commit `b529c700` changes the return value to a tuple of values rather than
a single string value. Handle that in the test.
|
|
|
|
|
|
|
|
* tests/unit/computations/test_rust_correlation.py: The underlying code
changed and the test needed to be updated to test the correct thing.
|
|
|
|
* gn3/computations/correlations.py: rename function
* mypy.ini: deactivate mypy error about missing imports
* tests/integration/test_correlation.py: mock correct function
* tests/unit/computations/test_correlation.py: test correct function
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The `fix_strains` function works on the trait data, not the basic trait
info. This commit fixes the arguments passed to the function, and also some
bugs in the function.
|
|
* compute zscore function
* test case for computing zscore
* function to compute pca
* generate scree plot data
* generate new pca trait data from zscores and eigen_vec
* remove redundant functions
* generate factor loading table data
* generate pca temp dataset dict
* variable naming and error fixes
* unit test for processing factor loadings
* minor fixes for generating temp pca dataset
* pass datetime as argument to generate_pca temp dataset function
* add unittest for caching pca datasets
* cache temp datasets
* ignore missing imports for sklearn
* mypy fixes
* pylint fixes
* refactor tests for pca
* remove ununsed imports
* fix for generating pca traits vals
* mypy and code refactoring
* pep8 formatting and add docstrings
* remove comments /pep8 formatting
* sort eigen vectors based on eigen values
* minor fix for zscores
* fix for rounding variance ratios
* refactor tests
* rename module to pca
* rename datasets to traits
* fix failing tests
* fix caching function
* fixes return x and y coordinates for scree plot
* expand exception scope
* fix for deprecated numpy.matrix function
* fix for failing tests
* pep8 fixes
* remove comments
* fix merge conflict
* pylint fixes
* rename module name to test_pca
|
|
|
|
When the encoding is not specified explicitly, the system default encoding is
used. This is not recommended.
* gn3/computations/ctl.py (call_ctl_script),
gn3/computations/gemma.py (generate_pheno_txt_file),
gn3/computations/parsers.py (parse_genofile),
gn3/computations/partial_correlations.py (partial_correlations_fast),
gn3/computations/rqtl.py (process_rqtl_output, process_perm_output),
gn3/computations/wgcna.py (dump_wgcna_data, call_wgcna_script),
gn3/fs_helpers.py (jsonfile_to_dict): Explicitly specify UTF-8 to be the file
encoding.
*
tests/unit/computations/test_gemma.py (TestGemma.test_generate_pheno_txt_file),
tests/unit/computations/test_wgcna.py (TestWgcna.test_create_json_file): Test
for call to open with encoding='utf-8' argument.
|
|
* Use `with` in place of plain `open`
* Use f-strings in place of `str.format()`
* Remove string interpolation from queries - provide data as query parameters
* other minor fixes
|
|
Use pytest's `mark` feature to explicitly categorise the tests and run them
per category
|
|
Add property tests using pytest and hypothesis to test that the expected
properties hold for the
`gn3.computations.partial_correlations.dictify_by_samples`
function.
|
|
|
|
|
|
|
|
Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
|
|
Notes:
https://github.com/genenetwork/genenetwork3/pull/56#issuecomment-973798918
* As mentioned in the notes, rather than rounding to an arbitrary number of
decimal places, it is a much better practice to use approximate comparisons
of floats for the tests.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Fix some obvious linting errors and remove obsolete code
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Replace the code that was in the process of being migrated from R in
GeneNetwork1 with calls to pingouin functions that achieve the same thing.
Since the functions in this case are computing correlations and partial
correlations, rather than having home-rolled functions to do that, this
commit makes use of the tried and tested pingouin functions.
This avoids complicating our code with edge-case checks, and leverages the
performance optimisations done in pingouin.
|
|
* gn3/computations/partial_correlations.py: Remove rounding. Fix computation
of remaining covariates
*
tests/unit/computations/partial_correlations_test_data/pcor_rec_blackbox_test.txt:
reduce the number of covariates to between one (1) and three (3)
* tests/unit/computations/test_partial_correlations.py: fix some minor bugs
It turns out that the computation complexity increases exponentially, with
the number of covariates. Therefore, to get a somewhat sensible test time,
while retaining a large-ish number of tests, this commit reduces the number
of covariates to between 1 and 3.
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* When the z value is a Sequence of sequences of values, each of the internal
sequences should form a column of its own, and not a row, as it was
originally set up to do.
|
|
partial-correlations
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* To improve the usefulness of already existing code, provide the parser
function as an argument to the `parse_input_line` function.
This was found to be useful when writing code to compare the `pcor.test`
function in GN1 and the `pingouin.partial_corr` function.
The format of the data generated when getting results for the `pcor.test`
function shared a lot with that of the `pcor.rec` function, but it was
different in a few, places, and the differences were non-trivial, needing
different parsing processes.
In such a case, it was found necessary to just pass in the function to do
the actual parsing, rather than create code with the same form as the
existing one, save for the function being called.
|
|
correlations2.compute_correlation computes the Pearson correlation
coefficient. Outsource this computation to scipy.stats.pearsonr. When the
inputs are constant, the Pearson correlation coefficient does not exist and is
represented by NaN. Update the tests to reflect this.
* gn3/computations/correlations2.py: Remove import of sqrt from math.
(compute_correlation): Reimplement using scipy.stats.pearsonr.
* tests/unit/computations/test_correlation.py: Import math.
(TestCorrelation.test_compute_correlation): When inputs are constant, set
expected correlation coefficient to NaN.
|
|
Floating point numbers should only be compared approximately. Different
implementations of functions might produce slightly different results.
* tests/unit/computations/test_correlation.py: Import assert_almost_equal from
numpy.testing.
(TestCorrelation.test_compute_correlation): Compare floats using
assert_almost_equal instead of assertEqual.
* tests/unit/test_heatmaps.py: Import assert_allclose from numpy.testing.
(TestHeatmap.test_cluster_traits): Use assert_allclose instead of assertEqual.
|
|
|
|
partial-correlations
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* gn3/computations/partial_correlations.py: implement remaining portion of
`partial_correlation_recursive` function.
* tests/unit/computations/test_partial_correlations.py: add parsing for new
data format and update tests
|
|
Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Delete the older test data that was only testing with `z` being a vector and
replace it with data that has `z` being a vector or matrix from data item to
data item.
|