| Age | Commit message (Expand) | Author |
|---|
| 2022-03-18 | Clean all csv fields before diffing...There was a subtle bug where "csvdiff" generated an error related to
"different column headings" caused something akin to diffing: "a, b \n, ..."
with "a, b\n, ...".
* gn3/csvcmp.py (csv_diff): Clean csv texts before any diffing.
* tests/unit/test_csvcmp.py (test_csv_diff_same_columns): Modify test case to
capture aforementioned bug.
| BonfaceKilz |
| 2022-03-18 | Create new function for cleaning individual fields in csv text...* gn3/csvcmp.py (clean_csv_text): New function.
* tests/unit/test_csvcmp.py: Import "csv_text".
(test_clean_csv_text): Test case for the above.
| BonfaceKilz |
| 2022-03-15 | Feature/refactored pca (#79)...* compute zscore function
* test case for computing zscore
* function to compute pca
* generate scree plot data
* generate new pca trait data from zscores and eigen_vec
* remove redundant functions
* generate factor loading table data
* generate pca temp dataset dict
* variable naming and error fixes
* unit test for processing factor loadings
* minor fixes for generating temp pca dataset
* pass datetime as argument to generate_pca temp dataset function
* add unittest for caching pca datasets
* cache temp datasets
* ignore missing imports for sklearn
* mypy fixes
* pylint fixes
* refactor tests for pca
* remove ununsed imports
* fix for generating pca traits vals
* mypy and code refactoring
* pep8 formatting and add docstrings
* remove comments /pep8 formatting
* sort eigen vectors based on eigen values
* minor fix for zscores
* fix for rounding variance ratios
* refactor tests
* rename module to pca
* rename datasets to traits
* fix failing tests
* fix caching function
* fixes return x and y coordinates for scree plot
* expand exception scope
* fix for deprecated numpy.matrix function
* fix for failing tests
* pep8 fixes
* remove comments
* fix merge conflict
* pylint fixes
* rename module name to test_pca | Alexander Kabui |
| 2022-03-14 | Sort import lines | BonfaceKilz |
| 2022-03-14 | Given a csv text and permissible headers, extract invalid headers...* gn3/csvcmp.py (extract_invalid_csv_headers): New function.
* tests/unit/test_csvcmp.py: Import "extract_invalid_csv_headers".
(test_extract_invalid_csv_headers_with_some_wrong_headers): Test case for the
above.
| BonfaceKilz |
| 2022-03-14 | Get all permissible column data...* gn3/csvcmp.py: Import "Any" and "List".
(get_allowable_sampledata_headers): New function.
* tests/unit/test_csvcmp: Import "get_allowable_sampledata_headers".
(test_get_allowable_csv_headers): Test case for the above.
| BonfaceKilz |
| 2022-03-12 | Fix pylint errors in unit_tests | BonfaceKilz |
| 2022-03-12 | Append the strain name when extracting "actions"...* gn3/db/sample_data.py (__extract_actions): During updates, make sure that
the strain name is part of the returned string when extracting "actions".
* tests/unit/db/test_sample_data.py: Add test cases for the above.
| BonfaceKilz |
| 2022-03-12 | Apply auto-pep8 to sample_data.py and it's test file | BonfaceKilz |
| 2022-03-12 | Update how data is updated by re-using existing functions...* gn3/db/sample_data.py (get_sample_data_ids): Re-use "delete_sample_data" and
"insert_sample_data" when updating data; and also add logic for updating
modified data.
* tests/unit/db/test_sample_data.py: Add tests for the above.
| BonfaceKilz |
| 2022-03-12 | Create action dict that's created when updating data...* gn3/db/sample_data.py (__extract_actions): An update on a vector of data can
contain: inserts, deletes and updates. This functions extracts these actions
during an update.
* tests/unit/db/test_sample_data.py (test_extract_actions): Add test-case for
the above.
| BonfaceKilz |
| 2022-03-12 | Add test cases for inserting and deleting data...* tests/unit/db/test_sample_data.py (test_insert_sample_data): Test inserting
data.
(test_delete_sample_data): Test deleting data.
| BonfaceKilz |
| 2022-03-12 | Fill in empty values in csv text with: "x"...* gn3/csvcmp.py (fill_csv): Update this function to allow empty lists to be
filled with the default value(set in the args).
* tests/unit/test_csvcmp.py (test_fill_csv): Update test to capture above.
| BonfaceKilz |
| 2022-03-12 | Remove test cases related to sample data...Most of this functions were moved to their own module.
| BonfaceKilz |
| 2022-03-12 | Extract a strain name given a csv string and it's header...* gn3/csvcmp.py (extract_strain_name): New function.
* gn3/db/sample_data (delete_sample_data): Use the aforementioned function.
(insert_sample_data): Ditto.
* tests/unit/test_csvcmp: Test cases for above.
| BonfaceKilz |
| 2022-03-12 | Don't add extra key "Column" to dict if there are no changes...gn3/csvcmp.py (csv_diff): If the diff is empty, don't add an extra key
"Column" to the dictionary.
tests/unit/test_csvcmp (test_csv_diff_only_column_change): Add test-case for
the above.
| BonfaceKilz |
| 2022-03-12 | Test edges cases for csv files when running csvdiff...* tests/unit/test_csvcmp.py (test_csv_diff): Delete it.
(test_csv_diff_same_columns): Test csv_diff against csv texts with the same
columns.
(test_csv_diff_different_columns): Test csv texts against csv texts with
different varying columns.
| BonfaceKilz |
| 2022-03-12 | Create new method for filling csv with a default value...* gn3/csvcmp.py (fill_csv): Given a csv text with uneven or incomplete fields
whole length are less than width, fill them with a value which defaults to
"x".
* tests/unit/test_csvcmp.py (test_fill_csv): Test cases for the above.
| BonfaceKilz |
| 2022-03-12 | Add methods for working with csv data...gn3/csvcmp.py: New file
(create_dirs_if_not_exists): From a list of dirs, create them if they don't
exist.
(remove_insignificant_edits): Given a dict with a "Modification" key, remove
edits with "delta < ε".
(csv_diff): Generate a csv_diff using the "csvdiff" tool packaged in guix.
tests/unit/test_csvcmp.py: Add some tests for "gn3/csvcmp.py"
| BonfaceKilz |
| 2022-03-12 | Append case attributes to csv data if they exist | BonfaceKilz |
| 2022-03-11 | Fix some linting issues | Frederick Muriuki Muriithi |
| 2022-03-08 | Remove unused function and its tests | Frederick Muriuki Muriithi |
| 2022-03-08 | Fix tests, and issues caught by tests...Fix some issues caught by tests due to changes introducing the hand-off of the
partial correlations computations to an external process
Fix some issues due to the changes that introduce context managers for
database connections
Update some tests to take the above two changes into consideration
| Frederick Muriuki Muriithi |
| 2022-02-24 | gn3: Explicitly specify UTF-8 to be the file encoding....When the encoding is not specified explicitly, the system default encoding is
used. This is not recommended.
* gn3/computations/ctl.py (call_ctl_script),
gn3/computations/gemma.py (generate_pheno_txt_file),
gn3/computations/parsers.py (parse_genofile),
gn3/computations/partial_correlations.py (partial_correlations_fast),
gn3/computations/rqtl.py (process_rqtl_output, process_perm_output),
gn3/computations/wgcna.py (dump_wgcna_data, call_wgcna_script),
gn3/fs_helpers.py (jsonfile_to_dict): Explicitly specify UTF-8 to be the file
encoding.
*
tests/unit/computations/test_gemma.py (TestGemma.test_generate_pheno_txt_file),
tests/unit/computations/test_wgcna.py (TestWgcna.test_create_json_file): Test
for call to open with encoding='utf-8' argument.
| Arun Isaac |
| 2022-02-21 | Fix minor issues introduced while fixing linting errors | Frederick Muriuki Muriithi |
| 2022-02-21 | Fix a myriad of linter issues...* Use `with` in place of plain `open`
* Use f-strings in place of `str.format()`
* Remove string interpolation from queries - provide data as query parameters
* other minor fixes
| Frederick Muriuki Muriithi |
| 2022-02-21 | Add test to ensure samples are read correctly from .geno files...* tests/unit/db/test_genotypes2.py: New file
| Frederick Muriuki Muriithi |
| 2022-02-21 | Test partial corrs API with mix of existing and non-existing control traits...Test that the partial correlations endpoint handles a mix of existing and
non-existing control traits gracefully and issues a warning to the user.
Summary of changes:
* gn3/computations/partial_correlations.py: Issue a warning for all
non-existing control traits
* gn3/db/partial_correlations.py: update queries - use `INNER JOIN` for tables
instead of comma-separated list of tables
* tests/integration/conftest.py: Add `db_conn` fixture to provide a database
connection to the tests. This will probably be changed in the future to
connect to a temporary database for tests.
* tests/integration/test_partial_correlations.py: Add test to check for
correct behaviour with a mix of existing and non-existing control traits
| Frederick Muriuki Muriithi |
| 2022-02-19 | Test partial corrs endpoint with non-existing control traits...Test that if the endpoint is queried and not a single one of the control
traits exists in the database, then the endpoint will respond with a
404 (not-found) status code.
Summary of changes:
* gn3/computations/partial_correlations.py: Check whether any control trait is
found. If none is found, return "not-found" message.
* gn3/db/partial_correlations.py: Fix bug in Geno query.
* tests/integration/test_partial_correlations.py: Add test for non-existing
control traits. Rename function to make it clearer what it is testing
for. Remove obsoleted comments.
| Frederick Muriuki Muriithi |
| 2022-02-18 | Remove erroneous test based on wrong assumptions...Related to commit 75dcfe295af57b16428c586cc11dbaa827a5feba
This commit removes the related test that was checking for the wrong thing.
| Frederick Muriuki Muriithi |
| 2022-02-18 | Test partial correlations endpoint with non-existent primary traits...Test that the partial correlations endpoint responds with an appropriate
"not-found" message and the corresponding 404 status code in the case where a
request is made and the primary trait requested for does not exist in the
database.
Summary of the changes in each file:
* gn3/api/correlation.py: generalise the building of the response
* gn3/computations/partial_correlations.py: return with a "not-found" if the
primary trait does not exist in the database
* gn3/db/partial_correlations.py: Fix a number of bugs that led to exceptions
in the case that the primary trait did not exist
* pytest.ini: register a `slow` pytest marker
* tests/integration/test_partial_correlations.py: Add a new test to check for
an appropriate 404 response in case of a primary trait that does not exist
in the database.
| Frederick Muriuki Muriithi |
| 2022-02-17 | Test partial correlations endpoint with missing data in POST request...Add a test for the partial correlations endpoint, with:
- no data in the request
- missing items in the data
Fix the bugs caught by the test
| Frederick Muriuki Muriithi |
| 2022-02-17 | Create client fixture for integration tests...Create a client fixture to help with the integration tests
| Frederick Muriuki Muriithi |
| 2022-02-17 | Use pytest's "mark" feature to categorise tests...Use pytest's `mark` feature to explicitly categorise the tests and run them
per category
| Frederick Muriuki Muriithi |
| 2022-02-17 | Add property tests for `dictify_by_samples`...Add property tests using pytest and hypothesis to test that the expected
properties hold for the
`gn3.computations.partial_correlations.dictify_by_samples`
function.
| Frederick Muriuki Muriithi |
| 2022-02-09 | Remove obsolete `PublishFreeze` table | Frederick Muriuki Muriithi |
| 2022-01-05 | test_traits.py: Update failing test | BonfaceKilz |
| 2021-12-14 | linting: Fix obvious linting issues | Frederick Muriuki Muriithi |
| 2021-12-14 | Provide expected values to use for tests | Frederick Muriuki Muriithi |
| 2021-12-10 | refactor sample r unittests | Alexander Kabui |
| 2021-12-10 | refactor unittest for normalizing sample values | Alexander Kabui |
| 2021-11-29 | Update test for data changes...Issue: https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
| Frederick Muriuki Muriithi |
| 2021-11-23 | Fix a myriad of linting errors...* Fix linting errors like:
- Unused variables
- Undeclared variable errors (mostly caused by typos, and wrong names)
- Missing documentation strings for functions
etc.
| Frederick Muriuki Muriithi |
| 2021-11-22 | Add test to query builders...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Add some tests for the query builders to ensure that the queries are built
up correctly.
| Frederick Muriuki Muriithi |
| 2021-11-19 | Avoid rounding: compare floats approximately...Notes:
https://github.com/genenetwork/genenetwork3/pull/56#issuecomment-973798918
* As mentioned in the notes, rather than rounding to an arbitrary number of
decimal places, it is a much better practice to use approximate comparisons
of floats for the tests.
| Frederick Muriuki Muriithi |
| 2021-11-19 | Replace use of assert_allclose. Disable linter error...Notes:
https://github.com/genenetwork/genenetwork3/pull/56#issuecomment-973798918
* From the notes above, the assert_allclose is a better function for figuring
out what failed, unlike the allclose that simply just returns a True/False
value.
This commit restores the use of the assert_allclose function, and then
disables the linter error due to the fact that there is no use of the `self`
keyword.
| Frederick Muriuki Muriithi |
| 2021-11-18 | Fix some linting errors...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Fix some obvious linting errors and remove obsolete code
| Frederick Muriuki Muriithi |
| 2021-11-18 | Replace code migrated from R with pingouin functions...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* Replace the code that was in the process of being migrated from R in
GeneNetwork1 with calls to pingouin functions that achieve the same thing.
Since the functions in this case are computing correlations and partial
correlations, rather than having home-rolled functions to do that, this
commit makes use of the tried and tested pingouin functions.
This avoids complicating our code with edge-case checks, and leverages the
performance optimisations done in pingouin.
| Frederick Muriuki Muriithi |
| 2021-11-15 | Fix bugs in recursive partial correlations...* gn3/computations/partial_correlations.py: Remove rounding. Fix computation
of remaining covariates
*
tests/unit/computations/partial_correlations_test_data/pcor_rec_blackbox_test.txt:
reduce the number of covariates to between one (1) and three (3)
* tests/unit/computations/test_partial_correlations.py: fix some minor bugs
It turns out that the computation complexity increases exponentially, with
the number of covariates. Therefore, to get a somewhat sensible test time,
while retaining a large-ish number of tests, this commit reduces the number
of covariates to between 1 and 3.
| Frederick Muriuki Muriithi |
| 2021-11-15 | Fix the columns in built data frame...Issue:
https://github.com/genenetwork/gn-gemtext-threads/blob/main/topics/gn1-migration-to-gn2/partial-correlations.gmi
* When the z value is a Sequence of sequences of values, each of the internal
sequences should form a column of its own, and not a row, as it was
originally set up to do.
| Frederick Muriuki Muriithi |
