1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
|
#+STARTUP: inlineimages
#+TITLE: Quality Control Application
** Project Goals
The project seeks to handle the checking of data files for correct syntax and
other errors before allowing the code to be uploaded.
The files are *"tab-separated"* values (TSV) files, and must conform to the
following criteria:
*** Line-Level Checks
- Must be tab-separated
-
*** Cell-Level Checks
- No empty data cells
- no data cells with spurious characters like `eeeee`, `5.555iloveguix`, etc.
- decimal numbers must conform to the following criteria:
- - when checking an average file decimal numbers must contain exactly three places to the right side of the dot.
- - when checking a standard error file decimal numbers must contain six or greater places to the right side of the dot.
- - there must be a number to the left side of the dot (e.g. 0.55555 is allowed but .55555 is not).
- check line endings to make sure they are Unix and not DOS
- check strain headers against a source of truth (see strains.csv)
** Development
For reproducibility, this project is developed using guix.
To launch a guix shell for development, do:
#+BEGIN_SRC shell
guix shell --container --network --manifest=manifest.scm
#+END_SRC
to get an environment that is isolated from the rest of your system.
*** Checks
Run tests with:
#+BEGIN_SRC shell
pytest
#+END_SRC
To run the linter over the code base, run:
#+BEGIN_SRC shell
pylint tests quality_control qc_app
#+END_SRC
To check for correct type usage in the application, run:
#+BEGIN_SRC shell
mypy --show-error-codes .
#+END_SRC
** Running QC
*** Command-Line Version
Clone this repository
#+BEGIN_SRC shell
git clone http://git.genenetwork.org/fredmanglis/gnqc_py.git
#+END_SRC
then install the application
#+BEGIN_SRC shell
$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install .
#+END_SRC
To run qc against a file, the syntax is:
#+BEGIN_SRC shell
python3 -m qc [--strainsfile <strainsfile-path>] [--verbose] <filetype> <filepath>
#+END_SRC
where
- ~<filetype>~ is one of "*average*" or "*standard-error*"
- ~<filepath>~ is either an absolute path to the file, or a path relative to the
current working directory
- if the ~--strainsfile~ option is not provided, it will default to the one in
the root directory of this repository
- the ~--verbose~ option is a flag, defaulting to ~False~ that controls the
display of optional progress messages
To view the usage information for the application, run
#+BEGIN_SRC shell
python3 -m qc --help
#+END_SRC
**** TODO Figure out how to put qc.py in a /bin or /scripts directory and still be able to import the modules in the repo
**** TODO Reduce the command to simply ~qc [--strainsfile <strainsfile-path>] [--verbose] <filetype> <filepath>~
*** Web Version
Coming soon...
|
