From 4406c59eb37a28a7e499443427dc05ef46dc9da9 Mon Sep 17 00:00:00 2001 From: Frederick Muriuki Muriithi Date: Fri, 16 Feb 2024 05:35:14 +0300 Subject: Documentation: Add section on configurations. --- README.org | 127 +++++++++++++++++++++++++++++++++++++++++++++++++- etc/default_config.py | 4 +- 2 files changed, 127 insertions(+), 4 deletions(-) diff --git a/README.org b/README.org index d7a0dcb..9e0324a 100644 --- a/README.org +++ b/README.org @@ -1,5 +1,6 @@ #+STARTUP: inlineimages #+TITLE: GeneNetwork Quality Control Application +#+OPTIONS: ^:{} ** Project Goals @@ -40,6 +41,130 @@ This repository can be used as a guix channel. "F370 F409 854B 90E3 52F3 AB01 362B 0BB8 B81D 5A42")))) #+END_SRC +** Configuration + +This system requires that some configurations be in place to operate correctly. +The following sections detail the configuration variables. + +*** System Configurations + +**** *Config*: =LOG_LEVEL= +*Default*: WARNING + +We use logging to track the operation of the system, noting major events, and +conditions in the system as it runs. The =LOG_LEVEL= variable controls how much +of that information we want to track, and the severity of said information. + +**** *Config*: =SECRET_KEY= + +*Mandatory* + +This is a key used for signing cookies for more secure tamper-proof sessions. +Without it, Flask will generate a random key for each running thread/process of +the system, leading to some really difficult to debug consequences. + +/Avoid the aggravation! Just set a secret key and be done with it/. + +For some more information on Flask's =SECRET_KEY= have a look at the following +resources: + +- https://flask.palletsprojects.com/en/3.0.x/config/#SECRET_KEY +- https://flask.palletsprojects.com/en/3.0.x/quickstart/#sessions +- https://stackoverflow.com/a/48596852 + +**** *Config*: `UPLOAD_FOLDER` + +*Default*: /tmp/qc_app_files + +This is the directory where all uploaded files are stored. + +Files are to be stored here temporary, while they are still being processed. +Down the line, we will probably have a different location for files that have +passed QC and whose data has been accepted into the system. + +*Development Note*: Other files we store here that we will need to extract to a +separate configuration are: + +- Files with outputs from background processes triggered by this system +- /list others here…/ + +**** *Config*: =SQL_URI= +*Mandatory* + +This defines the connection URI to the database. + +This is a *MANDATORY* configuration variable: without it, the uploader will not +work as expected, if at all. + +Presently, we use the MariaDB database, and thus, the URI to the database will +take the form + +#+BEGIN_EXAMPLE +mysql://:@:/ +#+END_EXAMPLE + +The parts of the URI of note are: + +- ==: This is the username to use to connect to the MariaDB server +- ==: A password to authenticate the user with the MariaDB server +- ==: The domain name or IP address of the computer hosting the + MariaDB server +- ==: The port to connect to for the MariaDB server, on the computer + hosting MariaDB +- ==: The database to connect to + +*** Redis Configurations + +**** *Config*: =REDIS_URL= + +*Default*: redis:// + +This is the connection URI to the Redis server. The default connection parameter +above tries to connect to a redis server on the same host that the uploader is +running on. + +*Note to Fred*: We should probably rename this =REDIS_URI= for consistency. + +The Redis connection URI has the general form: + +#+BEGIN_EXAMPLE +redis://:@:/ +#+END_EXAMPLE + +and the parts of the URI of note are: + +- ==: This is the username to use to connect to the Redis server +- ==: A password to authenticate the user with the Redis server +- ==: The domain name or IP address of the computer hosting the + Redis server +- ==: The port to connect to for the Redis server, on the computer + hosting Redis +- ==: A number identifying the database to connect to + + Here is [a resource](https://github.com/lettuce-io/lettuce-core/wiki/Redis-URI-and-connection-details) + on other ways to construct Redis connection URIs. + +**** *Config*: =JOBS_TTL_SECONDS= + +*Default*: 1209600 seconds (14 days) + +This is the "Time-To-Live" for the keys in redis. Most of the keys we store in +Redis from this system do not need to be retained forever. The TTL determines +how long (in seconds) to keep the keys around for. + +**** *Config*: =GNQC_REDIS_PREFIX= + +*Default*: GNQC + +This system could find itself sharing a Redis server with other (related) +applications such as [GN2](https://github.com/genenetwork/genenetwork2) and +[GN3](https://github.com/genenetwork/genenetwork3). + +The =GNQC_REDIS_PREFIX= configuration is prepended to the keys being saved by +this system to redis. For example, if ~GNQC_REDIS_PREFIX=GNQC~ and you want to +set up a hash with the key =users=, the effective key in redis will be +~GNQC:users~. + ** Development For reproducibility, this project is developed using guix. @@ -136,7 +261,7 @@ guix package [-p /path/to/qc/profile] -f guix.scm :CUSTOM_ID: run-cli-version :END: -Install the application as shown in the [[Installing QC]] section above. +Install the application as shown in the /Installing QC/ section above. To run qc against a file, the syntax is: #+BEGIN_SRC shell diff --git a/etc/default_config.py b/etc/default_config.py index 45272de..7a9da0f 100644 --- a/etc/default_config.py +++ b/etc/default_config.py @@ -11,6 +11,4 @@ UPLOAD_FOLDER = "/tmp/qc_app_files" REDIS_URL = "redis://" JOBS_TTL_SECONDS = 1209600 # 14 days GNQC_REDIS_PREFIX="GNQC" -GN3_URL="http://localhost:8080" -SQL_URI = os.getenv( - "SQL_URI", "mysql://gn2:mysql_password@localhost/db_webqtl_s") +SQL_URI = "" -- cgit v1.2.3