blob: ea9fd264fb879a26dc536b042064745d5d253753 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
|
# Improving Logging in GN2
## What Are We Trying To Solve?
We prioritise maintaining user functionality over speed in GN [with time this speed will be improved]. As such we should be pay more attention at not breaking any currently working GN2 functionality. And when/if we do, trouble-shooting should be easy. On this front, one way is to stream-line logging in both GN2/GN3 and make it more script friendly - only report when something fails, not to instrument variables - and in so doing make the process of monitoring easier.
## Goals
- Have script-friendly error/info logs.
- Remove noise from GN2.
- Separate logging into different files: error logs, info logs. Add this somewhere with Flask itself instead of re-directing STDOUT to a file.
### Non-goals
- Logging in GN3.
- Parsing logs to extract goals.
- Getting rid of "gn.db" global object and in so doing removing "MySqlAlchemy" [that we really shouldn't be using].
- Adding log messages to existing functions.
## Actual Design
- Configure logger to separate logs into different files:
- INFO file - contains initial bootstrap messages and instrumentation we may have to do.
- ERROR file - contains real errors.
- Have those settings - the different log files - be part of GN2 start-up script.
- For error messages, use the following format for different error messages:
- "DATABASE: <message>"
- "REDIS: <message>"
- "COMPUTATION: <message>"
- "API: <message>"
- "MISC: <message>"
- Have time-stamped logs by month. E.g. "genenetwork2-08-2022.error.log" and "genenetwork2-08-2022.info.log". This way in future, we can actually run an analysis on what breaks often in GN2.
- Get rid of "utility.logger" module and replace it with Flask's or Python's in-built logging.
|
