Add a design-doc for better logging

1 files changed, 38 insertions, 0 deletions

diff --git a/topics/better-logging.gmi b/topics/better-logging.gmi
new file mode 100644
index 0000000..4c216aa
--- /dev/null
+++ b/topics/better-logging.gmi
@@ -0,0 +1,38 @@
+# Improving Logging in GN2
+
+## What Are We Trying To Solve?
+
+We prioritise maintaining user functionality over speed in GN [with time this speed will be improved].  As such we should be pay more attention at not breaking any currently working GN2 functionality.  And when/if we do, trouble-shooting should be easy.  On this front, one way is to stream-line logging in both GN2/GN3 and make it more script friendly - only report when something fails, not to instrument variables - and in so doing make the process of monitoring easier.
+
+## Goals
+
+- Remove noise from GN2.
+
+- Separate logging into different files: error logs, info logs.  Add this somewhere with Flask itself instead of re-directing STDOUT to a file.
+
+### Non-goals
+
+- Logging in GN3.
+
+- Parsing logs to extract goals.
+
+- Getting rid of "gn.db" global object and in so doing removing "MySqlAlchemy" [that we really shouldn't be using].
+
+- Adding log messages to existing functions.
+
+## Actual Design
+
+- Configure logger to separate logs into different files:
+  - INFO file - contains initial bootstrap messages and instrumentation we may have to do.
+  - ERROR file - contains real errors.
+
+- Have those settings - the different log files - be part of GN2 start-up script.
+
+- For error messages, use the following format for different error messages:
+  - "DATABASE: <message>"
+  - "REDIS: <message>"
+  - "COMPUTATION: <message>"
+  - "API: <message>"
+  - "MISC: <message>"
+
+- Have time-stamped logs by month. E.g. "genenetwork2-08-2022.error.log" and "genenetwork2-08-2022.info.log".  This way in future, we can actually run an analysis on what breaks often in GN2.