blob: d80bb0d50a79f2e96b4292cf3c16bb040dcc6343 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
|
# Improving Alerting/Logging in GN2
## Problem Statement
Current logging in the genenetwork ecosystem is noisy and difficult to parse programatically which makes it hard to:
* Integrate logs into some observability pipeline (E.g. sheepdog).
* Troubleshoot issues as they occur. We always learn of bugs from users.
## Goals
* Standardize logging format and config across GN2 flask apps and gn-guile.
* Adopt structured logging.
* Extend sheep-dog to be able to parse gn logs and send alerts on e-mail or matrix.
### Non-goals
* Logging in GN3.
* Parsing logs to extract goals.
* Getting rid of "gn.db" global object and in so doing removing "MySqlAlchemy" [that we really shouldn't be using].
* Adding log messages to existing functions.
## Actual Design
* Get rid of "utility.logger" module and replace it with Flask's or Python's in-built logging.
* Configure the logging system to automatically add the module name, line number, time-stamps etc.
* Use a better format for displaying the logs.
* Delete "with Bench..." calls.
## Resources
=> https://realpython.com/python-logging/ Logging in Python
=> https://signoz.io/guides/python-logging-best-practices/ Python Logging Best Practices - Obvious and Not-So-Obvious
=> https://signoz.io/blog/what-is-opentelemetry/ What is OpenTelemetry
|
