# Replace Redis with SQL

## Tags

* status: open
* priority: low
* assigned: fredm
* type: feature, feature-request, feature request
* keywords: gn-uploader, uploader, redis, sqlite, sqlite3

## Description

We currently (as of 2024-06-27) use Redis for tracking any asynchronous jobs (e.g. QC on uploaded files).

A lot of what we use redis for, we can do in one of the many SQL databases (we'll probably use SQLite3 anyway), which are more standardised, and easier to migrate data from and to. It has the added advantage that we can open multiple connections to the database, enabling the different processes to update the status and metadata of the same job consistently.

Changes done here can then be migrated to the other systems, i.e. GN2, GN3, and gn-auth, as necessary.

### 2025-12-31: Progress Update

Initial basic implementation can be found in:

=> https://git.genenetwork.org/gn-libs/tree/gn_libs/jobs
=> https://git.genenetwork.org/gn-uploader/commit/?id=774a0af9db439f50421a47249c57e5a0a6932301
=> https://git.genenetwork.org/gn-uploader/commit/?id=589ab74731aed62b1e1b3901d25a95fc73614f57

and others.

More work needs to be done to clean-up some minor annoyances.