path: root/topics/biohackathon/GNGSoC2023.gmi

# GNSoC 2023

GN Summer of Code

## Introduction

We are running a GN Summer of Code in small teams.

* Runs July + August 2023
* Weekly plenary where projects present progress - Thursday 9am EU, 10 am EAT.
* Projects should be (slightly) out of comfort zone
* Use gemtext documentation
* Option for publishing on progress by then end as a BLOG or BioHackrXiv

## CI for guix-bioinformatics (guix pull)

* lead: Arun
* team: Efraim, Pjotr, Sarthak

Making GN deployment rock solid

git repo genenetwork-machines, guix-bioinformatics

=> ../../issues/gnsoc-ci-rethink Tracking progress

### Week 1

* Proposal written
  - guix pull on guix-bioinformatics
  - updated guix is broken gemma (Pjotr)
* Efraim guix GN2 - so we can have a channel
* Next step build substitutes for guix-bioinformatics
  - once built they are shared
* And create a GN3 channel (Efraim?)

### Week 2

* RISC-V port progressing with node and zig 0.10
* guix-bioinformatics now has CI!
* ~700 packages, 240 are broken ;)

### Week 3

* Arun gives a presentation on laminar using guix-forge: slides:
=> https://issues.genenetwork.org/topics/ci-rethink-slides
* New system is simpler and has reproducibility issues
* Efraim is doing channels for GN2 and GN3
* Sarthak will try to run guix-forge

### Week 4

* Arun added channels to CI
* localhost
* cgit
* Efraim: gn2 -> channel; Arun tested

### Week 5

* Added Klaus server for git
* Fixed channels to work wit Python 3.9 instead of 3.10

### Week 6

* Towards a new workbench with cgit support

### Week 7

* Arun has cgit running on his own server - soon bringing up tux02 after resolving https

### Week 8

* cgit deployment at https://git.genenetwork.org/
* guix forge

### Week 9

* Discussion on importers for guix forge
* Talked about propagator networks - part of Seattle presentation

### More

=> https://ci.genenetwork.org/jobs/guix-bioinformatics

* CI/CD is up and running again (and broken)
* Rethink: channels and pull channels are used for CI/CD
* Move unused packages elsewhere

## Nextgen databases

lmdb+RDF

* lead: Bonface
* team: Fred, Alex
* contact: Pjotr

git repo genenetwork3

=> ../../topics/next-gen-databases/design-doc Design doc

### Week 1

* RDF dumps
* Parsing S-exp -> markdown
* Hashing tables (Fred)
  - automated updates
* Some progress on sample data from SQL -> lmdb (Alex)
* Next week: guile bindings for lmdb
  - improving RDF

### Week 2

* RDF structure to markdown dump
* Fred is running SPARQL queries
* Alex is adding lmdb phenotype API endpoints

### Week 3

* Bonface demoes new documentation & code

### Week 4

* Settled on prefixes terms and id
* Updated man pages
=> https://github.com/genenetwork/gn-docs/tree/master/rdf-documentation
* Started work on lmdb+guile:
=> https://github.com/BonfaceKilz/gn-data-vault

### Week 5

* Metadata - renamed prefixes
* Short names gn: gnt:
* Updated virtuoso
* parsing geno files - lmdb support

### Week 6

* Improvements on RDF
### Week 7

* RDF improvements with ontology
* Inconsistencies and privacy discussion today

### Week 8

* Transformed most tables now
* guile-lmdb started by Alex

### Week 9

* New renaming and modelling
* GeneRif
* Working on unique IDs
* SKOS

## LLMs & metadata (RDF)

* lead: Shelby
* team: Priscilla
* contact: Bonface, Pjotr, Rupert

=> ../lmms/llm-metadata Tracking progress

### Week 1

* Created issue page
* Downloading publications (Priscilla)
* Flask server
* Next: Connecting OpenAI
* Create matrix room

### Week 2

* Open AI API is working
* Shelby is integrating into a Flask interface for GeneNetwork
* Using a pubmed UI style

### Week 3

* Shelby shows code
* Plan to host
* Priscilla is working on SLA and document acquisition
* Hosting GN Q&A

### Week 4

* working on container
* fetched 1000 publication
* JSON documentation on references

### Week 5

* Guix container for LLM
* Expose container to Rupert
* Add a password

### Week 6

* Very close to a working flask app Rupert can try next week
### Week 7

* Working FLASK app ready for testing - Rupert will have a go

### Week 8

* Working prototype!
* references as JSON

### Week 9

* Shelby showed working references and challenges
* System is ready for viewing by Rob

## API to access data from GN

* lead: Rupert
* team: Flavia
* contacts: Bonface, Zach, Fred

Documentation and adding endpoints

git repo gn-docs & genenetwork3 & SPARQL

=> https://github.com/genenetwork/gn-docs

### Week 1

* Mapping out the API

=> https://github.com/genenetwork/gn-docs/blob/master/api/questions-to-ask-GN.md

* Ideas on structuring
* Questions on GN
* Next: unify access to information
* collecting questions from users
* settle on form of API
* create example URLs for mouse

### Week 2

* GraphQL Arun gives a mumi demo - schema allows for (partial) queries and querying the schema itself
* Pjotr convenience API demo - add endpoints in results
* Flavia added questions in gn-doc - e.g. for synteny search
=> https://issues.genenetwork.org/topics/xapian-search-queries Examples for synteny

### Week 3

* Rupert proposes endpoints and metadata traversing

=> https://github.com/genenetwork/gn-docs/blob/master/api/alternative-API-structure.md

### Week 4

* Start working on endpoints
* R api - reference GN/API
* synteny

### Week 5

* Added back-end support for wikidata - finding inconsistencies

### Week 6

* API ready for running in a production environment
* Using latest RDF

### Week 7

* Test version of API is running at https://luna.genenetwork.org/api/v2.0/
* We'll continue building up facilities

### Week 8

* R script to parse API by Rupert
* Move forward with populations/strains and datasets

### Week 9

* Progressed API software to include groups

## Editing data

* lead: Fred
* team: Arthur, Rupert, Zach
* contacts: Rob

### Week 1

* Edit phenotype metadata works
* Next: phenotype values and testing on live

### Week 2

* Fixing issues
* Meeting on requirements from Arthur and Zach

### Week 4

* Editing works!
* Discussed the approval procedure and edit button for everyone

### Week 5

* Editing has gone on production - fixing issues
* Discussion on REST API

### Week 6

* Progress on authorization and editing

### Week 7

* Fred has moved code into a new repo for https://github.com/genenetwork/gn-auth

### Week 8

* gn-auth is building locally and needs to go on the forge
* case attributes
### Week 9

* gn-auth continuation
* case-attribute editing progress and challenges

## Guix parametrization

* lead: Sarthak
* team: Pjotr, Gabor
* contacts: Ludo

### Week 1

=> https://blog.lispy.tech/parameterized-packages-an-update.html

* Next: focus on statically built packages optimized for arch.

### Week 2

* Looking into GeneNetwork3 service
* Enumerated types

### Week 3

* Preparing for BLOG on S-exp

### Week 4

* BLOG in a stage that we discuss naming conventions

### Week 5

* Proposed DSL for parameters

### Week 6

* Posted BLOG and started implementation

### Week 7

* Agreed on final deliverables for GSoC

### Week 8

* Milestone on dependencies!

### Week 9

* Sarthak showed us the working prototype

## Links

* Matrix room is GNSoC2023

=> https://fosdem.org/2023/schedule/event/tissue/ Arun's talk on our issue tracker
=> https://github.com/genenetwork/gn-gemtext-threads Git repo on issues/tasks/topics
=> https://issues.genenetwork.org/topics/biohackathon/GNGSoC2023 This page

For more info contact pjotr.public912 at thebird.nl