9 files changed, 277 insertions, 0 deletions

diff --git a/issues/gnqa/fetch-pubmed-references-to-gnqa.gmi b/issues/gnqa/fetch-pubmed-references-to-gnqa.gmi
new file mode 100644
index 0000000..63351d1
--- /dev/null
+++ b/issues/gnqa/fetch-pubmed-references-to-gnqa.gmi
@@ -0,0 +1,38 @@
+# Fetch Pubmed data to GNQA
+
+## Tags
+
+* assigned: alexm
+* keywords: llm, pubmed, api, references
+* type: enhancements
+* status: in progress
+
+## Description
+
+The task is to integrate PubMed references into the GNQA system by querying the PubMed API using GNQA publication titles. Then, display the data as reference information, which will provide users with more valuable details.
+
+
+## Tasks
+
+* [x] Connect to the PubMed API and perform HTTP requests.
+
+* [x] Query the API with the publication titles.
+
+* [] Display the PubMed information as reference information on the GN2 user interface.
+
+* [] dump the results to a DB e.g sqlite,lmdb
+
+* [x] If references are not found, perform a lossy search or list the closest three papers.
+
+* [] reimplement the reference ui to render the references as modal objects
+
+
+For lossy search, see:
+=> https://github.com/pjotrp/bioruby-bioinformatics-impact/blob/master/bin/pubmed2bib.rb
+
+see link to main  issue:
+=> https://issues.genenetwork.org/topics/lmms/llm-metadata
+
+
+
+
diff --git a/issues/gnqa/gn_llm_db_cache_integration.gmi b/issues/gnqa/gn_llm_db_cache_integration.gmi
new file mode 100644
index 0000000..86f7c80
--- /dev/null
+++ b/issues/gnqa/gn_llm_db_cache_integration.gmi
@@ -0,0 +1,35 @@
+# Implementing Efficient Database Caching for Query Responses in GN-LLM system
+
+
+## Tags:
+
+* assigned: alexm,shelby
+* keywords: llm,caching,database,query,response
+* type: enhancements
+
+* status: closed, done, completed
+
+
+## Description:
+
+This implementation task aims to enhance the performance and responsiveness of our GN-LLM (Large Language Model) system by incorporating a robust database caching mechanism. The focus will be on utilizing a database to store queries along with their corresponding answers and references, ensuring quicker retrieval and reduced computational load. Users can go back in time and see their search results at a given time.
+
+
+
+
+## Task
+
+* [x] implement endpoint for user caching 
+
+* [x] implement UI for QNQA search
+
+* [x] More customization features  like e.g user clearing their history
+
+See main issue:
+
+=> topics/lmms/llm-metadata
+
+## Note for commits See:
+
+=> https://github.com/genenetwork/genenetwork3/pull/165
+
diff --git a/issues/gnqa/gn_llm_integration_using_cached_searches.gmi b/issues/gnqa/gn_llm_integration_using_cached_searches.gmi
new file mode 100644
index 0000000..e20b5a3
--- /dev/null
+++ b/issues/gnqa/gn_llm_integration_using_cached_searches.gmi
@@ -0,0 +1,43 @@
+# GN2 Integration with LLM search using cached results
+
+## Tags
+
+* assigned: jnduli, alexm, bmunyoki
+* keywords: llm, genenetwork2
+* type: enhancement
+* status: open
+
+## Description
+
+We'd like to include LLM searches integrated into our GN searches, when someone attempts a Xapian search e.g. when I search for `wiki:rif group:mouse nicotine`, we'd do a corresponding search for `rif mouse nicotine` on LLMs, and show the results on the main page.
+
+Another example:
+
+xapian search: rif:glioma species:human group:gtex_v8
+llm search: glioma human gtex_v8
+
+
+This can be phased out into
+
+* [ ] 1. UI integration, where we modify the search page to include a dummy content box
+* [ ] 2. LLM search integration, where we perform a search and modify UI to show the results. This can either be async (i.e. the search results page waits for the LLM search results) or sync (i.e. we load the search results page after we've got the LLM results)
+* [x] 2.1 create a copy branch for the gnqa-api branch
+* [x] 2.2 create a PR containing all the branches
+* [ ] 2.3 how much would it take to get the qa_*** branch merged into main??
+* [ ] 3. Cache design and integration: we already have some
+
+cache using redis (gn search history), so we may use this for the moment.
+
+
+Let's use flag: `LLM_SEARCH_ENABLED` to enable/disable this feature during development to make sure we don't release this before it's ready.
+
+
+## Notes
+
+The branch for  merging to gn2:
+
+https://github.com/genenetwork/genenetwork2/pull/863
+
+The branch for merging to gn3:
+
+https://github.com/genenetwork/genenetwork3/pull/188
\ No newline at end of file
diff --git a/issues/gnqa/gnqa_integration_to_global_search_Design.gmi b/issues/gnqa/gnqa_integration_to_global_search_Design.gmi
new file mode 100644
index 0000000..94597de
--- /dev/null
+++ b/issues/gnqa/gnqa_integration_to_global_search_Design.gmi
@@ -0,0 +1,63 @@
+# GNQA Integration to Global Search Design Proposal
+
+## Tags
+* assigned: jnduli, alexm
+* keywords: llm, genenetwork2
+* type: feature
+* status: in-progress
+
+## Description
+This document outlines the design proposal for integrating GNQA into the Global Search feature.
+
+## High-Level Design
+
+### UI Design
+When the GN2 Global Search page loads:
+1. A request is initiated via HTMX to the GNQA search page with the search query.
+2. Based on the results, a page or subsection is rendered, displaying the query and the answer, and providing links to references.
+
+For more details on the UI design, refer to the pull request: 
+=> https://github.com/genenetwork/genenetwork2/pull/862
+
+### Backend Design
+The API handles requests to the Fahamu API and manages result caching. Once a request to the Fahamu API is successful, the results are cached using SQLite for future queries. Additionally, a separate API is provided to query cached results.
+
+## Deep Dive
+
+### Caching Implementation
+For caching, we will use SQLite3 since it is already implemented for search history. Based on our study, this approach will require minimal space:
+
+*Statistical Estimation:*  
+We calculated that this caching solution would require approximately 79MB annually for an estimated 20 users, each querying the system 5 times a day.
+
+Why average request size per user and how we determined this?
+The average request size was an upper bound calculation for documents returned from the Fahamu API.
+
+why we're assuming 20 users making 5 requests per day?
+
+We’re assuming 20 users making 5 requests per day to estimate typical usage of GN2 services
+### Error Handling
+* Handle cases where users are not logged in, as GNQA requires authentication.
+* Handle scenarios where there is no response from Fahamu.
+* Handle general errors.
+
+### Passing Questions to Fahamu
+We can choose to either pass the entire query from the user to Fahamu or parse the query to search for keywords.
+
+### Generating Possible Questions
+It is possible to generate potential questions based on the user's search and render those to Fahamu. Fahamu would then return possible related queries.
+
+## Related Issues
+=> https://issues.genenetwork.org/issues/gn_llm_integration_using_cached_searches
+
+## Tasks
+
+* [ ] Initiate a background task from HTMX to Fahamu once the search page loads.
+* [ ] Query Fahamu for data.
+* [ ] Cache results from Fahamu.
+* [ ] Render the UI page with the query and answer.
+* [ ] For "See more," render the entire GNQA page with the query, answer, references, and PubMed data.
+* [ ] Implement parsing for Xapian queries to normal queries.
+* [ ] Implement error handling.
+* [ ] reimplement how gnqa uses GN-AUTH in  gn3.
+* [ ] Query Fahamu to generate possible questions based on certain keywords.
diff --git a/issues/gnqa/implement-auth-to-gn-llm.gmi b/issues/gnqa/implement-auth-to-gn-llm.gmi
new file mode 100644
index 0000000..496a7cb
--- /dev/null
+++ b/issues/gnqa/implement-auth-to-gn-llm.gmi
@@ -0,0 +1,14 @@
+# Integrating gn-auth with gn-llm
+
+## Tags
+
+* assigned: alexm, Shelby
+* keywords: llm, auth
+* type: feature
+* priority: high
+* status: done, completed
+
+## Description
+
+The main objective is to integrate gn-authentication with gn-llm. This feature will ensure that only
+logged-in users can access the functionality, thereby preventing misuse by bots and unauthorized users.
\ No newline at end of file
diff --git a/issues/gnqa/implement-reference-rating-gn-llm.gmi b/issues/gnqa/implement-reference-rating-gn-llm.gmi
new file mode 100644
index 0000000..f646a6f
--- /dev/null
+++ b/issues/gnqa/implement-reference-rating-gn-llm.gmi
@@ -0,0 +1,26 @@
+# Implement Reference Rating for gn-llm
+
+## Tags
+
+* assigned: alexm,
+* keywords: llm, reference, rating
+* type: enhancements
+* status: closed, completed
+
+## Description
+
+For each reference in a query, we need to allow users to provide feedback or a rating on its relevance to a given task. This will help in improving future queries.
+
+## Tasks
+
+* [x] Create an API endpoint for reference rating.
+
+* [x] Implement the UI for this feature.
+
+
+For the UI and api imeplementation see commits:
+
+
+=> https://github.com/genenetwork/genenetwork2/pull/828/files
+
+=> https://github.com/genenetwork/genenetwork3/commit/482a8908dc08d6e5a13e576c4ba4bc3ff934bb8d
diff --git a/issues/gnqa/integrate_gn_llm_search.gmi b/issues/gnqa/integrate_gn_llm_search.gmi
new file mode 100644
index 0000000..5dfd9da
--- /dev/null
+++ b/issues/gnqa/integrate_gn_llm_search.gmi
@@ -0,0 +1,26 @@
+# GN-LLM Integration
+
+### Notes
+
+This Gist is intended to integrate GN-LLM (https://github.com/ShelbySolomonDarnell/GN-LLMs) into Genenetwork as a search.
+
+For reference on the initial implementation, please visit:
+
+=> https://github.com/ShelbySolomonDarnell/GN-LLMs
+
+## Tags
+
+* Assigned: alexm
+* Type: Feature
+* Priority: High
+* status: completed, done, closed.
+
+## Tasks
+
+* [x] Move API Endpoint + logic code from QNQA to GN3.
+* [x] Implement UI in GN2.
+* [x] Add GN authentication.
+
+## Notes
+
+The initial code was implemented as a full MVC application. The idea is to split the codebase into an API endpoint, which should be moved to the GN3 codebase, while the UI should reside in GN2.
\ No newline at end of file
diff --git a/issues/gnqa/merge-gnqa-to-production.gmi b/issues/gnqa/merge-gnqa-to-production.gmi
new file mode 100644
index 0000000..3d34bb1
--- /dev/null
+++ b/issues/gnqa/merge-gnqa-to-production.gmi
@@ -0,0 +1,16 @@
+# Merge GNQA to Production
+
+## Tags
+
+* assigned: alexm,
+* keywords: production, GNQA, integration
+
+## Description
+
+GNQA is currently in continuous development and needs to
+be pushed to production. We need to allow only logged-in users to access the services.
+
+## Tasks
+
+* [] Integrate GN-auth for the service
+* [] Push production to the current commit
\ No newline at end of file
diff --git a/issues/gnqa/refactor-gn-llm-code.gmi b/issues/gnqa/refactor-gn-llm-code.gmi
new file mode 100644
index 0000000..6e33737
--- /dev/null
+++ b/issues/gnqa/refactor-gn-llm-code.gmi
@@ -0,0 +1,16 @@
+# Refactoring gn-llm code and adding test modules
+
+## Tags:
+
+* assigned:alexm,shelby
+* keywords:refactoring,llm,tests
+* type: enchancements
+* status: in progress
+
+
+## Description
+
+The task is to Refactor `gn-llm` code for improved structure, readability, and maintainability. Enhance overall code quality by implementing best practices, design patterns, and incorporate thorough testing, including tests for C1 and CD components.
+
+
+