about summary refs log tree commit diff
diff options
context:
space:
mode:
authorJohannes Medagbe <johanmedagbe@gmail.com>2026-09-21 10:03:20 +0000
committerJohannes Medagbe <johanmedagbe@gmail.com>2026-09-22 12:18:32 +0000
commitdce53892085c02c8571d8a6495e39c449294aa5f (patch)
treeae64d32cc845d30ec2a4cc3514a23dc5683a09a1
downloadgenenetwork_ai_search-master.tar.gz
New: commit aisearch repo master
-rw-r--r--LICENSE28
-rw-r--r--README.md73
-rw-r--r--aisearch/data/full_benchmark.csv116
-rw-r--r--aisearch/data/small_benchmark.csv40
-rw-r--r--aisearch/pyproject.toml50
-rw-r--r--aisearch/results/barplot_all_metrics.pngbin0 -> 1216060 bytes
-rw-r--r--aisearch/results/barplot_extra_metrics.pngbin0 -> 873570 bytes
-rw-r--r--aisearch/results/barplot_model_local_new_metric.pngbin0 -> 366052 bytes
-rw-r--r--aisearch/results/barplot_model_new_metric.pngbin0 -> 463676 bytes
-rw-r--r--aisearch/results/barplot_traditional_metrics.pngbin0 -> 1057060 bytes
-rw-r--r--aisearch/results/boxplot_system_local_new_metric.pngbin0 -> 313753 bytes
-rw-r--r--aisearch/results/boxplot_system_new_metric.pngbin0 -> 596542 bytes
-rw-r--r--aisearch/results/dspy/gptoss_20b_dspy_results.csv4
-rw-r--r--aisearch/results/dspy/haiku_4.5_dspy_results.csv4
-rw-r--r--aisearch/results/dspy/haiku_4.5_opus_4.8_dspy_results.csv4
-rw-r--r--aisearch/results/dspy/opus_4.8_dspy_results.csv4
-rw-r--r--aisearch/results/dspy/qwen3_30b_dspy_results.csv4
-rw-r--r--aisearch/results/dspy/qwen3_30b_opus_4.8_dspy_results.csv4
-rw-r--r--aisearch/results/heatmap_model_new_metric.pngbin0 -> 426037 bytes
-rw-r--r--aisearch/results/local/gptoss_20b_dspy_results.csv4
-rw-r--r--aisearch/results/local/qwen3_30b_dspy_results.csv4
-rw-r--r--aisearch/results/simple/gptoss_20b_simple_results.csv6
-rw-r--r--aisearch/results/simple/qwen3_30b_simple_results.csv6
-rw-r--r--aisearch/scripts/agent_search.py81
-rw-r--r--aisearch/scripts/barplot_metrics.py86
-rw-r--r--aisearch/scripts/dspy_evaluate.py126
-rwxr-xr-xaisearch/scripts/fetch_metadata.py501
-rw-r--r--aisearch/scripts/grag_search.py76
-rw-r--r--aisearch/scripts/new_rdf2partial_text.py34
-rw-r--r--aisearch/scripts/plot_function.py137
-rw-r--r--aisearch/scripts/rag_search.py111
-rw-r--r--aisearch/scripts/ragent_search.py170
-rw-r--r--aisearch/scripts/rdf2partial_text.py70
-rw-r--r--aisearch/scripts/simple_evaluate.py132
-rw-r--r--aisearch/src/gnais/__init__.py0
-rw-r--r--aisearch/src/gnais/config.py120
-rw-r--r--aisearch/src/gnais/evaluation/__init__.py0
-rw-r--r--aisearch/src/gnais/evaluation/utils.py159
-rw-r--r--aisearch/src/gnais/search/__init__.py5
-rw-r--r--aisearch/src/gnais/search/agent.py88
-rw-r--r--aisearch/src/gnais/search/classification.py56
-rw-r--r--aisearch/src/gnais/search/corpus.py119
-rw-r--r--aisearch/src/gnais/search/grag.py160
-rw-r--r--aisearch/src/gnais/search/prompts.py159
-rw-r--r--aisearch/src/gnais/search/rag.py93
-rw-r--r--aisearch/src/gnais/search/ragent.py216
-rw-r--r--aisearch/src/gnais/search/tools.py717
-rw-r--r--aisearch/src/web/app.py348
-rw-r--r--aisearch/src/web/auth.py167
-rw-r--r--aisearch/src/web/static/css/style.css962
-rw-r--r--aisearch/src/web/templates/index.html233
-rw-r--r--aisearch/src/web/templates/login.html187
-rw-r--r--aisearch/src/web/templates/partials/response.html12
-rw-r--r--aisearch/src/web/templates/partials/response_body.html55
-rw-r--r--aisearch/src/web/templates/partials/stream_shell.html95
55 files changed, 5826 insertions, 0 deletions
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..f1882db
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, GeneNetwork
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..44b4fbc
--- /dev/null
+++ b/README.md
@@ -0,0 +1,73 @@
+<div align="center">
+  <h1>GNAIS</h1>
+</div>
+
+## What is GNAIS?
+
+**GNAIS** (GeneNetwork AI Search) is a search tool that helps digest metadata around GeneNetwork using language models. It enriches context of LLMs using GeneNetwork RDF data and metadata.
+
+**GNAIS** can perform:
+- document RAG search
+- graph RAG search
+- agentic search
+- hybrid search
+
+**GNAIS** was implemented using [DSPy](https://dspy.ai/).  Switching between LLM providers for the text generation model is as easy as changing a variable declaration.
+
+## Demos
+
+Here are a few examples of response generated with GNAIS with claude-haiku-4-5-20251001.
+
+1. Digest and explain phenotype datasets around cognition measured in BXD mice*
+
+```
+The BXD mouse phenotype datasets contain several cognition-related measurements.  The primary focus is on working memory deficits, with three related traits measuring age at onset (AAO) of working memory impairment in non-transgenic BXD littermates from the AD-BXD population: a combined sex-averaged trait (mean 11.12 months, LOD 1.89), a male-specific trait (mean 10.56 months, LOD 4.57), and a female-specific trait (mean 11.70 months, LOD 2.06).  These traits are part of a larger study examining cognitive aging in transgenic and non-transgenic AD-BXD mice.  Additionally, the dataset includes a motor impulsivity trait measured via the 5-choice serial reaction time task (5CSRT) in males, which assesses premature responding behavior (mean 9.35, LOD 3.60).  All cognition traits were submitted by Sarah Neuner and Catherine Kaczorowski, and the working memory traits are referenced in unpublished studies, while the impulsivity trait is published in PubMed:24703509."
+```
+
+2. Compare traits related to the central nervous system measured in BXD, BDL, CCB and MDP mice
+
+```
+Central nervous system traits across the four mouse datasets show distinct research focuses: BXD mice (10 traits) include stress-induced changes (CMS treatment effects on organ weights and sacrifice age), morphological measurements (brain region weights), and locomotor behavior in novel environments.  BDL mice (1 trait) measure aging-related fear conditioning and baseline anxiety responses.  CCB mice (8 traits) emphasize behavioral and cognitive phenotypes including fear learning/extinction, anxiety-like behavior, and social interaction measures (dominance, aggression, sniffing).  MDP mice (1 trait) focus on pharmacological responses, specifically ethanol sensitization.  The BXD dataset provides the most comprehensive CNS phenotyping with 10 traits covering pharmacology, morphology, and behavior.  CCB traits are particularly rich in behavioral/cognitive measures, while BDL and MDP datasets each contribute specialized phenotypes for aging and drug response respectively.  All traits are measured in adult or aging mice, with BXD and CCB using controlled behavioral testing paradigms and BXD additionally measuring stress-induced physiological changes.
+```
+
+## Installing GNAIS
+
+**GNAIS** was packaged.  You can install it in your virtual environment using the following commands:
+
+```python
+git clone https://github.com/genenetwork/gn-ai.git
+cd aisearch
+python -m venv .venv
+source .venv/bin/activate
+pip install -e .
+```
+
+## Using GNAIS
+
+To run **GNAIS**, you need to define a few parameters in your bash environment. We recommend setting them in a file.  Check `aisearch/.env.example`. Once defined, you can run your search query using scripts in `aisearch/scripts`
+
+```python
+python aisearch/scripts/grag_search.py <your-query>
+```
+
+
+## Running the Web App
+
+Install all the python dependencies:
+
+```
+pip install -e .
+```
+
+Copy over the local environment configuration file:
+
+```
+cp .env.example env  # Update all the variables
+```
+
+Start a flask development server:
+
+```
+cd aisearch
+hypercorn -w 1 -b 0.0.0.0:4000 web.app:app
+```
diff --git a/aisearch/data/full_benchmark.csv b/aisearch/data/full_benchmark.csv
new file mode 100644
index 0000000..3ce0ed5
--- /dev/null
+++ b/aisearch/data/full_benchmark.csv
@@ -0,0 +1,116 @@
+"query","answer","link for verification"
+"How many distinct taxonomic families are defined in GeneNetwork?","3: Vertebrates, Other Metazoa, Plants","http://rdf.genenetwork.org/v1/category/taxonomic_family"
+"Which vertebrates are available in GeneNetwork?","Glossophaga soricina, Rattus norvegicus, Mus musculus, Homo sapiens","http://rdf.genenetwork.org/v1/id/family_Vertebrates"
+"Do you have data on any metazoa?","Yes, this includes Oryzias latipes, Drosophila melanogaster, Caenorhabditis elegans","http://rdf.genenetwork.org/v1/id/family_Other_Metazoa"
+"Which reference genome assembly is used for Drosophila melanogaster in GeneNetwork, and what is its short name?","dm6","http://rdf.genenetwork.org/v1/id/Drosophila_melanogaster"
+"What is Oryzias latipes?","fish also called medaka","http://rdf.genenetwork.org/v1/id/Oryzias_latipes"
+"What is the model organism for human disease in GeneNetwork?","Mus musculus (mouse)","http://rdf.genenetwork.org/v1/id/Mus_musculus"
+"What are the reference populations in GeneNetwork?","BXD, BXH, LXS, MDP, CC, AKXD, AXBXA, CXB, and HXBBXH","http://rdf.genenetwork.org/v1/id/population_Reference_Populations__replicate_average__SE__N_"
+"What are the inbred sets in the HRDP-HXB/BXH population?","HRDP_HXB_BXH_BP and HRDP_HXB_BXH_Metb","http://rdf.genenetwork.org/v1/id/population_HRDP_HXB_BXH_Individual_Data"
+"Which populations are considered non-genetic (lack genetic panel)?","B6_Lens, B6D2, CIE_INIA, CIE_RMA, JAX_D2_Mono_RNA_Seq, C57BL_6JxC57BL_6NJF2, NZBXFVB_N2, Retina_RGC_Rheaume, EMSR, Scripps_2013, and D2GM","http://rdf.genenetwork.org/v1/id/population_Non_genetic_Cohort"
+"What are the sets in the Heterogeneous Stock population?","HSNIH_Palmer, HSNIH_RGSMC, NWU_WKYxF344_F2, SRxSHRSPF2, and HSNIH_Palmer_r4","http://rdf.genenetwork.org/v1/id/population_Crosses_and_Heterogeneous_Stock__individuals_"
+"Are genotypes available for the HIV-1Tg set?","No","http://rdf.genenetwork.org/v1/id/population_Groups_Without_Genotypes"
+"How many population categories are available in GeneNetwork?","8","http://rdf.genenetwork.org/v1/category/population_category"
+"What is the genetic type of the BXD family?","riset or recombinant lines","http://rdf.genenetwork.org/v1/id/set_BXD"
+"What does the genetic set DOL mean in full in GeneNetwork?","Inbredset described as 'DO Boon Lung Infection'","http://rdf.genenetwork.org/v1/id/set_DOL"
+"From which parental strains was the BXH family derived?","C57BL/6J and C3H/HeJ","http://rdf.genenetwork.org/v1/id/set_BXH"
+"Which population category does the B6D2F2 set belong to?","Crosses, AIL, HS","http://rdf.genenetwork.org/v1/id/set_B6D2F2"
+"Which BXD inbredset were used to study Alzheimer's disease?","BXD_JAX_AD and BXD_NIA_AD","http://rdf.genenetwork.org/v1/id/set_BXD_JAX_AD, http://rdf.genenetwork.org/v1/id/set_BXD_NIA_AD"
+"Which BXD set served as model in Gulf War Illness?","DOD_BXD_GWI","http://rdf.genenetwork.org/v1/id/set_DOD_BXD_GWI"
+"Which mouse set comes from the cross B6MRLF2 x D2MRLF2?","B6MRLF2_D2MRLF2","http://rdf.genenetwork.org/v1/id/set_B6MRLF2_D2MRLF2"
+"Which non-genetic cohort focused on lens biology?","B6_Lens","http://rdf.genenetwork.org/v1/id/set_B6_Lens"
+"Which mouse set belongs to the Reference Populations category and is a recombinant inbred panel derived from BALB/cByJ and C57BL/6ByJ?","CXB","http://rdf.genenetwork.org/v1/id/set_CXB"
+"Is the HSNIH-Palmer rat an outbred population?","Yes","http://rdf.genenetwork.org/v1/id/set_HSNIH_Palmer"
+"The Hybrid Rat Diversity Panel (HRDP) HXB/BXH Brain Proteome set belongs to which species?","Rattus norvegicus","http://rdf.genenetwork.org/v1/id/set_HRDP_HXB_BXH_BP"
+"Which set is coded as RHV in GeneNetwork?","The set code of the HIV-1Tg rat set","http://rdf.genenetwork.org/v1/id/set_HIV_1Tg"
+"Which human set is dedicated to brain connectivity and cognition research?","HCP","http://rdf.genenetwork.org/v1/id/set_HCP"
+"What is the UCI set focusing on brain aging?","Aging_Brain_UCI","http://rdf.genenetwork.org/v1/id/set_Aging_Brain_UCI"
+"Is there any set that was used to study pancreatic islet biology?","Yes, Islets_Gerling","http://rdf.genenetwork.org/v1/id/set_Islets_Gerling"
+"Give me examples of datasets used to dissect retinal transcriptomics in GeneNetwork","TIGEM_Retina_RNA_Seq and Retina_RGC_Rheaume","http://rdf.genenetwork.org/v1/id/set_TIGEM_Retina_RNA_Seq, http://rdf.genenetwork.org/v1/id/set_Retina_RGC_Rheaume"
+"Which human dataset in GeneNetwork focused on childhood neurodegenerative disease?","CANDLE","http://rdf.genenetwork.org/v1/id/set_CANDLE"
+"Do we have data on the CEPH family cell lines in GeneNetwork?","Yes, CEPH_2004 and CEPH_2009","http://rdf.genenetwork.org/v1/id/set_CEPH_2004, http://rdf.genenetwork.org/v1/id/set_CEPH_2009"
+"What are MAGIC lines?","A inbredset of Caenorhabditis elegans generated using Multiparent Advanced Generation InterCross (MAGIC)","http://rdf.genenetwork.org/v1/id/set_MAGIC_Lines"
+"Which set of medaka fish do we have data for in GeneNetwork?","MIKK","http://rdf.genenetwork.org/v1/id/set_MIKK"
+"Which set belongs to Arabidopsis thaliana?","BayXSha","http://rdf.genenetwork.org/v1/id/set_BayXSha"
+"What are the mapping tools in use in GeneNetwork?","HappyR, Rqtl, qtlreaper, FastMap, PLINK","https://rdf.genenetwork.org/v1/category/mapping_method"
+"Which mapping tool is preferred for heterogeneous stock and outbred populations?","Rqtl","http://rdf.genenetwork.org/v1/id/mapping_method_Rqtl"
+"What is the identifier GN602 for in GeneNetwork?","BXD phenotype dataset","http://rdf.genenetwork.org/v1/id/dataset_BXDPublish"
+"What is the highest LOD score, locus marker, and additive effect for serotonin in BXD male midbrain?","LOD score: 2.6063138915106485172, Locus: rsm10000012461, Additive: -1.3041285214285716076","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the PubMed publication associated with measurement of serotonin in the BXD male midbrain?","24102824","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the mean value of serotonin in BXD male midbrain?","12.037411717807545486","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the description of the trait OTU_310_CD_norm_log10?","Metagenome, microbiota, gastrointestinal system, metabolism: OTU310 Firmicutes;Clostridiales;Lachnospiraceae proportion assessed by 16S rRNA sequencing of cecum content from young adult males 3-6 months on a standard chow diet CD, EPFL LISP3 Cohort, 2014-2018 [normalised log fraction]","http://rdf.genenetwork.org/v1/id/phenotype_OTU_310_CD_norm_log10"
+"Which set does the trait OTU_310_CD_norm_log10 belong to?","BXD","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_OTU_310_CD_norm_log10"
+"Which human datasets do not have genotype data (i.e., lack gnt:has_genotype_data)?","GTEx, GTEx_v5, GTEx_v8, HCP, EBV_T_Cells_PERKINS, HB, HLC, HLT, HSB, TIGEM_Retina_RNA_Seq, Islets_Gerling, Brain_Normal_NIH_Gibbs, Aging_Brain_UCI, AD_cases_controls, AD_cases_controls_Myers, CANDLE, CEPH_2004, CEPH_2009","http://rdf.genenetwork.org/v1/id/Homo_sapiens"
+"Which mouse sets are from an intercross (genetic type)?","B6D2F2, BDF2_1999, BDF2_2005, B6BTBRF2, CTB6F2, B6MRLF2_D2MRLF2, B6D2F2_PSU, DOL, DO, CCBXD_TM, CC","http://rdf.genenetwork.org/v1/id/population_Crosses__AIL__HS"
+"Which sets from the reference populations are recombinant inbred lines (genetic type is riset)?","BXD, BXH, AKXD, LXS, MDP, MAGIC_Lines","http://rdf.genenetwork.org/v1/id/population_Reference_Populations__replicate_average__SE__N_"
+"Which human dataset contains both log2-transformed and TPM-normalized gene expression across multiple tissues?","GTEx_v8","http://rdf.genenetwork.org/v1/id/set_GTEx_v8"
+"Which species does the set Oregon_R_x_2b3 belong to?","The fruit fly Drosophila melanogaster","http://rdf.genenetwork.org/v1/id/set_Oregon_R_x_2b3"
+"The LXS panel traces back to an 8-way cross initiated in the 1950s by Gerald McClearn. Which two parental inbred strains define the LXS panel's alcohol-related phenotypes?","Inbred Long-Sleep (ILS) and Inbred Short-Sleep (ISS)","http://rdf.genenetwork.org/v1/id/set_LXS"
+"The HIV-1Tg rat model expresses HIV-1 viral proteins and is used to study neuroHIV. Which brain regions have transcriptomic datasets associated with this set in GeneNetwork?","Hippocampus (Hip), Striatum (Str), and Prefrontal Cortex (PFC)","http://rdf.genenetwork.org/v1/id/set_HIV_1Tg"
+"The CANDLE set in GeneNetwork focuses on childhood neurodegenerative disease. Which species is the data collected in?","Homo sapiens","http://rdf.genenetwork.org/v1/id/set_CANDLE"
+"Which dataset in GeneNetwork focused on Epstein-Barr virus-transformed T-cells?","EBV_T_Cells_PERKINS","http://rdf.genenetwork.org/v1/id/set_EBV_T_Cells_PERKINS"
+"What is the largest human transcriptome resource in GeneNetwork?","GTEx","http://rdf.genenetwork.org/v1/id/set_GTEx"
+"Which trait is identified by BXDPublish_17985?","serotonin in BXD male midbrain","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the link to download trait measurements for BXDPublish_OTU_310_CD_norm_log10?","https://cd.genenetwork.org/api3/lmdb/v1/data/traits/BXDPublish_23391.json","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_OTU_310_CD_norm_log10"
+"Which GeneNetwork trait page URL is linked to the trait_BXDPublish_5HT_MB_M?","https://genenetwork.org/show_trait?trait_id=17985&dataset=BXDPublish","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"Which molecule does the trait BXDPublish_5HT_MB_M measure?","Serotonin (5-HT)","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"List eye-related RNA-seq datasets available for BXD?","dataset_UTHSC_BXD_Aged_Eye_RNAseq_TPM_Log2_1120, dataset_UTHSC_BXD_All_Ages_Eye_RNAseq_TPM_Log, dataset_UTHSC_BXDOldEyeRNAseq_DESeq2_rlog2_0422, dataset_UTHSC_BXDYgEyeRNAseq_DESeq2_rlog2_0422, dataset_UTHSC_BXD_Young_Adult_Eye_RNAseq_TPM_Log, dataset_UTHSC_IndBXD_A_Ages_Eye_RNA_S_TPM_Log, dataset_UTHSC_mm9_B6D2_Ret_0916, dataset_UTHSC_mm9_B6D2_RetEx_0916, dataset_JAX_D2_Mono1vPBMC_Ret_RNA_Seq_0619, dataset_JAX_D2_Mono2vPBMC_Ret_RNA_Seq_0619, dataset_DBA2J_ONH_1212, dataset_ONCRetILM6_0412, dataset_Eye_M2_0908_R, dataset_Eye_AXBXA_1008_RankInv, dataset_Eye_M2_0908_R_ND, dataset_Eye_M2_0908_R_WT, dataset_Eye_M2_0908_R_MT, dataset_Eye_M2_0908_R_NB, dataset_Eye_M2_0908_WTWT","http://rdf.genenetwork.org/v1/id/set_BXD"
+"Identify BXD datasets related to gut microbiome","dataset_UTHSC_GutExL_0414 and dataset_UTHSC_GutGL_0414","http://rdf.genenetwork.org/v1/id/set_BXD"
+"Which BXD datasets focuse on neutrophil transcriptomics?","dataset_UTHSC_Neut_EL_1014 and dataset_UTHSC_Neut_1014","http://rdf.genenetwork.org/v1/id/set_BXD"
+"Which datasets provide data on BXD germ cells?","dataset_JAX_BXD_Germ_Cells_edgeR_0820, dataset_JAX_BXD_Germ_Cells_edgeR_log2_0820, dataset_JAX_BXD_Germ_Cells_edgeR_log2z_0820","http://rdf.genenetwork.org/v1/id/set_BXD"
+"The BXD family includes datasets for colon transcriptomics under different diets. List them.","dataset_EPFL_LISP_BXD_Col_CD_HFD0917, dataset_EPFL_LISP_BXD_Col_CD0917, dataset_EPFL_LISP_BXD_Col_HFD0917","http://rdf.genenetwork.org/v1/id/set_BXD"
+"Extract brain RNA-seq datasets available for the LXS set.","UCLA_BHHBF2_BRAIN_MALE, UCLA_BHHBF2_BRAIN_FEMALE, UCLA_BHF2_BRAIN_MALE, UCLA_BHF2_BRAIN_FEMALE, UCLA_CTB6B6CTF2_BRAIN_2005, UCLA_BHF2_BRAIN_0605, UCLA_BHHBF2_BRAIN_2005, UCLA_CTB6B6CTF2_BRAIN_FEMALE, UCLA_CTB6B6CTF2_BRAIN_MALE, UCAMC_LXSBrSal_RNA_Seq_0216 and dataset_UCAMC_LXSBrEtOH_RNA_Seq_0216","http://rdf.genenetwork.org/v1/id/set_LXS"
+"What are the AKXD mammary tumor datasets from NCI?","dataset_NCI_Agil_Mam_Tum_RMA_0409 and dataset_NCI_Mam_Tum_RMA_0409","http://rdf.genenetwork.org/v1/id/set_AKXD"
+"What thyroid datasets do we have in GeneNetwork for GTEx_v8?","dataset_GTEx_Thyro_0414, dataset_GTEXv5_Thy_0915, dataset_GTEXv8_Thy_0220, dataset_GTEXv8_Thy_tpm_0220","http://rdf.genenetwork.org/v1/id/set_GTEx_v8"
+"The GeneNetwork graph contains species from three kingdoms of life. Which kingdoms are represented?","Animalia (Vertebrates and Other Metazoa) and Plantae (Plants)","http://rdf.genenetwork.org/v1/category/taxonomic_family"
+"Which mammalian species are used in GeneNetwork?","Homo sapiens, Mus musculus, Rattus norvegicus, Glossophaga soricina","http://rdf.genenetwork.org/v1/id/family_Vertebrates"
+"For which plant species do we have data in GeneNetwork?","Arabidopsis thaliana, Glycine max, Solanum lycopersicum, Hordeum vulgare, Populus trichocarpa","http://rdf.genenetwork.org/v1/id/family_Plants"
+"Which trait measures the age at onset of working memory deficit in Alzheimer's disease BXD mice, and what is its highest LOD score?","trait_BXDPublish_AAO_ADBXD, LOD: 2.4026782929885950324, Locus: rsm10000005323","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_AAO_ADBXD"
+"What is the full description of the AAO_ADBXD trait in BXD mice?","Central nervous system, behavior, learning and memory: Age at onset of working memory deficit defined as first age at which performance dropped to chance 50%, male and females average of AD-BXD cases positive for 5XFAD transgene [months]","http://rdf.genenetwork.org/v1/id/phenotype_AAO_ADBXD"
+"Which PubMed article is associated with the AAO_ADBXD Alzheimer's onset trait in BXD mice?","30595332","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_AAO_ADBXD"
+"The MPTP neurotoxin is used to model Parkinson's disease in BXD mice. What locus has the highest LOD score for serotonin response in male striatum after MPTP treatment?","LOD: 3.9673994445052551683, Locus: rsm10000002155","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_CPU_Sal_MPTP_M"
+"Which PubMed study is associated with MPTP-induced serotonin changes in BXD male striatum?","23558233","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_CPU_Sal_MPTP_M"
+"What is the phenotype description of the 5HT_CPU_Sal_MPTP_M trait measuring MPTP response in male striatum?","MPTP 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine response 12.5 mg/kg ip, serotonin 5-HT concentration in caudate-putamen striatum in males 48 h after injection saline-MPTP group [ug/mg total protein]","http://rdf.genenetwork.org/v1/id/phenotype_5HT_CPU_Sal_MPTP_M"
+"Which dopamine metabolite was measured in male BXD striatum 48 hours after MPTP treatment, and what is its highest association score?","3-methoxytyramine (3-MT), LOD: 2.0516721070584273967, Locus: rsm10000001111","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3MT_CPU_MPTP_M"
+"What is the highest LOD score for the difference in 3-MT levels between saline and MPTP-treated male BXD striatum?","LOD: 2.5855803408910196239, Locus: rsm10000002558","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3MT_CPU_Sal_MPTP_M"
+"The 3NP toxin induces striatal lesions modeling Huntington's disease. What is the highest mapping score for 3NP neurodegeneration susceptibility in BXD mice?","LOD: 6.1340526876326206818, Locus: rsm10000005366","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3NP_Tox"
+"What is 3NP_Tox?","It is a trait: 3-nitropropionic acid neurodegeneration in striatum caudate putamen 50 mg/kg ip in young adults, presence or absence of lesions [-1 = lesions, +1 = no lesions]","http://rdf.genenetwork.org/v1/id/phenotype_3NP_Tox"
+"Is there any trait measured in the olfactory bulb of BXD mice one hour after saline injection?","Yes, THDOC (3a,5a-pregnane-3a,21-diol-20-one)","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5a_THDOC_OB"
+"Which PubMed study is associated with THDOC levels in BXD olfactory bulb?","21494628","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5a_THDOC_OB"
+"What is the mean alpha-aminoadipate to lysine ratio in BXD mouse plasma after overnight fasting, and what is its highest association score?","0.0045021891883398230949","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_AAP_Lys_CD"
+"Which PubMed study is associated with the AAP_Lys_CD plasma metabolite trait in BXD mice?","25215496","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_AAP_Lys_CD"
+"What is the full description of Alcohol_intake_W4D4_all_Females in BXD mice?","Ethanol response, consumption of 20% v/v EtOH using the drinking in the dark DID method, 4 h access on day 4 week 4 normally housed 4-6 month old females 5 week baseline [g/kg]","http://rdf.genenetwork.org/v1/id/phenotype_20__Alcohol_intake_W4D4_all_Females"
+"Which neurotransmitters were measured in the MPTP Parkinson's disease model in BXD mice?","Serotonin (5-HT), 5-HIAA (serotonin metabolite), and 3-methoxytyramine (3-MT, dopamine metabolite) in caudate-putamen striatum","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_CPU_Sal_MPTP_M"
+"What is the difference in serotonin levels between male and female BXD diencephalon?","Male (5HT_DC_M): Mean 13.318529409520765583, Female (5HT_DC_F): Mean 14.784413830987338301","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_DC_M, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_DC_F"
+"What is the typical value of the amphetamine-induced locomotor response measured in BXD mice at 5 mg/kg, and what is its highest LOD score?","Mean: 7220.8214285714284415 cm, LOD: 3.4041048067742516459, Locus: D4Mit178","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_AMPHLOC5mg"
+"What is AMPHLOC5mg in BXD mice?","Amphetamine response 5 mg/kg ip, locomotor activity from 0-60 min after injection [cm]","http://rdf.genenetwork.org/v1/id/phenotype_AMPHLOC5mg"
+"Which liver enzyme was measured in 14-week old female BXD mice, and what is its highest LOD score?","Alanine transaminase (ALT/ALAT/SGPT), LOD: 2.1671810310648718101, Locus: rsm10000012410","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_ALAT_F"
+"What is the PubMed publication reporting liver enzyme measurements in BXD mice?","22939713","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_ALAT_F"
+"What is the highest mapping score for alkaline phosphatase (ALP) measured in female BXD mice?","LOD: 6.7360430828111121926, Locus: rs31971961","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_ALP"
+"Which phenotype measured liver injury in female BXD mice before carbon tetrachloride challenge?","ALT day 0 mean","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_ALT_d0__mean_F"
+"What is the highest association score for acetaminophen (APAP) hepatocyte viability response in male BXD liver?","LOD: 4.7056254566002762019, Locus: rsm10000003730","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_APAP_log2"
+"What is the full description of APAP_log2?","Acetaminophen APAP hepatocyte viability response 9 hr after treatment 0.5 mM in vehicle, lactate dehydrogenase LDH release assay adjusted for DNA content, young adult male liver [log2 LDH/DNA ratio treated/control]","http://rdf.genenetwork.org/v1/id/phenotype_APAP_log2"
+"What is the additive effect for acoustic startle response to 110 dB at 10 kHz in BXD mice?","-5.7583334166666668708","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_ASR110dB10kHz"
+"Which PubMed publication references the study on acoustic startle response in BXD mice?","10371755","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_ASR110dB10kHz"
+"What is the highest LOD score for raw striatum volume in BXD mice?","LOD: 2.8594045664833709353, Locus: rsm10000003425","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_StriatumVolRaw"
+"Which PubMed article reported on striatum volume in BXD mice?","19191878","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_StriatumVolRaw"
+"Which marker had the highest LOD score for sleep need measured as slow wave sleep delta EEG power after 6 hours deprivation in BXD mice, and what is its LOD score?","Locus: rsm10000010045, LOD: 3.6721742203463301202","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_SleepNeed"
+"Which T cell populations were estimated by TIMER 2.0 in breast cancer tumors from BXD x C3-1TAg F1 females, and what are the most interesting loci?","CD4 effector memory T cells (LOD: 1.4524872961762274048 Locus: rsm10000014666), CD8 T cells (LOD: 1.8031565920123300284, Locus: rs29338660)","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_CD4_effector_memory_XCELL, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_CD8_TIMER"
+"Which gamma-delta T cell population is estimated by CIBERSORT in breast cancer tumors from BXD x C3-1TAg F1 females?","T cell gamma delta","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_gamma_delta_CIBERSORT"
+"Which naive T cell populations are estimated by XCELL in breast cancer tumors from BXD x C3-1TAg F1 females?","CD4 naive and CD8 naive T cells","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_CD4_naive_XCELL"
+"What is the highest mapping score for CD8 naive T cells in breast cancer tumors from BXD x C3-1TAg F1 females?","LOD: 2.6767753665261841967, Locus: rsm10000001634","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_CD8_naive_XCELL"
+"Which breast cancer model was used to study T cell infiltration phenotypes in the BXD panel?","C3-1TAg SV40 T/t-antigen transgenic model (JAX Stock 013591: FVB-TgC3-1TAgcJeg/JegJ)","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_CD8_TIMER"
+"What is the highest mapping signal for Streptococcus genus proportion in BXD mouse cecum on chow diet?","LOD: 2.6777190182706016586, Locus: rsm10000014670","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_Streptococcus_genus_CD_norm_log10"
+"Which microbiome genera were identified in BXD mice and reported in the PubMed publication 31634382?","Streptococcus, Staphylococcus, Stenotrophomonas","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_Streptococcus_genus_CD_norm_log10, https://rdf.genenetwork.org/v1/id/trait_BXDPublish_Staphylococcus_genus_CD_norm_log10, https://rdf.genenetwork.org/v1/id/trait_BXDPublish_Stenotrophomonas_genus_CD_norm_log10"
+"Compare most interesting loci of triacylglycerol TAG56:7_2 level in BXD male liver on chow and high diet?","The highest LOD score is at locus rs27728896 with a value of 1.9755325276024489067 on chow diet and at locus rsm10000014270 with a value of 3.5849473664517539362 on high diet","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_TAG_56_7__2_CD, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_TAG_56_7__2_HFD"
+"How else is the lipid species TAG56:7_2 in BXD liver called?","TG16:0_18:1_22:6","http://rdf.genenetwork.org/v1/id/phenotype_TAG_56_7__2_CD"
+"Which EPFL cohort studied liver lipidomics and microbiome phenotypes in BXD mice?","EPFL LISP3 Cohort","http://rdf.genenetwork.org/v1/id/phenotype_TAG_56_7__2_CD http://rdf.genenetwork.org/v1/id/phenotype_OTU_310_CD_norm_log10"
+"What is the highest mapping signal for serum TSH binding inhibition after 2 and 3 immunizations with TSHR A-subunit adenovirus in BXD females?","The highest LOD score is at locus rs3672987 with value of 7.8087209638496242192 (2 weeks) and 11.292110851469207233 (3 weeks)","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_TBI_2x, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_TBI_3x_"
+"Which PubMed publication is associated with TSHR immunization in BXD mice?","25051451","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_TBI_3x_"
+"Which autoimmune thyroid disease model is represented by the TBI and T4 traits in BXD mice?","Graves' disease model induced by TSHR A-subunit adenovirus immunization","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_TBI_3x_, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T4__3x_"
+"What is the mean value of the OTU_310_CD_norm_log10 microbiome trait in BXD mice?","7.8246294298479632801","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_OTU_310_CD_norm_log10"
+"What is BXDPublish_23391?","Identifier of the trait OTU_310_CD_norm_log10 in GeneNetwork","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_OTU_310_CD_norm_log10"
+"What is the URL to the trait 3NP neurodegeneration in GeneNetwork?","https://genenetwork.org/show_trait?trait_id=15959&dataset=BXDPublish","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3NP_Tox"
+"What is the GeneNetwork page for the trait alcohol intake measured in females?","https://genenetwork.org/show_trait?trait_id=20013&dataset=BXDPublish","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_20__Alcohol_intake_W4D4_all_Females"
+"Which liver metabolites were measured after overnight fasting in males at 29 weeks of age on normal chow?","AAP_Lys_CD, Sphingomyelin_d18_2_liver_CD, and TAG_56_7__2_CD","http://rdf.genenetwork.org/v1/id/phenotype_AAP_Lys_CD https://rdf.genenetwork.org/v1/id/phenotype_Sphingomyelin_d18_2_liver_CD, https://rdf.genenetwork.org/v1/id/phenotype_TAG_56_7__2_CD"
+"What is the mean dopamine metabolite (3-MT) level in male BXD striatum after saline injection?","0.39399999976158139869","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3MT_CPU_Sal_M"
\ No newline at end of file
diff --git a/aisearch/data/small_benchmark.csv b/aisearch/data/small_benchmark.csv
new file mode 100644
index 0000000..a24f983
--- /dev/null
+++ b/aisearch/data/small_benchmark.csv
@@ -0,0 +1,40 @@
+"query","answer","link for verification"
+"Do you have data on any metazoa?","Yes, this includes Oryzias latipes, Drosophila melanogaster, Caenorhabditis elegans","http://rdf.genenetwork.org/v1/id/family_Other_Metazoa"
+"What are the reference populations in GeneNetwork?","BXD, BXH, LXS, MDP, CC, AKXD, AXBXA, CXB, and HXBBXH","http://rdf.genenetwork.org/v1/id/population_Reference_Populations__replicate_average__SE__N_"
+"What are the sets in the Heterogeneous Stock population?","HSNIH_Palmer, HSNIH_RGSMC, NWU_WKYxF344_F2, SRxSHRSPF2, and HSNIH_Palmer_r4","http://rdf.genenetwork.org/v1/id/population_Crosses_and_Heterogeneous_Stock__individuals_"
+"From which parental strains was the BXH family derived?","C57BL/6J and C3H/HeJ","http://rdf.genenetwork.org/v1/id/set_BXH"
+"Which BXD inbredset were used to study Alzheimer's disease?","BXD_JAX_AD and BXD_NIA_AD","http://rdf.genenetwork.org/v1/id/set_BXD_JAX_AD, http://rdf.genenetwork.org/v1/id/set_BXD_NIA_AD"
+"The Hybrid Rat Diversity Panel (HRDP) HXB/BXH Brain Proteome set belongs to which species?","Rattus norvegicus","http://rdf.genenetwork.org/v1/id/set_HRDP_HXB_BXH_BP"
+"Give me examples of datasets used to dissect retinal transcriptomics in GeneNetwork","TIGEM_Retina_RNA_Seq and Retina_RGC_Rheaume","http://rdf.genenetwork.org/v1/id/set_TIGEM_Retina_RNA_Seq, http://rdf.genenetwork.org/v1/id/set_Retina_RGC_Rheaume"
+"What are MAGIC lines?","A inbredset of Caenorhabditis elegans generated using Multiparent Advanced Generation InterCross (MAGIC)","http://rdf.genenetwork.org/v1/id/set_MAGIC_Lines"
+"What is the highest LOD score, locus marker, and additive effect for serotonin in BXD male midbrain?","LOD score: 2.6063138915106485172, Locus: rsm10000012461, Additive: -1.3041285214285716076","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the PubMed publication associated with measurement of serotonin in the BXD male midbrain?","24102824","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the mean value of serotonin in BXD male midbrain?","12.037411717807545486","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the description of the trait OTU_310_CD_norm_log10?","Metagenome, microbiota, gastrointestinal system, metabolism: OTU310 Firmicutes;Clostridiales;Lachnospiraceae proportion assessed by 16S rRNA sequencing of cecum content from young adult males 3-6 months on a standard chow diet CD, EPFL LISP3 Cohort, 2014-2018 [normalised log fraction]","http://rdf.genenetwork.org/v1/id/phenotype_OTU_310_CD_norm_log10"
+"Which species does the set Oregon_R_x_2b3 belong to?","The fruit fly Drosophila melanogaster","http://rdf.genenetwork.org/v1/id/set_Oregon_R_x_2b3"
+"The LXS panel traces back to an 8-way cross initiated in the 1950s by Gerald McClearn. Which two parental inbred strains define the LXS panel's alcohol-related phenotypes?","Inbred Long-Sleep (ILS) and Inbred Short-Sleep (ISS)","http://rdf.genenetwork.org/v1/id/set_LXS"
+"The HIV-1Tg rat model expresses HIV-1 viral proteins and is used to study neuroHIV. Which brain regions have transcriptomic datasets associated with this set in GeneNetwork?","Hippocampus (Hip), Striatum (Str), and Prefrontal Cortex (PFC)","http://rdf.genenetwork.org/v1/id/set_HIV_1Tg"
+"Which trait is identified by BXDPublish_17985?","serotonin in BXD male midbrain","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"What is the URL of the GeneNetwork trait page for trait_BXDPublish_5HT_MB_M?","https://genenetwork.org/show_trait?trait_id=17985&dataset=BXDPublish","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_MB_M"
+"The BXD family includes datasets for colon transcriptomics under different diets. List them.","dataset_EPFL_LISP_BXD_Col_CD_HFD0917, dataset_EPFL_LISP_BXD_Col_CD0917, dataset_EPFL_LISP_BXD_Col_HFD0917","http://rdf.genenetwork.org/v1/id/set_BXD"
+"Extract brain RNA-seq datasets available for the LXS set.","UCLA_BHHBF2_BRAIN_MALE, UCLA_BHHBF2_BRAIN_FEMALE, UCLA_BHF2_BRAIN_MALE, UCLA_BHF2_BRAIN_FEMALE, UCLA_CTB6B6CTF2_BRAIN_2005, UCLA_BHF2_BRAIN_0605, UCLA_BHHBF2_BRAIN_2005, UCLA_CTB6B6CTF2_BRAIN_FEMALE, UCLA_CTB6B6CTF2_BRAIN_MALE, UCAMC_LXSBrSal_RNA_Seq_0216 and dataset_UCAMC_LXSBrEtOH_RNA_Seq_0216","http://rdf.genenetwork.org/v1/id/set_LXS"
+"Which mammalian species are used in GeneNetwork?","Homo sapiens, Mus musculus, Rattus norvegicus, Glossophaga soricina","http://rdf.genenetwork.org/v1/id/family_Vertebrates"
+"For which plant species do we have data in GeneNetwork?","Arabidopsis thaliana, Glycine max, Solanum lycopersicum, Hordeum vulgare, Populus trichocarpa","http://rdf.genenetwork.org/v1/id/family_Plants"
+"The MPTP neurotoxin is used to model Parkinson's disease in BXD mice. What locus has the highest LOD score for serotonin response in male striatum after MPTP treatment?","LOD: 3.9673994445052551683, Locus: rsm10000002155","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_CPU_Sal_MPTP_M"
+"Which PubMed study is associated with MPTP-induced serotonin changes in BXD male striatum?","23558233","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_CPU_Sal_MPTP_M"
+"Which dopamine metabolite was measured in male BXD striatum 48 hours after MPTP treatment, and what is its highest association score?","3-methoxytyramine (3-MT), LOD: 2.0516721070584273967, Locus: rsm10000001111","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3MT_CPU_MPTP_M"
+"What is the highest LOD score for the difference in 3-MT levels between saline and MPTP-treated male BXD striatum?","LOD: 2.5855803408910196239, Locus: rsm10000002558","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3MT_CPU_Sal_MPTP_M"
+"What is 3NP_Tox?","It is a trait: 3-nitropropionic acid neurodegeneration in striatum caudate putamen 50 mg/kg ip in young adults, presence or absence of lesions [-1 = lesions, +1 = no lesions]","http://rdf.genenetwork.org/v1/id/phenotype_3NP_Tox"
+"Is there any trait measured in the olfactory bulb of BXD mice one hour after saline injection?","Yes, THDOC (3a,5a-pregnane-3a,21-diol-20-one)","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5a_THDOC_OB"
+"Which neurotransmitters were measured in the MPTP Parkinson's disease model in BXD mice?","Serotonin (5-HT), 5-HIAA (serotonin metabolite), and 3-methoxytyramine (3-MT, dopamine metabolite) in caudate-putamen striatum","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_CPU_Sal_MPTP_M"
+"What is the difference in serotonin levels between male and female BXD diencephalon?","Male (5HT_DC_M): Mean 13.318529409520765583, Female (5HT_DC_F): Mean 14.784413830987338301","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_DC_M, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_5HT_DC_F"
+"What is the highest association score for acetaminophen (APAP) hepatocyte viability response in male BXD liver?","LOD: 4.7056254566002762019, Locus: rsm10000003730","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_APAP_log2"
+"What is the full description of APAP_log2?","Acetaminophen APAP hepatocyte viability response 9 hr after treatment 0.5 mM in vehicle, lactate dehydrogenase LDH release assay adjusted for DNA content, young adult male liver [log2 LDH/DNA ratio treated/control]","http://rdf.genenetwork.org/v1/id/phenotype_APAP_log2"
+"Which PubMed publication references the study on acoustic startle response in BXD mice?","10371755","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_ASR110dB10kHz"
+"What is the highest LOD score for raw striatum volume in BXD mice?","LOD: 2.8594045664833709353, Locus: rsm10000003425","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_StriatumVolRaw"
+"Which marker had the highest LOD score for sleep need measured as slow wave sleep delta EEG power after 6 hours deprivation in BXD mice, and what is its LOD score?","Locus: rsm10000010045, LOD: 3.6721742203463301202","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_SleepNeed"
+"Which T cell populations were estimated by TIMER 2.0 in breast cancer tumors from BXD x C3-1TAg F1 females, and what are the most interesting loci?","CD4 effector memory T cells (LOD: 1.4524872961762274048 Locus: rsm10000014666), CD8 T cells (LOD: 1.8031565920123300284, Locus: rs29338660)","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_CD4_effector_memory_XCELL, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T_cell_CD8_TIMER"
+"Which microbiome genera were identified in BXD mice and reported in the PubMed publication 31634382?","Streptococcus, Staphylococcus, Stenotrophomonas","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_Streptococcus_genus_CD_norm_log10, https://rdf.genenetwork.org/v1/id/trait_BXDPublish_Staphylococcus_genus_CD_norm_log10, https://rdf.genenetwork.org/v1/id/trait_BXDPublish_Stenotrophomonas_genus_CD_norm_log10"
+"Which autoimmune thyroid disease model is represented by the TBI and T4 traits in BXD mice?","Graves' disease model induced by TSHR A-subunit adenovirus immunization","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_TBI_3x_, http://rdf.genenetwork.org/v1/id/trait_BXDPublish_T4__3x_"
+"What is the GeneNetwork trait page for 3NP neurodegeneration?","https://genenetwork.org/show_trait?trait_id=15959&dataset=BXDPublish","http://rdf.genenetwork.org/v1/id/trait_BXDPublish_3NP_Tox"
+"Which liver metabolites were measured after overnight fasting in males at 29 weeks of age on normal chow?","AAP_Lys_CD, Sphingomyelin_d18_2_liver_CD, and TAG_56_7__2_CD","http://rdf.genenetwork.org/v1/id/phenotype_AAP_Lys_CD https://rdf.genenetwork.org/v1/id/phenotype_Sphingomyelin_d18_2_liver_CD, https://rdf.genenetwork.org/v1/id/phenotype_TAG_56_7__2_CD"
\ No newline at end of file
diff --git a/aisearch/pyproject.toml b/aisearch/pyproject.toml
new file mode 100644
index 0000000..6e08581
--- /dev/null
+++ b/aisearch/pyproject.toml
@@ -0,0 +1,50 @@
+[project]
+name = "gnais"
+version = "0.4.0"
+description = "Seach tool powered with AI to access GeneNetwork metadata"
+authors = [
+    {name = "Johannes Medagbe", email = "johanmedagbe@gmail.com"}
+]
+maintainers = [
+    {name = "Johannes Medagbe", email = "johanmedagbe@gmail.com"},
+    {name = "Munyoki Kilyungi", email = "me@bonfacemunyoki.com"},
+]
+license = {text = "MIT"}
+readme = "../README.md"
+requires-python = ">=3.11,<3.15"
+dependencies = [
+    "authlib (==1.5.1)",
+    "chromadb (==1.5.8)",
+    "requests (==2.32.5)",
+    "dspy (==3.1.3)",
+    "flask-caching (==2.3.1)",
+    "flask-limiter (==4.1.1)",
+    "quart-auth (==0.11.0)",
+    "langchain (==1.2.10)",
+    "langchain-community (==0.4.1)",
+    "langchain-core (==1.2.17)",
+    "langgraph (==1.0.10)",
+    "pydantic (==2.13.2)",
+    "python-dotenv (==1.1.1)",
+    "rank-bm25 (==0.2.2)",
+    "redis (==7.1.0)",
+    "sentence-transformers (==5.4.1)",
+    "torch (==2.11.0)",
+    "tqdm (==4.67.3)",
+    "sparqlwrapper (==2.0.0)",
+    "langchain-classic (==1.0.2)",
+    "langchain-huggingface (==1.2.1)",
+    "quart (==0.20.0)",
+    "quart-flask-patch (==0.3.0)",
+    "hypercorn (==0.18.0)",
+    "mem0ai[nlp] (==2.0.1)",
+    "rich (==13.9.4)",
+    "pandas (>=3.0.3,<4.0.0)",
+]
+
+[project.urls]
+Homepage = "https://github.com/genenetwork/gn-ai"
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"
diff --git a/aisearch/results/barplot_all_metrics.png b/aisearch/results/barplot_all_metrics.png
new file mode 100644
index 0000000..78051e1
--- /dev/null
+++ b/aisearch/results/barplot_all_metrics.png
Binary files differdiff --git a/aisearch/results/barplot_extra_metrics.png b/aisearch/results/barplot_extra_metrics.png
new file mode 100644
index 0000000..9f5385b
--- /dev/null
+++ b/aisearch/results/barplot_extra_metrics.png
Binary files differdiff --git a/aisearch/results/barplot_model_local_new_metric.png b/aisearch/results/barplot_model_local_new_metric.png
new file mode 100644
index 0000000..cd7e1e7
--- /dev/null
+++ b/aisearch/results/barplot_model_local_new_metric.png
Binary files differdiff --git a/aisearch/results/barplot_model_new_metric.png b/aisearch/results/barplot_model_new_metric.png
new file mode 100644
index 0000000..78b84cf
--- /dev/null
+++ b/aisearch/results/barplot_model_new_metric.png
Binary files differdiff --git a/aisearch/results/barplot_traditional_metrics.png b/aisearch/results/barplot_traditional_metrics.png
new file mode 100644
index 0000000..a3d0e93
--- /dev/null
+++ b/aisearch/results/barplot_traditional_metrics.png
Binary files differdiff --git a/aisearch/results/boxplot_system_local_new_metric.png b/aisearch/results/boxplot_system_local_new_metric.png
new file mode 100644
index 0000000..45cf4b4
--- /dev/null
+++ b/aisearch/results/boxplot_system_local_new_metric.png
Binary files differdiff --git a/aisearch/results/boxplot_system_new_metric.png b/aisearch/results/boxplot_system_new_metric.png
new file mode 100644
index 0000000..6ad5d9e
--- /dev/null
+++ b/aisearch/results/boxplot_system_new_metric.png
Binary files differdiff --git a/aisearch/results/dspy/gptoss_20b_dspy_results.csv b/aisearch/results/dspy/gptoss_20b_dspy_results.csv
new file mode 100644
index 0000000..396fdef
--- /dev/null
+++ b/aisearch/results/dspy/gptoss_20b_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+10.26,20.51,17.95,12.82,25.64
+12.82,17.95,12.82,17.95,17.95
+7.69,23.08,15.38,10.26,25.64
\ No newline at end of file
diff --git a/aisearch/results/dspy/haiku_4.5_dspy_results.csv b/aisearch/results/dspy/haiku_4.5_dspy_results.csv
new file mode 100644
index 0000000..a727f3d
--- /dev/null
+++ b/aisearch/results/dspy/haiku_4.5_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+28.21,25.64,30.77,28.21,46.15
+23.08,33.33,23.08,28.21,35.9
+17.95,23.08,33.33,33.33,38.46
diff --git a/aisearch/results/dspy/haiku_4.5_opus_4.8_dspy_results.csv b/aisearch/results/dspy/haiku_4.5_opus_4.8_dspy_results.csv
new file mode 100644
index 0000000..20a7b8d
--- /dev/null
+++ b/aisearch/results/dspy/haiku_4.5_opus_4.8_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+25.64,33.33,51.28,64.1,76.92
+23.81,35.72,55.91,63.87,81.16
+26.12,32.68,56.11,67.52,83.8
diff --git a/aisearch/results/dspy/opus_4.8_dspy_results.csv b/aisearch/results/dspy/opus_4.8_dspy_results.csv
new file mode 100644
index 0000000..f240009
--- /dev/null
+++ b/aisearch/results/dspy/opus_4.8_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+31.16,37.7,59.14,71.82,86.23
+26.92,34.66,52.32,68.52,79.74
+28.71,38.42,58.7,73.49,88.45
diff --git a/aisearch/results/dspy/qwen3_30b_dspy_results.csv b/aisearch/results/dspy/qwen3_30b_dspy_results.csv
new file mode 100644
index 0000000..1fab988
--- /dev/null
+++ b/aisearch/results/dspy/qwen3_30b_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+10.26,30.77,17.95,15.38,33.33
+17.95,20.51,17.95,17.95,38.46
+12.82,25.64,15.38,15.38,25.64
diff --git a/aisearch/results/dspy/qwen3_30b_opus_4.8_dspy_results.csv b/aisearch/results/dspy/qwen3_30b_opus_4.8_dspy_results.csv
new file mode 100644
index 0000000..f7e7058
--- /dev/null
+++ b/aisearch/results/dspy/qwen3_30b_opus_4.8_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+20.51,20.51,43.59,51.28,58.97
+22.33,24.82,42.34,53.9,59.46
+23.11,25.67,44.82,55.62,60.23
diff --git a/aisearch/results/heatmap_model_new_metric.png b/aisearch/results/heatmap_model_new_metric.png
new file mode 100644
index 0000000..ba5d5bb
--- /dev/null
+++ b/aisearch/results/heatmap_model_new_metric.png
Binary files differdiff --git a/aisearch/results/local/gptoss_20b_dspy_results.csv b/aisearch/results/local/gptoss_20b_dspy_results.csv
new file mode 100644
index 0000000..396fdef
--- /dev/null
+++ b/aisearch/results/local/gptoss_20b_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+10.26,20.51,17.95,12.82,25.64
+12.82,17.95,12.82,17.95,17.95
+7.69,23.08,15.38,10.26,25.64
\ No newline at end of file
diff --git a/aisearch/results/local/qwen3_30b_dspy_results.csv b/aisearch/results/local/qwen3_30b_dspy_results.csv
new file mode 100644
index 0000000..1fab988
--- /dev/null
+++ b/aisearch/results/local/qwen3_30b_dspy_results.csv
@@ -0,0 +1,4 @@
+base,rag,graph rag,agent,combi
+10.26,30.77,17.95,15.38,33.33
+17.95,20.51,17.95,17.95,38.46
+12.82,25.64,15.38,15.38,25.64
diff --git a/aisearch/results/simple/gptoss_20b_simple_results.csv b/aisearch/results/simple/gptoss_20b_simple_results.csv
new file mode 100644
index 0000000..bd7e391
--- /dev/null
+++ b/aisearch/results/simple/gptoss_20b_simple_results.csv
@@ -0,0 +1,6 @@
+,base_0,base_1,base_2,rag 0,rag 1,rag 2,graph rag 0,graph rag 1,graph rag 2,agent 0,agent 1,agent 2,combi 0,combi 1,combi 2
+precision (%),0.26301476300353505,0.2889702889579716,0.20755078254246676,0.23357440279560704,0.21645299144352245,0.21678876678018782,0.2613756613663501,0.22057387056511396,0.19908424907599526,0.20334665333966875,0.20469067968061092,0.20019723864902841,0.31415436414297093,0.2837736337610654,0.27863247861902846
+recall (%),0.22828930327954408,0.23692372441513876,0.16076046460151858,0.23395678394583205,0.2549857549738359,0.24924889923685412,0.22915140414345148,0.22556980056073386,0.23874643873775275,0.22784900283973328,0.21212121211044105,0.19748769747967457,0.3014667593494787,0.2943099635277914,0.2778813778664061
+f1 (%),0.23181725681241305,0.24134144066833566,0.165197622887357,0.2159597712185191,0.21805582331440115,0.22055722055277016,0.22410459910099512,0.21470196469781072,0.20691900691529705,0.21089797142055872,0.1919299905679058,0.17789238052051465,0.2882868329875755,0.27247186011786045,0.2707100591648955
+speed (s),18.675207627125275,17.571283542192898,18.616595769539856,75.24318775152548,68.74417766546591,64.19787680185757,186.85869849033844,175.8293514251709,161.0574466142899,317.49614357336975,356.45982903089276,355.61356481527673,508.0973210701576,471.30866420574677,398.941607151276
+n_token,1237.871794871795,1191.4615384615386,1221.974358974359,7827.7692307692305,7882.74358974359,7225.025641025641,11591.128205128205,12679.333333333334,12304.410256410256,7238.923076923077,8179.128205128205,7343.974358974359,7794.2307692307695,6779.4358974358975,6754.974358974359
diff --git a/aisearch/results/simple/qwen3_30b_simple_results.csv b/aisearch/results/simple/qwen3_30b_simple_results.csv
new file mode 100644
index 0000000..743508b
--- /dev/null
+++ b/aisearch/results/simple/qwen3_30b_simple_results.csv
@@ -0,0 +1,6 @@
+,base_0,base_1,base_2,rag 0,rag 1,rag 2,graph rag 0,graph rag 1,graph rag 2,agent 0,agent 1,agent 2,combi 0,combi 1,combi 2
+precision (%),0.23803418802112608,0.2790598290431617,0.2864468864303438,0.19109686609003615,0.2911172161016248,0.3380952380785822,0.19288678309566984,0.15842490842007195,0.10398860398312756,0.20329670329017022,0.20066512434341807,0.18821178820779982,0.3287240537156438,0.2615083634213096,0.32069597068691497
+recall (%),0.18307433306599552,0.16985535830821608,0.15426610426163537,0.20789452711491885,0.25462315460959833,0.2739963739830963,0.21082621081915262,0.20551670550806625,0.11680911680171732,0.21837606836893095,0.22646520145839125,0.25213675212683023,0.40617715616170597,0.3617877492718898,0.36025641023950117
+f1 (%),0.19322224615893044,0.20000078268798996,0.1914271625776011,0.17679713448627324,0.24054748669567216,0.2768523873722878,0.18099336560569812,0.1726752779355906,0.10826210825912078,0.1991870621685638,0.20830910830612226,0.20500354346259028,0.3420496215647281,0.283821144593049,0.31657113316540375
+speed (s),30.362717157755142,29.20200067300063,30.13693245863303,117.16252853320195,113.74057094256084,101.67406824918893,318.00476088279333,343.9846796500377,344.6213834530268,842.0525810046074,853.5273833152576,810.4330469217056,1639.90965063144,1577.5455975899329,1484.4385216602911
+n_token,1904.3076923076924,1844.7692307692307,1894.871794871795,4701.589743589743,4976.897435897436,4560.871794871795,6636.5641025641025,6916.692307692308,7123.641025641025,4756.0,4731.589743589743,4812.051282051282,5479.615384615385,5673.128205128205,4987.5641025641025
diff --git a/aisearch/scripts/agent_search.py b/aisearch/scripts/agent_search.py
new file mode 100644
index 0000000..a9e7fc8
--- /dev/null
+++ b/aisearch/scripts/agent_search.py
@@ -0,0 +1,81 @@
+"""This is the main module of the package using agent tool calling"""
+
+import argparse
+import asyncio
+import os
+from typing import Any
+
+import dspy
+import torch
+from gnais.config import Config
+from gnais.search.agent import agent_search
+from mem0 import Memory
+from mem0.configs.base import MemoryConfig
+
+
+def digest(query: str, memory: Any = None, user_id: str = "default_user"):
+    async def _run():
+        output = ""
+        async for chunk in agent_search(
+            query=query,
+            sparql_url=Config.SPARQL_ENDPOINT,
+            memory=memory,
+            user_id=user_id,
+        ):
+            if isinstance(chunk, dict) and "final" in chunk:
+                final = chunk["final"]
+                output = final
+                print(final, end="", flush=True)
+            else:
+                output += str(chunk)
+                print(chunk, end="", flush=True)
+        print()
+        return output
+
+    return asyncio.run(_run())
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("query", help="Search query")
+    parser.add_argument("--user-id", default="test-user", help="Mem0 user identity")
+    args = parser.parse_args()
+
+    torch.manual_seed(Config.SEED)
+    if torch.cuda.is_available():
+        torch.cuda.manual_seed_all(Config.SEED)
+
+    dspy.configure(lm=Config.DEFAULT_LLM)
+    os.environ[f"{Config.MEMORY_MODEL.split('/')[0].upper()}_API_KEY"] = Config.API_KEY
+    memory_config = MemoryConfig(
+        llm={
+            "provider": "litellm",
+            "config": {
+                "model": Config.MEMORY_MODEL,
+                "temperature": 0.1,
+                "max_tokens": 2_000,
+                "api_key": Config.API_KEY,
+            },
+        },
+        embedder={
+            "provider": "huggingface",
+            "config": {
+                "model": "Qwen/Qwen3-Embedding-0.6B",
+                "embedding_dims": 1024,
+                "model_kwargs": {
+                    "trust_remote_code": True,
+                    "device": "cuda" if torch.cuda.is_available() else "cpu",
+                },
+            },
+        },
+        vector_store={
+            "provider": "chroma",
+            "config": {
+                "collection_name": "mem0",
+                "path": os.path.join(Config.DB_PATH, "mem0_chroma"),
+            },
+        },
+    )
+    memory = Memory(config=memory_config)
+    output = digest(args.query, memory=memory, user_id=args.user_id)
+    print(output)
diff --git a/aisearch/scripts/barplot_metrics.py b/aisearch/scripts/barplot_metrics.py
new file mode 100644
index 0000000..0932f80
--- /dev/null
+++ b/aisearch/scripts/barplot_metrics.py
@@ -0,0 +1,86 @@
+"""
+Script to plot results with traditional metrics using local models
+"""
+
+import os
+
+import matplotlib.pyplot as plt
+import pandas as pd
+import seaborn as sns
+
+INPUT1_PATH = os.getenv("INPUT1_PATH")
+if INPUT1_PATH is None:
+    raise FileNotFoundError("Set INPUT1_PATH or path to simple results directory")
+
+INPUT2_PATH = os.getenv("INPUT2_PATH")
+if INPUT2_PATH is None:
+    raise FileNotFoundError("Set INPUT2_PATH or path to dspy results directory")
+
+OUTPUT_PATH = os.getenv("OUTPUT_PATH")
+if OUTPUT_PATH is None:
+    raise FileNotFoundError("Set OUTPUT_PATH or path to save final plot")
+
+
+def format_results(simple_path: str, dspy_path: str) -> pd.DataFrame:
+    df_list = []
+    simple_files = os.listdir(simple_path)
+    for f in simple_files:
+        model = f.strip("_simple_results.csv")
+        data = pd.read_csv(f"{simple_path}/{f}", index_col=0)
+        trans_data = data.transpose()
+        trans_data = pd.concat(
+            [trans_data[column] * 100 for column in trans_data.columns], axis=1
+        )
+        trans_data["model"] = model
+        trans_data["system"] = [sys[:-2] for sys in list(trans_data.index)]
+        df_list.append(trans_data)
+    final_concat = pd.concat(df_list)
+    dspy_files = os.listdir(dspy_path)
+    cumul = []
+    for f in dspy_files:
+        data = pd.read_csv(f"{dspy_path}/{f}")
+        aligned = pd.concat([data[column] for column in data.columns])
+        cumul.extend(aligned)
+    final_concat["satisfaction frequency (%)"] = cumul
+    return final_concat
+
+
+def plot(data: pd.DataFrame, columns: list[str], output_path: str, title: str):
+    fig, axes = plt.subplots(len(columns), 1, figsize=(10, 10))
+    axes = axes.flatten()
+    for ind, column in enumerate(columns):
+        sns.barplot(
+            data=data,
+            ax=axes[ind],
+            x="system",
+            y=column,
+            hue="model",
+            palette="ocean_r",
+        )
+    fig.suptitle(title, fontsize=15)
+    plt.tight_layout()
+    plt.savefig(output_path, dpi=1000)
+    plt.show()
+
+
+def plot_results(data: pd.DataFrame, output_path: str):
+    metrics = list(data.columns)[:3]
+    metrics.append(data.columns[-1])
+    plot(
+        data,
+        metrics,
+        f"{output_path}/barplot_all_metrics.png",
+        "Performance comparison of standard and new metrics",
+    )
+    extras = list(data.columns)[3:5]
+    plot(
+        data,
+        extras,
+        f"{output_path}/barplot_extra_metrics.png",
+        "Insights from additional metrics",
+    )
+
+
+if __name__ == "__main__":
+    df = format_results(INPUT1_PATH, INPUT2_PATH)
+    plot_results(df, OUTPUT_PATH)
diff --git a/aisearch/scripts/dspy_evaluate.py b/aisearch/scripts/dspy_evaluate.py
new file mode 100644
index 0000000..b7d6d6f
--- /dev/null
+++ b/aisearch/scripts/dspy_evaluate.py
@@ -0,0 +1,126 @@
+"""Script for performance evaluation of GN AI systems using DSPy.Evaluate and custom decision function"""
+
+import os
+from typing import Any
+
+import dspy
+import pandas as pd
+import torch
+from gnais.config import Config
+from gnais.evaluation.utils import (
+    agent_digest,
+    get_dataset,
+    graph_rag_digest,
+    hybrid_digest,
+    mark,
+    rag_digest,
+)
+
+
+def evaluator(
+    gold: dspy.Example,
+    pred: str,
+    trace=None,
+    pred_name=None,
+    pred_trace=None,
+) -> int:
+    query = gold["query"]
+    true_response = gold["answer"]
+    precision, recall, f1 = mark(query, pred, true_response)
+    return 1 if (f1 >= 0.6 or recall >= 0.7 or precision >= 0.7) else 0
+
+
+def run_eval(
+    runner: dspy.Evaluate,
+    evaluation_set: list[dspy.Example],
+    search_func: Any,
+) -> dspy.evaluate.EvaluationResult:
+
+    search = dspy.Tool(
+        name="search",
+        desc="Answer the question asked by the user.",
+        args={
+            "query": {
+                "type": "string",
+                "desc": "question asked by the user",
+            },
+        },
+        func=search_func,
+    )
+
+    class Signature(dspy.Signature):
+        query: str = dspy.InputField(desc="User question")
+        answer: str = dspy.OutputField(
+            desc="Answer to user's query with details and explanations"
+        )
+
+    # Wrap hybrid search into a DSPy module that is compatible with DSPy evaluation
+    program = dspy.ReAct(
+        signature=Signature, tools=[search], max_iters=2
+    )  # limit iterations
+
+    return runner(program)
+
+
+if __name__ == "__main__":
+    DATASET_PATH = os.getenv("DATASET_PATH")
+    if DATASET_PATH is None:
+        raise FileNotFoundError("DATASET_PATH must be set for evaluation")
+
+    OUTPUT_PATH = os.getenv("OUTPUT_PATH")
+    if OUTPUT_PATH is None:
+        raise FileNotFoundError("OUTPUT_PATH must be set for evaluation")
+
+    N_ITERATIONS = int(os.getenv("N_ITERATIONS"))
+    if N_ITERATIONS is None:
+        raise ValueError("N_ITERATIONS must be set for evaluation")
+
+    JUDGE_MODEL = os.getenv("JUDGE_MODEL")
+    if JUDGE_MODEL is None:
+        raise ValueError("JUDGE MODEL must be set for evaluation")
+
+    torch.manual_seed(Config.SEED)
+    if torch.cuda.is_available():
+        torch.cuda.manual_seed_all(Config.SEED)
+
+    dspy.configure(lm=Config.DEFAULT_LLM)
+
+    judge_llm = dspy.LM(
+        model=JUDGE_MODEL,
+        api_key=Config.API_KEY,
+        max_tokens=1_000,
+        temperature=1,
+        cache=False,
+        verbose=False,
+    )
+
+    evaluation_set = get_dataset(DATASET_PATH)
+    evaluate = dspy.Evaluate(
+        devset=evaluation_set,
+        metric=evaluator,
+        num_threads=1,
+        provide_traceback=True,
+        display_table=False,
+        display_progress=True,
+        lm=judge_llm,
+    )
+    collection = {}
+
+    # Run evaluation set with LLM only
+    base = dspy.ChainOfThought("query -> answer: str")
+    base_output = [evaluate(base).get("score") for n in range(N_ITERATIONS)]
+    collection["base"] = base_output
+
+    # Run evaluation set with GN systems
+    for system in [rag_digest, graph_rag_digest, agent_digest, hybrid_digest]:
+        system_name = " ".join(system.__name__.split("_")[:-1])
+        print(f"Running evaluation for {system_name}")
+        temp = []
+        for n in range(N_ITERATIONS):
+            print(f"Iteration {n+1}")
+            system_output = run_eval(evaluate, evaluation_set, system)
+            temp.append(system_output.get("score"))
+        collection[system_name] = temp
+        print(f"Evaluation completed for {system_name}")
+    final = pd.DataFrame(collection)
+    final.to_csv(OUTPUT_PATH, index=False)
diff --git a/aisearch/scripts/fetch_metadata.py b/aisearch/scripts/fetch_metadata.py
new file mode 100755
index 0000000..36b9570
--- /dev/null
+++ b/aisearch/scripts/fetch_metadata.py
@@ -0,0 +1,501 @@
+#! /usr/bin/env python3
+"""Fetch RDF triples grouped by subject and convert to natural language sentences.
+
+This script fetches data from GeneNetwork's SPARQL endpoint, grouping triples by
+subject and converting them to human-readable sentences. It processes different
+entity types in parallel using threads and writes each type to its own file.
+"""
+
+import argparse
+import json
+import os
+import random
+import time
+from collections import defaultdict
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from typing import Dict, List, Optional, Tuple
+from urllib.error import HTTPError
+
+from SPARQLWrapper import JSON, SPARQLWrapper
+
+SPARQL_ENDPOINT = "https://rdf.genenetwork.org/sparql"
+DEFAULT_GRAPH = "http://rdf.genenetwork.org/v1"
+
+PREFIXES = {
+    "rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
+    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
+    "skos": "http://www.w3.org/2004/02/skos/core#",
+    "dct": "http://purl.org/dc/terms/",
+    "gnc": "http://rdf.genenetwork.org/v1/category/",
+    "gnt": "http://rdf.genenetwork.org/v1/term/",
+    "gn": "http://rdf.genenetwork.org/v1/id/",
+    "xkos": "http://rdf-vocabulary.ddialliance.org/xkos#",
+    "fabio": "http://purl.org/spar/fabio/",
+    "prism": "http://prismstandard.org/namespaces/basic/2.0/",
+    "owl": "http://www.w3.org/2002/07/owl#",
+    "xsd": "http://www.w3.org/2001/XMLSchema#",
+    "dcat": "http://www.w3.org/ns/dcat#",
+    "qb": "http://purl.org/linked-data/cube#",
+    "schema": "https://schema.org/",
+    "pubmed": "http://rdf.ncbi.nlm.nih.gov/pubmed/",
+    "obo": "http://purl.obolibrary.org/obo/",
+    "bfo": "http://purl.obolibrary.org/obo/BFO_",
+    "sdmx-measure": "http://purl.org/linked-data/sdmx/2009/measure#",
+    "foaf": "http://xmlns.com/foaf/0.1/",
+}
+
+TEMPLATES = {
+    # RDF / RDFS
+    "http://www.w3.org/1999/02/22-rdf-syntax-ns#type": "is a {obj}",
+    "http://www.w3.org/2000/01/rdf-schema#label": "is labeled as '{obj}'",
+    "http://www.w3.org/2000/01/rdf-schema#comment": "has the note: {obj}",
+    # SKOS
+    "http://www.w3.org/2004/02/skos/core#prefLabel": "is called '{obj}'",
+    "http://www.w3.org/2004/02/skos/core#altLabel": "is also known as '{obj}'",
+    "http://www.w3.org/2004/02/skos/core#definition": "is defined as: {obj}",
+    "http://www.w3.org/2004/02/skos/core#member": "includes member {obj}",
+    "http://www.w3.org/2004/02/skos/core#notation": "has notation {obj}",
+    "http://www.w3.org/2004/02/skos/core#inScheme": "is in scheme {obj}",
+    # DCTERMS
+    "http://purl.org/dc/terms/title": "has title '{obj}'",
+    "http://purl.org/dc/terms/description": "is described as: '{obj}'",
+    "http://purl.org/dc/terms/identifier": "has identifier {obj}",
+    "http://purl.org/dc/terms/abstract": "has abstract '{obj}'",
+    "http://purl.org/dc/terms/creator": "was created by {obj}",
+    "http://purl.org/dc/terms/contributor": "has contributor {obj}",
+    "http://purl.org/dc/terms/created": "was created on {obj}",
+    "http://purl.org/dc/terms/references": "references {obj}",
+    "http://purl.org/dc/terms/hasVersion": "has version {obj}",
+    # GeneNetwork Terms
+    "http://rdf.genenetwork.org/v1/term/has_set_code": "provides a unique identifier code for a resource set {obj}",
+    "http://rdf.genenetwork.org/v1/term/genetic_type": "describes the genetic architecture of a resource set as {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_species": "belongs to species {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_strain": "lists all strains that belong to this resource including {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_taxonomic_family": "links to taxonomic family {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_uniprot_taxon_id": "has UniProt taxonomic id {obj}",
+    "http://rdf.genenetwork.org/v1/term/short_name": "has short name {obj}",
+    "http://rdf.genenetwork.org/v1/term/uses_mapping_method": "uses mapping method {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_reference_population": "has reference population {obj}",
+    "http://rdf.genenetwork.org/v1/term/alias": "has alias {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_phenotype": "measures phenotype {obj}",
+    "http://rdf.genenetwork.org/v1/term/abbreviation": "has abbreviation {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_lab_code": "has lab code {obj}",
+    "http://rdf.genenetwork.org/v1/term/lod_score": "has LOD score (Peak -logP) {obj}",
+    "http://rdf.genenetwork.org/v1/term/mean": "has mean value {obj}",
+    "http://rdf.genenetwork.org/v1/term/submitter": "was submitted by {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_genotype_files": "has genotype files {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_marker_count": "has {obj} DNA markers/SNPs",
+    "http://rdf.genenetwork.org/v1/term/symbol": "has gene symbol {obj}",
+    "http://rdf.genenetwork.org/v1/term/chr": "is on chromosome {obj}",
+    "http://rdf.genenetwork.org/v1/term/chromosome": "is on chromosome {obj}",
+    "http://rdf.genenetwork.org/v1/term/mb": "is at {obj} megabases",
+    "http://rdf.genenetwork.org/v1/term/sequence": "has sequence {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_target_id": "has target id {obj}",
+    "http://rdf.genenetwork.org/v1/term/targets_region": "targets region {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_specificity": "has specificity {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_blat_score": "has BLAT score {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_blat_mb_start": "has BLAT start position {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_blat_mb_end": "has BLAT end position {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_blat_seq": "has BLAT sequence {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_target_seq": "has target sequence {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_homologene_id": "has HomoloGene ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_uniprot_id": "has UniProt ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_pub_chem_id": "has PubChem ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_kegg_id": "has KEGG ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_omim_id": "has OMIM ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_chebi_id": "has ChEBI ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/initial": "has initial {obj}",
+    "http://rdf.genenetwork.org/v1/term/reason": "was modified because {obj}",
+    "http://rdf.genenetwork.org/v1/term/belongs_to_category": "belongs to category {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_gene_id": "has gene ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/gene_symbol": "has gene symbol {obj}",
+    "http://rdf.genenetwork.org/v1/term/transcript": "has transcript {obj}",
+    "http://rdf.genenetwork.org/v1/term/strand": "is on strand {obj}",
+    "http://rdf.genenetwork.org/v1/term/tx_start": "has transcription start {obj}",
+    "http://rdf.genenetwork.org/v1/term/tx_end": "has transcription end {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_align_id": "has alignment ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_protein_id": "has protein ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_rgd_id": "has RGD ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_geo_series_id": "has GEO series ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_go_tree_value": "has GO tree value {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_case_info": "has case info: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_citation": "has citation: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_contributors": "has contributors: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_data_processing_info": "has data processing info: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_experiment_design": "has experiment design: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_experiment_design_info": "has experiment design info: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_experiment_type": "has experiment type: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_molecular_trait": "has molecular trait {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_phenotype_data": "has phenotype data {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_phenotype_trait": "has phenotype trait {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_platform_info": "has platform info: {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_probeset_data": "has probeset data {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_samples": "has samples {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_specifics": "has specifics: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_summary": "has summary: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/has_tissue_info": "has tissue info: '{obj}'",
+    "http://rdf.genenetwork.org/v1/term/uses_genechip": "uses genechip platform {obj}",
+    "http://rdf.genenetwork.org/v1/term/uses_normalization_method": "uses normalization method {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_probeset": "has probeset {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_genotype_data": "has genotype data {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_sequence": "has sequence {obj}",
+    "http://rdf.genenetwork.org/v1/term/source": "has source {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_kg_id": "has KG ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_unigen_id": "has UniGene ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/additive": "has additive effect {obj}",
+    "http://rdf.genenetwork.org/v1/term/se": "has standard error {obj}",
+    "http://rdf.genenetwork.org/v1/term/pvalue": "has p-value {obj}",
+    "http://rdf.genenetwork.org/v1/term/h2": "has heritability (h2) {obj}",
+    "http://rdf.genenetwork.org/v1/term/assigned_species": "has assigned species {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_family_order_id": "has family order ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/has_population_order_id": "has population order ID {obj}",
+    "http://rdf.genenetwork.org/v1/term/locus": "has locus {obj}",
+    "http://rdf.genenetwork.org/v1/term/mb_mm8": "has position (mm8) {obj} megabases",
+    # FOAF
+    "http://xmlns.com/foaf/0.1/homepage": "has homepage {obj}",
+    "http://xmlns.com/foaf/0.1/name": "has name '{obj}'",
+    "http://xmlns.com/foaf/0.1/familyName": "has family name '{obj}'",
+    "http://xmlns.com/foaf/0.1/givenName": "has given name '{obj}'",
+    "http://xmlns.com/foaf/0.1/mbox": "has email {obj}",
+    # XKOS
+    "http://rdf-vocabulary.ddialliance.org/xkos#depth": "is at depth level {obj}",
+    # FABIO
+    "http://purl.org/spar/fabio/hasPubMedId": "has PubMed ID {obj}",
+    "http://purl.org/spar/fabio/Journal": "was published in journal '{obj}'",
+    "http://purl.org/spar/fabio/page": "is on pages {obj}",
+    "http://purl.org/spar/fabio/hasPublicationYear": "was published in year {obj}",
+    # PRISM
+    "http://prismstandard.org/namespaces/basic/2.0/volume": "is in volume {obj}",
+    "http://prismstandard.org/namespaces/basic/2.0/publicationDate": "was published on {obj}",
+    # DCAT
+    "http://www.w3.org/ns/dcat#distribution": "has distribution {obj}",
+    "http://www.w3.org/ns/dcat#isPartOf": "is part of {obj}",
+    # OWL
+    "http://www.w3.org/2002/07/owl#equivalentClass": "is equivalent to {obj}",
+}
+
+
+TYPE_QUERIES = [
+    "a dcat:Dataset",
+    "a gnc:set",
+    "a gnc:species",
+    "a gnc:reference_population",
+    "a skos:Concept",
+    "a gnc:molecular_trait",
+    "a gnc:phenotype",
+    "a gnc:strain",
+    "a fabio:ResearchPaper",
+    "a gnc:gene",
+    "a gnc:gene_symbol",
+    "a gnc:dna_marker",
+    "a gnc:ncbi_wiki_entry",
+]
+
+
+def type_pattern_to_filename(type_pattern: str) -> str:
+    """Convert a type pattern to a safe filename."""
+    return f"{type_pattern.split(':')[-1].lower()}.json"
+
+
+def uri_to_prefixed(uri: str) -> str:
+    """Convert full URI to prefixed form."""
+    for prefix, ns in PREFIXES.items():
+        if uri.startswith(ns):
+            return f"{prefix}:{uri[len(ns) :]}"
+    return uri
+
+
+def triple_to_sentence(subj: str, pred: str, obj: str) -> str:
+    """Convert triple to natural language sentence."""
+    obj_prefixed = uri_to_prefixed(obj)
+    if pred in TEMPLATES:
+        return f"{subj} {TEMPLATES[pred].format(obj=obj_prefixed)}."
+    pred_prefixed = uri_to_prefixed(pred)
+    return f"{subj} {pred_prefixed} {obj_prefixed}."
+
+
+def fetch_with_retry(
+    sparql: SPARQLWrapper, max_retries: int = 3, base_delay: float = 2.0
+) -> Optional[Dict]:
+    """Execute SPARQL query with retry logic and exponential backoff."""
+    for attempt in range(max_retries):
+        try:
+            return sparql.queryAndConvert()
+        except HTTPError as e:
+            if e.code == 504 and attempt < max_retries - 1:
+                # Exponential backoff with jitter
+                delay = base_delay * (2**attempt) + random.uniform(0, 1)
+                print(
+                    f"    Timeout (attempt {attempt + 1}/{max_retries}), retrying in {delay:.1f}s..."
+                )
+                time.sleep(delay)
+            else:
+                raise
+    return None
+
+
+def fetch_count_for_type(type_pattern: str) -> int:
+    """Get the total count of triples for a type pattern.
+
+    Args:
+        type_pattern: The type pattern to count
+
+    Returns:
+        Total count of triples
+    """
+    sparql = SPARQLWrapper(SPARQL_ENDPOINT)
+    sparql.setReturnFormat(JSON)
+    sparql.setTimeout(60)
+
+    query = f"""
+    PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+    PREFIX dcat: <http://www.w3.org/ns/dcat#>
+    PREFIX gnc: <http://rdf.genenetwork.org/v1/category/>
+    PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+    PREFIX fabio: <http://purl.org/spar/fabio/>
+    PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+
+    SELECT (COUNT(*) AS ?count)
+    FROM <{DEFAULT_GRAPH}>
+    WHERE {{
+      ?s ?p ?o ;
+         {type_pattern} .
+    }}"""
+
+    sparql.setQuery(query)
+
+    try:
+        result = fetch_with_retry(sparql)
+        if result:
+            count_str = result["results"]["bindings"][0]["count"]["value"]
+            return int(count_str)
+    except Exception as e:
+        print(f"    Warning: Could not get count: {e}")
+
+    return 0
+
+
+def fetch_by_type_pattern(type_pattern: str, page_size: int = 5000) -> List[Dict]:
+    """Fetch triples for a specific type pattern using pagination.
+
+    First gets the count, then calculates pages needed.
+
+    Args:
+        type_pattern: The type pattern to query
+        page_size: Number of triples per page
+    Returns:
+
+        List of SPARQL binding dictionaries
+    """
+    # First, get the count
+    print(f"  Getting count for {type_pattern}...")
+    total_count = fetch_count_for_type(type_pattern)
+
+    if total_count == 0:
+        print(f"  No data found for {type_pattern}")
+        return []
+
+    num_pages = (total_count + page_size - 1) // page_size  # Ceiling division
+    print(f"  Total: {total_count} triples, fetching {num_pages} pages...")
+
+    all_bindings = []
+    sparql = SPARQLWrapper(SPARQL_ENDPOINT)
+    sparql.setReturnFormat(JSON)
+    sparql.setTimeout(120000)
+
+    for page in range(num_pages):
+        offset = page * page_size
+
+        query = f"""
+        PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+        PREFIX dcat: <http://www.w3.org/ns/dcat#>
+        PREFIX gnc: <http://rdf.genenetwork.org/v1/category/>
+        PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+        PREFIX fabio: <http://purl.org/spar/fabio/>
+        PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+
+        SELECT ?s ?p ?o
+        FROM <{DEFAULT_GRAPH}>
+        WHERE {{
+          ?s ?p ?o ;
+             {type_pattern} .
+        }}
+        LIMIT {page_size}
+        OFFSET {offset}"""
+
+        sparql.setQuery(query)
+
+        try:
+            result = fetch_with_retry(sparql)
+            if result is None:
+                break
+
+            bindings = result.get("results", {}).get("bindings", [])
+            all_bindings.extend(bindings)
+
+            # Progress indicator
+            if (page + 1) % 5 == 0 or page == num_pages - 1:
+                print(
+                    f"    Fetched page {page + 1}/{num_pages} ({len(all_bindings)}/{total_count} triples) :: {type_pattern_to_filename(type_pattern)}"
+                )
+
+            # Small delay between pages
+            if page < num_pages - 1:
+                time.sleep(0.3)
+
+        except Exception as e:
+            print(f"    ERROR on page {page + 1}: {e}")
+            break
+
+    return all_bindings
+
+
+def group_by_subject(bindings: List[Dict]) -> Dict[str, List[Tuple[str, str]]]:
+    """Group triples by subject URI."""
+    grouped = defaultdict(list)
+    for b in bindings:
+        subj = b["s"]["value"]
+        pred = b["p"]["value"]
+        obj = b["o"]["value"]
+        grouped[subj].append((pred, obj))
+    return grouped
+
+
+def build_sentences(grouped: Dict[str, List[Tuple[str, str]]]) -> List[str]:
+    """Build list of sentence groups, one per subject."""
+    result = []
+    for subj_uri, triples in grouped.items():
+        subject = uri_to_prefixed(subj_uri)
+        sentences = [triple_to_sentence(subject, pred, obj) for pred, obj in triples]
+        result.append(" ".join(sentences))
+    return result
+
+
+def process_type_query(
+    type_pattern: str, output_dir: str, page_size: int = 5000
+) -> Tuple[str, int, bool]:
+    """Process a single type query and write results to a file.
+
+    Args:
+        type_pattern: The type pattern to query
+        output_dir: Directory to write the output file
+        page_size: Number of triples per page
+
+    Returns:
+        Tuple of (type_pattern, count, success)
+    """
+    filename = type_pattern_to_filename(type_pattern)
+    output_path = os.path.join(output_dir, filename)
+
+    # Add delay at start to stagger requests when running in parallel
+    time.sleep(random.uniform(0.5, 2.0))
+
+    try:
+        print(f"Fetching {type_pattern}...")
+        bindings = fetch_by_type_pattern(type_pattern, page_size=page_size)
+
+        if not bindings:
+            with open(output_path, "w") as f:
+                json.dump([], f, indent=2)
+            return type_pattern, 0, True
+
+        print(f"  Processing {len(bindings)} triples for {type_pattern}")
+
+        grouped = group_by_subject(bindings)
+        print(f"  Grouped into {len(grouped)} subjects")
+
+        sentences = build_sentences(grouped)
+
+        with open(output_path, "w") as f:
+            json.dump(sentences, f, indent=2)
+
+        print(f"  Wrote {len(sentences)} sentences to {filename}")
+        return type_pattern, len(sentences), True
+
+    except Exception as e:
+        print(f"  ERROR processing {type_pattern}: {e}")
+        return type_pattern, 0, False
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Fetch RDF triples as natural language, one file per type"
+    )
+    parser.add_argument(
+        "--output-dir",
+        "-o",
+        default="metadata_output",
+        help="Output directory for JSON files (default: metadata_output)",
+    )
+    parser.add_argument(
+        "--workers",
+        "-w",
+        type=int,
+        default=2,
+        help="Number of parallel workers (default: 2, use 1 for sequential)",
+    )
+    parser.add_argument(
+        "--page-size",
+        "-p",
+        type=int,
+        default=5000,
+        help="Number of triples per page (default: 5000)",
+    )
+    args = parser.parse_args()
+
+    os.makedirs(args.output_dir, exist_ok=True)
+    print(f"Output directory: {args.output_dir}")
+    print(f"Processing {len(TYPE_QUERIES)} type queries with {args.workers} workers...")
+    print(f"Page size: {args.page_size}")
+    print()
+
+    results = []
+
+    if args.workers == 1:
+        # Sequential processing - more polite to the server
+        for type_pattern in TYPE_QUERIES:
+            result = process_type_query(type_pattern, args.output_dir, args.page_size)
+            results.append(result)
+            # Delay between types
+            time.sleep(1.0)
+    else:
+        # Parallel processing with limited workers
+        with ThreadPoolExecutor(max_workers=args.workers) as executor:
+            future_to_type = {
+                executor.submit(
+                    process_type_query, tq, args.output_dir, args.page_size
+                ): tq
+                for tq in TYPE_QUERIES
+            }
+
+            for future in as_completed(future_to_type):
+                type_pattern = future_to_type[future]
+                try:
+                    _, count, success = future.result()
+                    results.append((type_pattern, count, success))
+                except Exception as e:
+                    print(f"  UNEXPECTED ERROR for {type_pattern}: {e}")
+                    results.append((type_pattern, 0, False))
+
+    # Summary
+    print()
+    print("=" * 60)
+    print("SUMMARY")
+    print("=" * 60)
+
+    successful = sum(1 for _, _, s in results if s)
+    total_sentences = sum(c for _, c, s in results if s)
+
+    for type_pattern, count, success in sorted(results):
+        status = "✓" if success else "✗"
+        print(f"{status} {type_pattern}: {count} sentences")
+
+    print()
+    print(f"Successfully processed: {successful}/{len(TYPE_QUERIES)}")
+    print(f"Total sentences written: {total_sentences}")
+    print(f"Output files in: {args.output_dir}")
+
+
+if __name__ == "__main__":
+    print("script starting", flush=True)
+    main()
diff --git a/aisearch/scripts/grag_search.py b/aisearch/scripts/grag_search.py
new file mode 100644
index 0000000..29d4c09
--- /dev/null
+++ b/aisearch/scripts/grag_search.py
@@ -0,0 +1,76 @@
+import argparse
+import asyncio
+import os
+import sys
+from typing import Any
+
+import dspy
+import torch
+from gnais.config import Config
+from gnais.search.grag import graph_rag_search
+from gnais.search.prompts import GN_FACT_EXTRACTION_PROMPT, GN_UPDATE_MEMORY_PROMPT
+from mem0 import Memory
+from mem0.configs.base import MemoryConfig
+
+
+def digest(query: str, memory: Memory = None, user_id: str = "default_user"):
+    async def _run():
+        async for chunk in graph_rag_search(
+            query=query,
+            sparql_url=Config.SPARQL_ENDPOINT,
+            memory=memory,
+            user_id=user_id,
+        ):
+            print(chunk, end="", flush=True)
+        print()
+        print("Done")
+
+    return asyncio.run(_run())
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("query", help="Search query")
+    parser.add_argument("--user-id", default="test-user", help="Mem0 user identity")
+    args = parser.parse_args()
+
+    torch.manual_seed(Config.SEED)
+    if torch.cuda.is_available():
+        torch.cuda.manual_seed_all(Config.SEED)
+
+    dspy.configure(lm=Config.DEFAULT_LLM)
+
+    os.environ[f"{Config.MEMORY_MODEL.split('/')[0].upper()}_API_KEY"] = Config.API_KEY
+    memory_config = MemoryConfig(
+        custom_fact_extraction_prompt=GN_FACT_EXTRACTION_PROMPT,
+        custom_update_extraction_prompt=GN_UPDATE_MEMORY_PROMPT,
+        llm={
+            "provider": "litellm",
+            "config": {
+                "model": Config.MEMORY_MODEL,  # DEFAULT_MODEL is set to be bigger
+                "temperature": 0.1,
+                "max_tokens": 2_000,
+                "api_key": Config.API_KEY,
+            },
+        },
+        embedder={
+            "provider": "huggingface",
+            "config": {
+                "model": "Qwen/Qwen3-Embedding-0.6B",
+                "embedding_dims": 1024,
+                "model_kwargs": {
+                    "trust_remote_code": True,
+                    "device": "cuda" if torch.cuda.is_available() else "cpu",
+                },
+            },
+        },
+        vector_store={
+            "provider": "chroma",
+            "config": {
+                "collection_name": "mem0",
+                "path": os.path.join(Config.DB_PATH, "mem0_chroma"),
+            },
+        },
+    )
+    memory = Memory(config=memory_config)
+    print(digest(args.query, memory=memory, user_id=args.user_id))
diff --git a/aisearch/scripts/new_rdf2partial_text.py b/aisearch/scripts/new_rdf2partial_text.py
new file mode 100644
index 0000000..29b99f8
--- /dev/null
+++ b/aisearch/scripts/new_rdf2partial_text.py
@@ -0,0 +1,34 @@
+import json
+import os
+import string
+
+dir = os.getenv("DIR")
+if dir is None:
+    raise ValueError("DIR not specified")
+
+collection = {}
+
+files = [os.path.join(dir, file) for file in os.listdir(dir) if "ttl" in file]
+
+
+def clean(text) -> str:
+    return text.strip(string.punctuation).strip()
+
+
+for file in files:
+    with open(file) as f:
+        lines = f.readlines()
+        for ind, line in enumerate(lines):
+            line = line.strip()
+            if len(line) > 1:
+                contents = line.split(" ")
+                key = clean(contents[0])
+                value = " ".join(clean(content) for content in contents[1:])
+                if key not in collection:
+                    collection[key] = [value]
+                else:
+                    if value not in collection[key]:
+                        collection[key].append(value)
+
+with open(f"{dir}aggr_rdf.txt", "w") as new:
+    new.write(json.dumps(collection))
diff --git a/aisearch/scripts/plot_function.py b/aisearch/scripts/plot_function.py
new file mode 100644
index 0000000..1cdbc43
--- /dev/null
+++ b/aisearch/scripts/plot_function.py
@@ -0,0 +1,137 @@
+"""
+Script to plot results with new metric using local models
+"""
+
+import os
+
+import matplotlib.pyplot as plt
+import pandas as pd
+import seaborn as sns
+
+INPUT_PATH = os.getenv("INPUT_PATH")
+if INPUT_PATH is None:
+    raise FileNotFoundError("Set INPUT_PATH or path to results directory")
+
+OUTPUT1_PATH = os.getenv("OUTPUT1_PATH")
+if OUTPUT1_PATH is None:
+    raise FileNotFoundError(
+        "Set OUTPUT1_PATH or path to save final boxplot for systems"
+    )
+
+OUTPUT2_PATH = os.getenv("OUTPUT2_PATH")
+if OUTPUT2_PATH is None:
+    raise FileNotFoundError("Set OUTPUT2_PATH or path to save final barplot for models")
+
+
+def format_results(path: str) -> tuple[pd.DataFrame]:
+    files = os.listdir(path)
+    full_list = []
+    aggr_list = []
+    for f in files:
+        model = f.strip("_dspy_results.csv")
+        data = pd.read_csv(f"{path}/{f}")
+
+        aggr = pd.DataFrame(
+            data.aggregate("median"), columns=["satisfaction frequency (%)"]
+        )
+        aggr["model"] = model
+        aggr["system"] = list(aggr.index)
+        aggr_list.append(aggr)
+
+        full = data.transpose()
+        full = pd.concat([full[column] for column in full.columns])
+        full_df = pd.DataFrame(full, columns=["satisfaction frequency (%)"])
+        full_df["model"] = model
+        full_df["system"] = list(full.index)
+        full_list.append(full_df)
+
+    full_concat = pd.concat(full_list)
+    full_concat["model_order"] = pd.Categorical(
+        full_concat["model"],
+        categories=[
+            "gptoss_20b",
+            "qwen3_30b",
+            "haiku_4.5",
+            "qwen3_30b_opus_4.8",
+            "haiku_4.5_opus_4.8",
+            "opus_4.8",
+        ],
+        ordered=False,
+    )
+    full_concat["system_order"] = pd.Categorical(
+        full_concat["system"],
+        categories=["base", "rag", "graph rag", "agent", "combi"],
+        ordered=False,
+    )
+
+    aggr_concat = pd.concat(aggr_list)
+    aggr_concat["model_order"] = pd.Categorical(
+        aggr_concat["model"],
+        categories=[
+            "gptoss_20b",
+            "qwen3_30b",
+            "haiku_4.5",
+            "qwen3_30b_opus_4.8",
+            "haiku_4.5_opus_4.8",
+            "opus_4.8",
+        ],
+        ordered=False,
+    )
+    aggr_concat["system_order"] = pd.Categorical(
+        aggr_concat["system"],
+        categories=["base", "rag", "graph rag", "agent", "combi"],
+        ordered=False,
+    )
+    return full_concat.sort_values(
+        by=["system_order", "model_order"]
+    ), aggr_concat.sort_values(by=["system_order", "model_order"])
+
+
+def plot_results(
+    full_data: pd.DataFrame,
+    aggr_data: pd.DataFrame,
+    output1_path: str,
+    output2_path: str,
+):
+    sns.boxplot(
+        data=full_data, x="system", y="satisfaction frequency (%)", color="#708090"
+    )
+    sns.swarmplot(
+        data=full_data,
+        x="system",
+        y="satisfaction frequency (%)",
+        hue="model",
+        palette="ocean_r",
+    )
+    plt.savefig(output1_path, dpi=1000)
+    plt.show()
+    # heatmap_data = aggr_data.pivot(
+    #     index="model", columns="system", values="satisfaction frequency (%)"
+    # )
+    # heatmap_data = heatmap_data.reindex(
+    #     index=[
+    #         "opus_4.8",
+    #         "haiku_4.5_opus_4.8",
+    #         "qwen3_30b_opus_4.8",
+    #         "haiku_4.5",
+    #         "qwen3_30b",
+    #         "gptoss_20b",
+    #     ],
+    #     columns=["base", "rag", "graph rag", "agent", "hybrid"],
+    # )
+    sns.barplot(
+        full_data,
+        x="system",
+        y="satisfaction frequency (%)",
+        hue="model",
+        palette="ocean_r",
+    )
+    plt.suptitle("Model performance with customized metric", fontsize=10)
+    plt.tight_layout()
+    plt.savefig(output2_path, dpi=1000)
+    plt.show()
+
+
+if __name__ == "__main__":
+    full_df, aggr_df = format_results(INPUT_PATH)
+    plot_results(full_df, aggr_df, OUTPUT1_PATH, OUTPUT2_PATH)
diff --git a/aisearch/scripts/rag_search.py b/aisearch/scripts/rag_search.py
new file mode 100644
index 0000000..20d9863
--- /dev/null
+++ b/aisearch/scripts/rag_search.py
@@ -0,0 +1,111 @@
+"""This is the main module of the package"""
+
+import argparse
+import asyncio
+import os
+from functools import partial
+
+import dspy
+import torch
+from mem0 import Memory
+from mem0.configs.base import MemoryConfig
+
+from gnais.config import Config
+from gnais.search.classification import classify_search, extract_keywords
+from gnais.search.corpus import (
+    create_ensemble_retriever,
+    get_chroma_db,
+    get_docs,
+    init_chroma_db,
+)
+from gnais.search.prompts import GN_FACT_EXTRACTION_PROMPT, GN_UPDATE_MEMORY_PROMPT
+from gnais.search.rag import rag_search
+
+
+def _digest(
+    query: str, memory: Memory, user_id: str, corpus_path: str, db_path: str
+) -> str:
+    """Run the full RAG pipeline for a single query and return the answer.
+
+    This is a convenience wrapper around the same logic used by the CLI
+    script, packaged as a plain synchronous function.
+    """
+
+    async def _run() -> str:
+
+        docs = get_docs(corpus_path)
+        chroma_db = get_chroma_db(embed_model="Qwen/Qwen3-Embedding-0.6B")
+        decision = classify_search(query).get("decision")
+        retriever = create_ensemble_retriever(
+            chroma_db=chroma_db,
+            docs=docs,
+            keyword_weight=0.7 if decision == "keyword" else 0.5,
+        )
+
+        parts = []
+        async for chunk in rag_search(
+            query=(
+                extract_keywords(query).get("keywords")
+                if decision == "keyword"
+                else query
+            ),
+            retriever=retriever,
+            memory=memory,
+            user_id=user_id,
+        ):
+            parts.append(str(chunk))
+        return "".join(parts)
+
+    return asyncio.run(_run())
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("query", help="Search query")
+    parser.add_argument("--user-id", default="test-user", help="Mem0 user identity")
+    args = parser.parse_args()
+
+    torch.manual_seed(Config.SEED)
+    if torch.cuda.is_available():
+        torch.cuda.manual_seed_all(Config.SEED)
+
+    dspy.configure(lm=Config.DEFAULT_LLM)
+
+    os.environ[f"{Config.MEMORY_MODEL.split('/')[0].upper()}_API_KEY"] = Config.API_KEY
+    memory_config = MemoryConfig(
+        custom_fact_extraction_prompt=GN_FACT_EXTRACTION_PROMPT,
+        custom_update_extraction_prompt=GN_UPDATE_MEMORY_PROMPT,
+        llm={
+            "provider": "litellm",
+            "config": {
+                "model": Config.MEMORY_MODEL,
+                "temperature": 0.1,
+                "max_tokens": 2_000,
+                "api_key": Config.API_KEY,
+            },
+        },
+        embedder={
+            "provider": "huggingface",
+            "config": {
+                "model": "Qwen/Qwen3-Embedding-0.6B",
+                "embedding_dims": 1024,
+                "model_kwargs": {
+                    "trust_remote_code": True,
+                    "device": "cuda" if torch.cuda.is_available() else "cpu",
+                },
+            },
+        },
+        vector_store={
+            "provider": "chroma",
+            "config": {
+                "collection_name": "mem0",
+                "host": "localhost",
+                "port": 8001,
+            },
+        },
+    )
+    memory = Memory(config=memory_config)
+    digest = partial(
+        _digest, memory=memory, corpus_path=Config.CORPUS_PATH, db_path=Config.DB_PATH
+    )
+    print(digest(query=args.query, user_id=args.user_id))
diff --git a/aisearch/scripts/ragent_search.py b/aisearch/scripts/ragent_search.py
new file mode 100644
index 0000000..6f11b8c
--- /dev/null
+++ b/aisearch/scripts/ragent_search.py
@@ -0,0 +1,170 @@
+"""Main module of hybrid search for GeneNetwork"""
+
+import argparse
+import asyncio
+import os
+
+import dspy
+import torch
+from gnais.config import Config
+from gnais.search.prompts import GN_FACT_EXTRACTION_PROMPT, GN_UPDATE_MEMORY_PROMPT
+from gnais.search.ragent import hybrid_search
+from mem0 import Memory
+from mem0.configs.base import MemoryConfig
+from rich.columns import Columns
+from rich.console import Console
+from rich.live import Live
+from rich.panel import Panel
+from rich.text import Text
+
+
+class _HybridDisplay:
+    """Terminal UI with three live panels for RAG / GRAG / Agent."""
+
+    _COLORS = {
+        "rag": "cyan",
+        "grag": "green",
+        "agent": "yellow",
+        "synthesis": "magenta",
+    }
+
+    def __init__(self):
+        self._texts = {
+            "rag": Text("", end=""),
+            "grag": Text("", end=""),
+            "agent": Text("", end=""),
+            "synthesis": Text("", end=""),
+        }
+        self._status = {
+            "rag": "⏳ waiting",
+            "grag": "⏳ waiting",
+            "agent": "⏳ waiting",
+            "synthesis": "⏳ waiting",
+        }
+        self._live = Live(self._render(), refresh_per_second=10)
+
+    def _render(self):
+        panels = []
+        for src in ("rag", "grag", "agent", "synthesis"):
+            color = self._COLORS[src]
+            title = f"[bold {color}]{src.upper()}[/] — {self._status[src]}"
+            panels.append(
+                Panel(
+                    self._texts[src],
+                    title=title,
+                    border_style=color,
+                    expand=True,
+                )
+            )
+        return Columns(panels, equal=True)
+
+    def start(self):
+        self._live.start()
+
+    def stop(self):
+        self._live.stop()
+
+    def append(self, source: str, text: str):
+        if source in self._texts:
+            self._texts[source].append(text)
+        self._live.update(self._render())
+
+    def set_status(self, source: str, status: str):
+        if source in self._status:
+            self._status[source] = status
+        self._live.update(self._render())
+
+
+def digest(query: str, user_id: str, memory: Memory = None):
+    async def _run():
+        display = _HybridDisplay()
+        display.start()
+
+        final_html = None
+        try:
+            async for event in hybrid_search(query, user_id=user_id, memory=memory):
+                source = event["source"]
+                kind = event["kind"]
+                content = event["content"]
+
+                if source == "hybrid" and kind == "final":
+                    final_html = content
+                    break
+
+                if source == "synthesis" and kind == "chunk":
+                    display.set_status(source, "🔄 synthesizing")
+                    display.append(source, content)
+                    continue
+
+                if kind == "chunk":
+                    display.append(source, content)
+                elif kind == "final":
+                    display.set_status(source, "✅ complete")
+                    display.append(source, content)
+                elif kind == "error":
+                    display.set_status(source, f"❌ error")
+                    display.append(source, f"\n[ERROR] {content}\n")
+                elif kind == "done":
+                    display.set_status(source, "✅ done")
+        finally:
+            display.stop()
+
+        console = Console()
+        console.print()
+        if final_html:
+            console.rule("[bold magenta]HYBRID SYNTHESIS[/bold magenta]")
+            console.print(final_html)
+        else:
+            console.print("[bold red]No final synthesis received.[/bold red]")
+        console.print()
+        return final_html
+
+    return asyncio.run(_run())
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--user-id", default="test-user", help="Mem0 user identity")
+    parser.add_argument("query", help="Search query")
+    args = parser.parse_args()
+
+    torch.manual_seed(Config.SEED)
+    if torch.cuda.is_available():
+        torch.cuda.manual_seed_all(Config.SEED)
+
+    dspy.configure(lm=Config.DEFAULT_LLM)
+    os.environ[f"{Config.MEMORY_MODEL.split('/')[0].upper()}_API_KEY"] = Config.API_KEY
+    memory_config = MemoryConfig(
+        custom_fact_extraction_prompt=GN_FACT_EXTRACTION_PROMPT,
+        custom_update_extraction_prompt=GN_UPDATE_MEMORY_PROMPT,
+        llm={
+            "provider": "litellm",
+            "config": {
+                "model": Config.MEMORY_MODEL,
+                "temperature": 0.1,
+                "max_tokens": 2_000,
+                "api_key": Config.API_KEY,
+            },
+        },
+        embedder={
+            "provider": "huggingface",
+            "config": {
+                "model": "Qwen/Qwen3-Embedding-0.6B",
+                "embedding_dims": 1024,
+                "model_kwargs": {
+                    "trust_remote_code": True,
+                    "device": "cuda" if torch.cuda.is_available() else "cpu",
+                },
+            },
+        },
+        vector_store={
+            "provider": "chroma",
+            "config": {
+                "collection_name": "mem0",
+                "path": os.path.join(Config.DB_PATH, "mem0_chroma"),
+            },
+        },
+    )
+    memory = Memory(config=memory_config)
+
+    digest(args.query, args.user_id, memory=memory)
diff --git a/aisearch/scripts/rdf2partial_text.py b/aisearch/scripts/rdf2partial_text.py
new file mode 100644
index 0000000..632b5ea
--- /dev/null
+++ b/aisearch/scripts/rdf2partial_text.py
@@ -0,0 +1,70 @@
+import json
+import os
+import string
+from copy import copy
+
+dir = os.getenv("DIR")
+if dir is None:
+    raise ValueError("DIR not specified")
+
+collection = {}
+
+files = [os.path.join(dir, file) for file in os.listdir(dir) if "ttl" in file]
+
+
+def clean(text) -> str:
+    return text.strip(string.punctuation).strip()
+
+
+for file in files:
+    with open(file) as f:
+        lines = f.readlines()
+        mem = ""
+        for ind, line in enumerate(lines):
+            if (not line.startswith("@")) and len(line) > 1:
+                if line.startswith("<"):
+                    line = line.strip()
+                    contents = line.split(" ")
+                    key = (
+                        clean(contents[0].split("/")[-1])
+                        if "/" in contents[0]
+                        else clean(contents[0].split(":")[-1])
+                    )
+                    mem = copy(key)
+                    value = " ".join(
+                        (
+                            clean(content.split("/")[-1])
+                            if "/" in content
+                            else clean(content.split(":")[-1])
+                        )
+                        for content in contents[1:]
+                    )
+                    key = clean(key)
+                    value = clean(value)
+                    if key not in collection:
+                        collection[key] = [value]
+                    else:
+                        if value not in collection[key]:
+                            collection[key].append(value)
+                else:
+                    line = line.strip()
+                    key = copy(mem)
+                    contents = line.split(" ")
+                    value = " ".join(
+                        (
+                            clean(content.split("/")[-1])
+                            if "/" in content
+                            else clean(content.split(":")[-1])
+                        )
+                        for content in contents
+                    )
+                    key = clean(key)
+                    value = clean(value)
+                    if key not in collection:
+                        collection[key] = [value]
+                    else:
+                        if value not in collection[key]:
+                            collection[key].append(value)
+
+with open(f"{dir}aggr_rdf.txt", "w") as new:
+    new.write(json.dumps(collection))
diff --git a/aisearch/scripts/simple_evaluate.py b/aisearch/scripts/simple_evaluate.py
new file mode 100644
index 0000000..9fc79b1
--- /dev/null
+++ b/aisearch/scripts/simple_evaluate.py
@@ -0,0 +1,132 @@
+"""Script for performance evaluation of GN AI systems using mere averaging"""
+
+import os
+import time
+from typing import Any
+
+import dspy
+import numpy as np
+import pandas as pd
+import torch
+from gnais.config import Config
+from gnais.evaluation.utils import (
+    agent_digest,
+    get_dataset,
+    graph_rag_digest,
+    hybrid_digest,
+    mark,
+    rag_digest,
+)
+
+
+def make_program(
+    search_func: Any,
+) -> dspy.predict.react.ReAct:
+
+    search = dspy.Tool(
+        name="search",
+        desc="Answer the question asked by the user.",
+        args={
+            "query": {
+                "type": "string",
+                "desc": "question asked by the user",
+            },
+        },
+        func=search_func,
+    )
+
+    class Signature(dspy.Signature):
+        query: str = dspy.InputField(desc="User question")
+        answer: str = dspy.OutputField(
+            desc="Answer to user's query with details and explanations"
+        )
+
+    # Wrap hybrid search into a DSPy module that is compatible with DSPy evaluation
+    program = dspy.ReAct(
+        signature=Signature, tools=[search], max_iters=2
+    )  # limit iterations
+
+    return program
+
+
+def run_eval(
+    runner: Any, evaluation_set: list[dspy.Example], judge_llm: dspy.LM
+) -> dict[str, float]:
+    precisions, recalls, f1s, speeds, n_tokens = [], [], [], [], []
+    for example in evaluation_set:
+        query = example.get("query")
+        true_response = example.get("answer")
+
+        start = time.time()
+        result = runner(query=query)
+        end = time.time()
+        generated_response = result.get("answer")
+
+        with dspy.context(lm=judge_llm):
+            precision, recall, f1 = mark(query, generated_response, true_response)
+        precisions.append(precision)
+        recalls.append(recall)
+        f1s.append(f1)
+        speeds.append(end - start)
+
+    metrics = {
+        "precision": np.mean(precisions).item(),
+        "recall": np.mean(recalls).item(),
+        "f1": np.mean(f1s).item(),
+        "speed": np.mean(speeds).item(),
+    }
+    return metrics
+
+
+if __name__ == "__main__":
+    DATASET_PATH = os.getenv("DATASET_PATH")
+    if DATASET_PATH is None:
+        raise FileNotFoundError("DATASET_PATH must be set for evaluation")
+
+    OUTPUT_PATH = os.getenv("OUTPUT_PATH")
+    if OUTPUT_PATH is None:
+        raise FileNotFoundError("OUTPUT_PATH must be set for evaluation")
+
+    N_ITERATIONS = int(os.getenv("N_ITERATIONS"))
+    if N_ITERATIONS is None:
+        raise ValueError("N_ITERATIONS must be set for evaluation")
+
+    JUDGE_MODEL = os.getenv("JUDGE_MODEL")
+    if JUDGE_MODEL is None:
+        raise ValueError("JUDGE MODEL must be set for evaluation")
+
+    torch.manual_seed(Config.SEED)
+    if torch.cuda.is_available():
+        torch.cuda.manual_seed_all(Config.SEED)
+
+    dspy.configure(lm=Config.DEFAULT_LLM)
+
+    judge_llm = dspy.LM(
+        model=JUDGE_MODEL,
+        api_key=Config.API_KEY,
+        max_tokens=1_000,
+        temperature=1,
+        cache=False,
+        verbose=False,
+    )
+
+    evaluation_set = get_dataset(DATASET_PATH)
+    collection = {}
+
+    # Run evaluation set with LLM only
+    base = dspy.ChainOfThought("query -> answer: str")
+    for n in range(N_ITERATIONS):
+        base_metrics = run_eval(base, evaluation_set, judge_llm)
+        collection[f"base_{n}"] = base_metrics
+
+    # Run evaluation set with GN systems
+    for system in [rag_digest, graph_rag_digest, agent_digest, hybrid_digest]:
+        system_name = " ".join(system.__name__.split("_")[:-1])
+        print(f"Running evaluation for {system_name}")
+        for n in range(N_ITERATIONS):
+            print(f"Iteration {n+1}")
+            system_metrics = run_eval(make_program(system), evaluation_set, judge_llm)
+            collection[f"{system_name} {n}"] = system_metrics
+        print(f"Evaluation completed for {system_name}")
+    final = pd.DataFrame(collection)
+    final.to_csv(OUTPUT_PATH)
diff --git a/aisearch/src/gnais/__init__.py b/aisearch/src/gnais/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/aisearch/src/gnais/__init__.py
diff --git a/aisearch/src/gnais/config.py b/aisearch/src/gnais/config.py
new file mode 100644
index 0000000..6c1573c
--- /dev/null
+++ b/aisearch/src/gnais/config.py
@@ -0,0 +1,120 @@
+__all__ = ("Config",)
+import os
+
+import dspy
+from dotenv import load_dotenv
+
+load_dotenv()
+
+
+class Config:
+    CORPUS_PATH = os.environ.get("CORPUS_PATH")
+    if CORPUS_PATH is None:
+        raise RuntimeError("CORPUS_PATH is not set")
+
+    DB_PATH = os.environ.get("DB_PATH")
+    if DB_PATH is None:
+        raise RuntimeError("DB_PATH is not set")
+
+    MEM0_PATH = os.path.join(DB_PATH, "mem0_chroma")
+
+    SEED = int(os.environ.get("SEED"))
+    if SEED is None:
+        raise RuntimeError("SEED is not set")
+
+    DEFAULT_MODEL = os.environ.get("DEFAULT_MODEL")
+    if DEFAULT_MODEL is None:
+        raise RuntimeError("DEFAULT_MODEL is not set")
+
+    ALTERNATIVE_MODEL = os.environ.get("ALTERNATIVE_MODEL")
+    if ALTERNATIVE_MODEL is None:
+        raise RuntimeError("ALTERNATIVE_MODEL is not set")
+
+    MEMORY_MODEL = os.environ.get("MEMORY_MODEL")
+    if MEMORY_MODEL is None:
+        raise RuntimeError("MEMORY_MODEL is not set")
+
+    MODEL_TYPE = int(os.environ.get("MODEL_TYPE"))
+    if MODEL_TYPE is None:
+        raise RuntimeError("MODEL_TYPE is not set")
+    if MODEL_TYPE not in (
+        1,
+        2,
+        3,
+    ):
+        raise ValueError("MODEL_TYPE must be 1, 2 or 3")
+
+    API_KEY = os.environ.get("API_KEY")
+    if MODEL_TYPE and API_KEY is None:
+        raise RuntimeError("API_KEY is not set")
+
+    SPARQL_ENDPOINT = os.environ.get("SPARQL_ENDPOINT")
+    if SPARQL_ENDPOINT is None:
+        raise RuntimeError("SPARQL_ENDPOINT is not set")
+
+    AUTH_SERVER_URL = os.environ.get("AUTH_SERVER_URL")
+    if AUTH_SERVER_URL is None:
+        raise RuntimeError("AUTH_SERVER_URL is not set")
+
+    SECRET_KEY = os.environ.get("SECRET_KEY")
+    if SECRET_KEY is None:
+        raise RuntimeError("SECRET_KEY is not set")
+
+    PORT = os.environ.get("PORT")
+    if PORT is None:
+        raise RuntimeError("PORT for local model is not set")
+
+    if MODEL_TYPE == 1:  # only frontier models
+        DEFAULT_LLM = dspy.LM(
+            model=DEFAULT_MODEL,
+            api_key=API_KEY,
+            max_tokens=50_000,
+            temperature=1,
+            cache=False,  # essential for evaluation; switch on for local deploy
+            verbose=False,
+        )
+        ALTERNATIVE_LLM = dspy.LM(
+            model=ALTERNATIVE_MODEL,
+            api_key=API_KEY,
+            max_tokens=50_000,
+            temperature=1,
+            cache=False,  # essential for evaluation; switch on for local deploy
+            verbose=False,
+        )
+    elif MODEL_TYPE == 2:  # only local models
+        DEFAULT_LLM = dspy.LM(
+            model=f"ollama_chat/{DEFAULT_MODEL}",
+            api_key="local",
+            api_base=f"http://localhost:{PORT}",
+            max_tokens=50_000,
+            temperature=1,
+            cache=False,  # essential for evaluation; switch on for local deploy
+            verbose=False,
+        )
+        ALTERNATIVE_LLM = dspy.LM(
+            model=f"ollama_chat/{ALTERNATIVE_MODEL}",
+            api_key="local",
+            api_base=f"http://localhost:{PORT}",
+            max_tokens=50_000,
+            temperature=1,
+            cache=False,  # essential for evaluation; switch on for local deploy
+            verbose=False,
+        )
+    elif MODEL_TYPE == 3:  # smart combination of frontier and local models
+        DEFAULT_LLM = dspy.LM(
+            model=DEFAULT_MODEL,
+            api_key=API_KEY,
+            max_tokens=50_000,
+            temperature=1,
+            cache=False,  # essential for evaluation; switch on for local deploy
+            verbose=False,
+        )
+        ALTERNATIVE_LLM = dspy.LM(
+            model=f"ollama_chat/{ALTERNATIVE_MODEL}",
+            api_key="local",
+            api_base=f"http://localhost:{PORT}",
+            max_tokens=50_000,
+            temperature=1,
+            cache=False,  # essential for evaluation; switch on for local deploy
+            verbose=False,
+        )
diff --git a/aisearch/src/gnais/evaluation/__init__.py b/aisearch/src/gnais/evaluation/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/aisearch/src/gnais/evaluation/__init__.py
diff --git a/aisearch/src/gnais/evaluation/utils.py b/aisearch/src/gnais/evaluation/utils.py
new file mode 100644
index 0000000..8e9e335
--- /dev/null
+++ b/aisearch/src/gnais/evaluation/utils.py
@@ -0,0 +1,159 @@
+"""Common utilities for evaluation"""
+
+import asyncio
+import uuid
+from concurrent.futures import ThreadPoolExecutor
+from typing import Any
+
+import dspy
+import pandas as pd
+
+from gnais.config import Config
+from gnais.search.agent import agent_search
+from gnais.search.classification import classify_search, extract_keywords
+from gnais.search.corpus import create_ensemble_retriever, get_chroma_db, get_docs
+from gnais.search.grag import graph_rag_search
+from gnais.search.rag import rag_search
+from gnais.search.ragent import hybrid_search
+
+
+def get_dataset(
+    dataset_path: str,
+    column_names: list[str] = ["query", "answer"],
+) -> list[dspy.Example]:
+    data = pd.read_csv(
+        dataset_path,
+        usecols=column_names,
+    )
+    data_dicts = data[column_names].to_dict(orient="records")
+    evaluation_set = [
+        dspy.Example({name: x[name] for name in column_names}).with_inputs(
+            column_names[0]
+        )
+        for x in data_dicts
+    ]
+    return evaluation_set
+
+
+def mark(
+    query: str,
+    generated_response: str,
+    true_response: str,
+) -> tuple[float]:
+
+    assess = dspy.ChainOfThought("question -> answer: int")
+    context = f"""Your task is to compare elements of response in the answer generated by AI to the ones in the true answer.
+    You must compare only informations that are strictly relevant to the query.
+    User query: {query}\nAI answer: {generated_response}\nTrue answer: {true_response}"""
+
+    tp_question = f"{context}\nThe objective is to compute the number of true positives by comparing the generated answer and true answer. True positives are elements of response to the query that are mentioned in the generated answer and somewhat highlighted in the true answer as well. How many true positives do you count?"
+    tp_score = assess(question=tp_question).get("answer")
+
+    fp_question = f"{context}\nThe objective is to compute the number of false positives by comparing the generated answer and true answer. False positives are explicit and highly relevant elements of response to the query that are present in the generated answer and are not supported in any ways by the true answer. They should not be covered by a reformulation of the true answer. How many false positives do you count?"
+    fp_score = assess(question=fp_question).get("answer")
+
+    fn_question = f"{context}\nThe objective is to compute the number of false negatives by comparing the generated answer and true answer. False negatives are elements of response to the query that feature in the true answer and are not supported by the generated answer. How many false negatives do you count?"
+    fn_score = assess(question=fn_question).get("answer")
+
+    print(
+        f"Question: {query}\nTrue answer: {true_response}\nAI answer: {generated_response}"
+    )
+    print(f"TP: {tp_score}\nFP: {fp_score}\nFN: {fn_score}")
+
+    tiny_term = 1e-10
+    precision = tp_score / (tp_score + fp_score + tiny_term)
+    recall = tp_score / (tp_score + fn_score + tiny_term)
+    f1 = (2 * tp_score) / ((2 * tp_score) + fp_score + fn_score + tiny_term)
+
+    return precision, recall, f1
+
+
+_TOOL_EXECUTOR = ThreadPoolExecutor(max_workers=1, thread_name_prefix="search")
+
+
+def _run_async(async_fn, *args, **kwargs):
+    """Run async function in a separate thread with a fresh event loop"""
+
+    def _worker():
+        return asyncio.run(async_fn(*args, **kwargs))
+
+    return _TOOL_EXECUTOR.submit(_worker).result()
+
+
+def rag_digest(query: str, memory: Any = None) -> str:
+    async def _run() -> str:
+        docs = get_docs(Config.CORPUS_PATH)
+        chroma_db = get_chroma_db(embed_model="Qwen/Qwen3-Embedding-0.6B")
+        decision = classify_search(query).get("decision")
+        retriever = create_ensemble_retriever(
+            chroma_db=chroma_db,
+            docs=docs,
+            keyword_weight=0.7 if decision == "keyword" else 0.5,
+        )
+
+        parts = []
+        async for chunk in rag_search(
+            query=(
+                extract_keywords(query).get("keywords")
+                if decision == "keyword"
+                else query
+            ),
+            retriever=retriever,
+            memory=memory,
+            user_id=str(uuid.uuid4()),
+        ):
+            parts.append(str(chunk))
+        return "".join(parts)
+
+    return _run_async(_run)
+
+
+def sparql_digest(
+    query: str, handler: Any, memory: Any = None, user_id: str = "default_user"
+) -> str:
+    async def _run():
+        output = ""
+        async for chunk in handler(
+            query=query,
+            sparql_url=Config.SPARQL_ENDPOINT,
+            memory=memory,
+            user_id=user_id,
+        ):
+            if isinstance(chunk, dict) and "final" in chunk:
+                final = chunk["final"]
+                output = final
+                print(final, end="", flush=True)
+            else:
+                output += str(chunk)
+                print(chunk, end="", flush=True)
+        print()
+        return output
+
+    return _run_async(_run)
+
+
+def graph_rag_digest(query: str, memory: Any = None) -> str:
+    return sparql_digest(
+        query, graph_rag_search, memory=memory, user_id=str(uuid.uuid4())
+    )
+
+
+def agent_digest(query: str, memory: Any = None) -> str:
+    return sparql_digest(query, agent_search, memory=memory, user_id=str(uuid.uuid4()))
+
+
+def hybrid_digest(query: str, memory: Any = None) -> str:
+    async def _run():
+        final_html = None
+        async for event in hybrid_search(
+            query, memory=memory, user_id=str(uuid.uuid4())
+        ):
+            source = event["source"]
+            kind = event["kind"]
+            content = event["content"]
+            if source == "hybrid" and kind == "final":
+                final_html = content
+                break
+        return final_html
+
+    return _run_async(_run)
diff --git a/aisearch/src/gnais/search/__init__.py b/aisearch/src/gnais/search/__init__.py
new file mode 100644
index 0000000..5d2dccd
--- /dev/null
+++ b/aisearch/src/gnais/search/__init__.py
@@ -0,0 +1,5 @@
+from gnais.search.agent import agent_search
+from gnais.search.classification import classify_search, extract_keywords
+from gnais.search.grag import graph_rag_search
+from gnais.search.rag import rag_search
+from gnais.search.ragent import hybrid_search
diff --git a/aisearch/src/gnais/search/agent.py b/aisearch/src/gnais/search/agent.py
new file mode 100644
index 0000000..03b6dc5
--- /dev/null
+++ b/aisearch/src/gnais/search/agent.py
@@ -0,0 +1,88 @@
+"""Agent with SPARQL tool calling for AI search in GeneNetwork"""
+
+from typing import Any
+
+import dspy
+from gnais.search.prompts import GENERAL_SYSTEM_PROMPT
+from gnais.search.tools import (
+    MemoryTools,
+    check_link,
+    make_sparql_fetch_tool,
+    route_model,
+    with_memory,
+)
+
+
+class AgentSig(dspy.Signature):
+    query: str = dspy.InputField()
+    chat_history: list = dspy.InputField(desc="Previous conversation context")
+    solution: str = dspy.OutputField(
+        desc="""Return the answer as valid HTML only. Use the chat
+history as context.  Use only safe body tags such as <p>, <ul>, <ol>,
+<li>, <a>, <strong>, <em>, <code>, <pre>, and <br>.  Do not use
+Markdown. Do not wrap the answer in ```html fences.
+
+Link correctness is mandatory. Every <a href="..."> URL MUST come directly
+from a SPARQL tool result returned by the configured Virtuoso endpoint.
+Never invent, infer, concatenate, normalize, rewrite, shorten, or guess URLs.
+Never create links from literals, labels, names, descriptions, IDs, QNames,
+CURIEs, blank nodes, or user text. Only bind and use URI/IRI values returned
+by SPARQL variables.
+
+Before emitting any hyperlink, verify that the href value is exactly one of
+the URI/IRI strings present in the latest relevant SPARQL result set. If no
+verified URI/IRI is available, omit the hyperlink and render plain text instead.
+Broken links are worse than no links.
+
+The final answer must include detailed reasoning where useful, followed by a
+clear final answer, but all content must remain valid HTML."""
+    )
+
+
+def _make_agent_stream(sparql_url: str):
+    routed = route_model()(dspy.Predict(AgentSig))
+    chosen_model = routed.choose_model()
+    chosen_lm = routed.options[chosen_model]
+    print(f"Choice made: {chosen_model} for Agent")
+    tools = [
+        make_sparql_fetch_tool(sparql_url, chosen_lm),
+        check_link,
+    ]
+    react = dspy.ReAct(
+        signature=AgentSig,
+        tools=tools,
+        max_iters=7,
+    )
+    react.set_lm(chosen_lm)
+    return dspy.streamify(
+        react,
+        stream_listeners=[
+            dspy.streaming.StreamListener(
+                signature_field_name="next_thought",
+                allow_reuse=True,
+            ),
+            dspy.streaming.StreamListener(signature_field_name="solution"),
+        ],
+        include_final_prediction_in_output_stream=True,
+    )
+
+
+@with_memory(memory_type="agent")
+async def agent_search(
+    query: str,
+    sparql_url: str,
+    system_prompt: str = GENERAL_SYSTEM_PROMPT,
+    user_id: str = "default_user",
+    memory=None,
+    chat_history: list = [],
+):
+    yield {"status": "Planning search strategy…"}
+    yield {"status": "Streaming response…"}
+    async for value in _make_agent_stream(sparql_url)(
+        query=f"{system_prompt}\nQuery: {query}",
+        chat_history=chat_history,
+    ):
+        if isinstance(value, dspy.Prediction):
+            yield {"final": value.solution}
+        else:
+            yield getattr(value, "chunk", str(value))
diff --git a/aisearch/src/gnais/search/classification.py b/aisearch/src/gnais/search/classification.py
new file mode 100644
index 0000000..833a16e
--- /dev/null
+++ b/aisearch/src/gnais/search/classification.py
@@ -0,0 +1,56 @@
+from functools import lru_cache
+
+import dspy
+from gnais.search.tools import route_model
+
+
+class Classification(dspy.Signature):
+    input_text: str = dspy.InputField()
+    decision: str = dspy.OutputField(desc='"keyword" or "semantic"')
+
+
+class Extraction(dspy.Signature):
+    input_text: str = dspy.InputField()
+    keywords: str = dspy.OutputField()
+
+
+@lru_cache(maxsize=2048)
+def extract_keywords(query: str) -> str:
+    """Extract list of keywords from query
+
+    Args:
+        query: user query
+
+    Returns:
+        list of keywords
+    """
+    return route_model()(dspy.Predict(Extraction))(
+        input_text=f"""
+You are extremely good at extracting keywords from a search query related to specific entities (traits, markers, etc) in GeneNetwork.
+Produce a list of space separated keywords featured in the query below. Only return that list.
+
+{query}"""
+    )
+
+
+@lru_cache(maxsize=2048)
+def classify_search(query: str) -> str:
+    """Classify user query as keyword search or semantic search
+
+    Args:
+        query: user query
+
+    Returns:
+        type of search for query processing
+    """
+    return route_model()(dspy.Predict(Classification))(
+        input_text=f"""
+You are an experienced search classifier.
+You can accurately tell from a query if a keyword search or semantic search is more appropriate to provide satisfactory answers to the user.
+A keyword search is appropriate when specific entities feature in the query (i.e trait id, marker code, etc.).
+A semantic search is better when the system needs to understand the meaning of the query and make implicit connections.
+Infer the type of search that should be performed given the query below:
+
+{query}
+"""
+    )
diff --git a/aisearch/src/gnais/search/corpus.py b/aisearch/src/gnais/search/corpus.py
new file mode 100644
index 0000000..420a7a1
--- /dev/null
+++ b/aisearch/src/gnais/search/corpus.py
@@ -0,0 +1,119 @@
+import json
+import warnings
+from functools import lru_cache
+from pathlib import Path
+from typing import Any
+
+import chromadb
+import torch
+from langchain_classic.retrievers import EnsembleRetriever
+from langchain_community.retrievers import BM25Retriever
+from langchain_community.vectorstores import Chroma
+from langchain_huggingface import HuggingFaceEmbeddings
+from tqdm import tqdm
+
+warnings.filterwarnings("ignore")
+
+
+def get_docs(corpus_dir: str) -> dict:
+    corpus_dir = Path(corpus_dir)
+    metadata = []
+    for corpus in corpus_dir.iterdir():
+        with open(corpus, "r") as data:
+            metadata.extend(json.load(data))
+    return metadata
+
+
+def _docs_to_tuple(docs: list) -> tuple:
+    """Convert docs list to a hashable tuple for caching."""
+    if not docs:
+        return tuple()
+    if isinstance(docs[0], str):
+        return tuple(docs)
+    return tuple(json.dumps(d, sort_keys=True) for d in docs)
+
+
+@lru_cache(maxsize=4)
+def get_embed_model(model_name: str):
+    """Load (and cache) the embedding model so it is only instantiated once.
+
+    Automatically uses CUDA when a GPU is available, otherwise falls back to
+    CPU.  (sentence-transformers does not accept torch_dtype.)"""
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    return HuggingFaceEmbeddings(
+        model_name=model_name,
+        model_kwargs={"trust_remote_code": True, "device": device},
+        encode_kwargs={"batch_size": 1024},
+    )
+
+
+@lru_cache(maxsize=2048)
+def _cached_bm25_retriever(docs_tuple: tuple, k: int):
+    """Build (and cache) the BM25 retriever.  Tokenizing the corpus is
+    expensive, so we only do it once per unique (docs, k) pair."""
+    return BM25Retriever.from_texts(
+        texts=list(docs_tuple),
+        metadatas=[{"source": f"Document {ind + 1}"} for ind in range(len(docs_tuple))],
+        k=k,
+    )
+
+
+def init_chroma_db(
+    docs: list,
+    embed_model: Any,
+    chroma_host: str = "localhost",
+    chroma_port: int = 8000,
+    chunk_size: int = 1024,
+):
+    client = chromadb.HttpClient(host=chroma_host, port=chroma_port)
+    db = Chroma(
+        client=client,
+        embedding_function=get_embed_model(embed_model),
+    )
+    for i in tqdm(range(0, len(docs), chunk_size)):
+        chunk = docs[i : i + chunk_size]
+        metadatas = [
+            {"source": f"Document {ind + 1}"} for ind in range(i, i + len(chunk))
+        ]
+        db.add_texts(
+            texts=chunk,
+            metadatas=metadatas,
+        )
+    # No db.persist() needed — the server handles persistence
+    return db
+
+
+def get_chroma_db(
+    chroma_host: str = "localhost",
+    chroma_port: int = 8000,
+    embed_model: Any = None,
+):
+    client = chromadb.HttpClient(host=chroma_host, port=chroma_port)
+    return Chroma(
+        client=client,
+        embedding_function=get_embed_model(embed_model),
+    )
+
+
+def create_ensemble_retriever(
+    chroma_db: Any, docs: list, keyword_weight: float = 0.5, **kwargs
+):
+    k = kwargs.get("k") if kwargs.get("k") else 3
+    c = kwargs.get("c") if kwargs.get("c") else 60
+    weights = (
+        kwargs.get("weights")
+        if kwargs.get("weights")
+        else [1 - keyword_weight, keyword_weight]
+    )
+
+    bm25_retriever = _cached_bm25_retriever(_docs_to_tuple(docs), k)
+
+    return EnsembleRetriever(
+        retrievers=[
+            # might need fine-tuning
+            chroma_db.as_retriever(search_kwargs={"k": k}),
+            bm25_retriever,
+        ],
+        weights=weights,
+        c=c,
+    )
diff --git a/aisearch/src/gnais/search/grag.py b/aisearch/src/gnais/search/grag.py
new file mode 100644
index 0000000..cfb6979
--- /dev/null
+++ b/aisearch/src/gnais/search/grag.py
@@ -0,0 +1,160 @@
+"""Module with GraphRAG system for AI search in GeneNetwork"""
+
+import asyncio
+import functools
+
+import dspy
+from gnais.search import tools
+from gnais.search.prompts import GENERAL_SYSTEM_PROMPT, SPARQL_SYSTEM_PROMPT
+from gnais.search.tools import build_schema_hint, route_model, sparql_fetch, with_memory
+
+
+class KeywordSPARQLGenerator(dspy.Signature):
+    """
+    Extract the essential keywords from the user's query, then generate fast,
+    efficient SPARQL SELECT queries that avoid Virtuoso timeouts (504 errors).
+    Use the schema hint to match keywords to exact properties and classes.
+
+    CRITICAL PERFORMANCE RULES (to prevent 504s):
+    1. Always add `LIMIT` - start with `LIMIT 50`, increase only if needed. Never omit `LIMIT`.
+    2. Never use `SELECT *` - list only the variables you actually need.
+    3. Avoid expensive operations: no Cartesian products, no cross joins, no full graph scans.
+    4. Use specific FILTER patterns that leverage indexes:
+       - Prefer `STRSTARTS(?label, "prefix")` over `CONTAINS` or regex.
+       - Avoid `FILTER regex(...)` - it disables indexes.
+       - Use `FILTER(?value = "exact")` or `IN` with small lists.
+    5. Prefer property paths over multiple joins when traversing a chain.
+    6. Use VALUES blocks for small sets of constants instead of UNION or OPTIONAL.
+    7. Avoid ORDER BY on large result sets - if needed, combine with `LIMIT` and a narrow `WHERE` clause.
+    8. Never use nested subqueries unless absolutely necessary; flatten them.
+    9. Use `OPTIONAL` only for truly optional patterns – otherwise, use a simple triple pattern.
+    10. Limit the number of generated queries - output at most 10 (not 20) per request.
+
+    CRITICAL SCHEMA RULES (remember):
+    - Literal properties (e.g., gnt:gene_symbol, dct:title) hold strings/numbers. Use `FILTER(?literal = "value")`, not `?o a <Class>`.
+    - ALWAYS include `FROM <http://rdf.genenetwork.org/v1>` between SELECT and WHERE.
+    - Strain URIs: use gn:set_BXD, gn:set_B6D2F1, gn:set_HMDP — NEVER gn:BXD, gn:B6D2F1, gn:HMDP.
+    - gnc:phenotype is metadata only (abbreviation, description, lab_code, submitter, contributor). Phenotype TRAITS (with mean, locus, lod_score, additive, sequence, has_trait_page) are gnc:phenotype_trait.
+    - Probesets and DNA markers use gnt:chr for chromosome. Genes use gnt:chromosome.
+    - gnt:has_uniprot_id, gnt:has_homologene_id, gnt:has_kegg_id, gnt:has_omim_id, gnt:has_chebi_id, gnt:has_pub_chem_id exist on gnc:probeset, NOT on gnc:gene.
+    - gnt:has_align_id, gnt:has_protein_id, gnt:has_rgd_id, gnc:has_kg_id, gnc:has_unigen_id exist on gnc:gene, NOT on gnc:dna_marker. Markers use skos:prefLabel or skos:altLabel for names.
+    - gnt:locus on phenotype_trait contains chromosome positions (e.g. "1-59904011"), NOT phenotype names. Never FILTER gnt:locus for trait names like "liver_weight". Instead link trait -> has_phenotype -> phenotype and filter gnt:abbreviation.
+    - gnt:symbol is for probesets. gnt:gene_symbol is for genes and strains.
+    - Datasets are dcat:Dataset. They have dct:title, gnt:has_strain, gnt:has_tissue_info, gnt:has_samples, gnt:has_summary, gnt:has_citation, gnt:has_contributors, gnt:has_case_info, gnt:has_platform_info, gnt:has_specifics, gnt:has_data_processing_info, gnt:has_experiment_design, gnt:has_experiment_type, gnt:has_genotype_files.
+    - Do NOT use taxon: for species. Use gn:Mus_musculus, gn:Rattus_norvegicus, gn:Homo_sapiens, etc.
+    - Gene chips / platforms are skos:Concept with skos:inScheme gnc:gene_chip. They have gnt:has_go_tree_value, gnt:has_geo_series_id.
+    - Mapping methods and averaging methods are skos:Concept in gnc:mapping_method and gnc:avg_method schemes.
+    - gnt:has_trait_page gives the URL directly. Never build trait URLs manually.
+    - Object properties (e.g., gnt:has_phenotype_trait) link to resources. Chain with `?s gnt:has_phenotype_trait ?o . ?o rdf:type <Class>`.
+    - Only use properties listed in the provided schema. Do NOT invent new ones.
+    - EVERY query MUST start with the PREFIX declarations from the schema.
+    - To extract information about a specific trait, find the closest matches in SNAPSHOT_OBJECTS and use them to infer the proper RDF subject for the trait.
+    - To get trait page or URL, use the predicate `gnt:has_trait_page` with the trait as subject. Its object correspond to the URL. Never build trait URLs manually.
+    - To get publication for a trait, search for object having the predicate `dcterms:references` and the trait as subject.
+    - To get highest LOD score and corresponding marker and additive effect, use the predicates `gnt:lod_score`, `gnt:locus`, and `gnt:additive`, respectively.
+    - Mean trait measurements can be accessed by looking up object with predicate gnt:mean and the trait as subject.
+    - Trait descriptions can be accessed via the predicate `dcterms:description`
+    - When you have the description of a trait, you should use regex pattern in the object term with the predicate `dcterms:description` to find the corresponding trait (subject)
+    Example of an efficient query:
+
+    ```
+    PREFIX gnt: <http://rdf.genenetwork.org/v1/term/>
+    PREFIX gnc: <http://rdf.genenetwork.org/v1/category/>
+    PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+
+    SELECT ?gene ?symbol FROM <http://rdf.genenetwork.org/v1> WHERE {
+        ?gene rdf:type gnc:gene .
+        ?gene gnt:gene_symbol ?symbol .
+        FILTER(STRSTARTS(?symbol, "Shh"))
+    } LIMIT 50
+    ```
+    """
+
+    original_query: str = dspy.InputField(desc="User query")
+    schema_hint: str = dspy.InputField(desc="GeneNetwork schema from Virtuoso")
+    keywords: str = dspy.OutputField(
+        desc="Comma-separated essential keywords from the query"
+    )
+    sparql_queries: list[str] = dspy.OutputField(
+        desc="Top 15 valid SPARQL SELECT queries with PREFIX declarations."
+    )
+
+
+class GraphRAG(dspy.Signature):
+    original_query: str = dspy.InputField(desc="Query provided")
+    sparql_results: str = dspy.InputField(desc="JSON results from the SPARQL query")
+    chat_history: list = dspy.InputField(desc="History of conversation")
+    feedback: str = dspy.OutputField(
+        desc="System response to the query with detailed answers and the final answer, formatted as valid HTML using tags such as <p>, <ul>, <li>, <a>, <strong>, <em>, and <br>.  Links from sparql results can only be formed by valid IRIs and NOT literals.  Use the orinial query, sparql results and chat history when answering."
+    )
+
+
+def _make_grag_stream():
+    grag = dspy.Predict(GraphRAG)
+    routed_grag = route_model()(grag)
+    chosen_model = routed_grag.choose_model()
+    print(f"Choice made: {chosen_model} for Graph RAG")
+    grag.set_lm(routed_grag.options[chosen_model])
+
+    return dspy.streamify(
+        grag,
+        stream_listeners=[
+            dspy.streaming.StreamListener(
+                signature_field_name="feedback",
+                allow_reuse=True,
+            )
+        ],
+        include_final_prediction_in_output_stream=True,
+    )
+
+
+@with_memory(memory_type="grag")
+async def graph_rag_search(
+    query: str,
+    sparql_url: str,
+    system_prompt: str = GENERAL_SYSTEM_PROMPT,
+    memory=None,
+    user_id: str = "default_user",
+    chat_history: list = [],
+):
+    grag_prompt = f"{system_prompt}\nQuery: {query}"
+    sparql_prompt = f"{SPARQL_SYSTEM_PROMPT}\nQuery: {query}"
+    schema_hint = build_schema_hint(sparql_url)
+    yield {"status": "Generating keywords and SPARQL queries…"}
+
+    kw_sparql = dspy.Predict(KeywordSPARQLGenerator)
+    routed_kw = route_model()(kw_sparql)
+    chosen_model = routed_kw.choose_model()
+    print(f"Choice made: {chosen_model} for KeywordSPARQL")
+    kw_sparql.set_lm(routed_kw.options[chosen_model])
+
+    loop = asyncio.get_running_loop()
+    combined = await loop.run_in_executor(
+        tools.LLM_EXECUTOR,
+        functools.partial(
+            kw_sparql,
+            original_query=sparql_prompt,
+            schema_hint=schema_hint,
+        ),
+    )
+    keywords = getattr(combined, "keywords", "")
+    sparql_queries = getattr(combined, "sparql_queries", [])
+    yield {"status": f"Extracted keywords: {keywords}"}
+    if sparql_queries is None:
+        sparql_queries = []
+    for i, query in enumerate(sparql_queries):
+        yield {"status": f"({i+1}). sparql query: {query}"}
+
+    yield {"status": "Querying knowledge graph…"}
+    sparql_results = await sparql_fetch(sparql_queries, sparql_url)
+
+    yield {"status": "Streaming response…"}
+    async for value in _make_grag_stream()(
+        original_query=grag_prompt,
+        sparql_results=sparql_results,
+        chat_history=chat_history,
+    ):
+        if isinstance(value, dspy.Prediction):
+            yield {"final": value.feedback}
+        else:
+            yield value.chunk
diff --git a/aisearch/src/gnais/search/prompts.py b/aisearch/src/gnais/search/prompts.py
new file mode 100644
index 0000000..cee16ce
--- /dev/null
+++ b/aisearch/src/gnais/search/prompts.py
@@ -0,0 +1,159 @@
+GENERAL_SYSTEM_PROMPT = """You are a scientific literature/document/ontology/knowledge analyst and a world class semantic data engineer. Answer the question/query below using information at your disposal only.
+
+RULES:
+1. Work even with partial data
+2. Do not repeat information.
+3. Do not apologize for query errors.
+4. Ensure response is in line with the query and do not deviate.
+5. Do not include irrelevant information.
+Example: Do not refer to publications when the query is about phenotypes in GeneNetwork.
+
+**Prefix expansion** – When you see any of these prefixes in the results or query, expand them to their full IRI before using in <a href>:
+
+dcat: → http://www.w3.org/ns/dcat#
+dct: → http://purl.org/dc/terms/
+gn: → http://rdf.genenetwork.org/v1/id/
+owl: → http://www.w3.org/2002/07/owl#
+gnc: → http://rdf.genenetwork.org/v1/category/
+gnt: → http://rdf.genenetwork.org/v1/term/has_trait_page
+obo: → http://purl.obolibrary.org/obo/
+bfo: → http://purl.obolibrary.org/obo/BFO_
+sdmx-measure: → http://purl.org/linked-data/sdmx/2009/measure#
+skos: → http://www.w3.org/2004/02/skos/core#
+rdf: → http://www.w3.org/1999/02/22-rdf-syntax-ns#
+rdfs: → http://www.w3.org/2000/01/rdf-schema#
+xsd: → http://www.w3.org/2001/XMLSchema#
+qb: → http://purl.org/linked-data/cube#
+xkos: → http://rdf-vocabulary.ddialliance.org/xkos#
+pubmed: → http://rdf.ncbi.nlm.nih.gov/pubmed/
+schema: → https://schema.org/
+
+**Linking rule**: Only create `<a href>` for IRIs or URLs that are returned from the SPARQL results (after expansion if needed). Do not invent or construct links. If no IRI exists, use plain text.
+
+**Output**: Clean HTML."""
+
+SPARQL_SYSTEM_PROMPT = """You are a world class knowledge engineer and SPARQL optimization specialist operating a SPARQL query system. Your task is to construct SPARQL queries that return results efficiently.
+
+**Guidelines**:
+- Use only schema terms and predicates that exist in the data. Never invent properties or IRIs.
+- Bind real IRIs from the dataset (use full IRIs or known prefixes).
+- Force result‑returning patterns with FILTER, VALUES, and/or LIMIT.
+- Avoid `SELECT *` – list only needed variables.
+- Avoid broad scans, unnecessary OPTIONAL blocks, and expensive regex.
+- Ensure every query returns at least one row (test mentally).
+
+**Prefixes you can use** (expand as needed):
+
+dcat: → http://www.w3.org/ns/dcat#
+dct: → http://purl.org/dc/terms/
+gn: → http://rdf.genenetwork.org/v1/id/
+owl: → http://www.w3.org/2002/07/owl#
+gnc: → http://rdf.genenetwork.org/v1/category/
+gnt: → http://rdf.genenetwork.org/v1/term/has_trait_page
+obo: → http://purl.obolibrary.org/obo/
+bfo: → http://purl.obolibrary.org/obo/BFO_
+sdmx-measure: → http://purl.org/linked-data/sdmx/2009/measure#
+skos: → http://www.w3.org/2004/02/skos/core#
+rdf: → http://www.w3.org/1999/02/22-rdf-syntax-ns#
+rdfs: → http://www.w3.org/2000/01/rdf-schema#
+xsd: → http://www.w3.org/2001/XMLSchema#
+qb: → http://purl.org/linked-data/cube#
+xkos: → http://rdf-vocabulary.ddialliance.org/xkos#
+pubmed: → http://rdf.ncbi.nlm.nih.gov/pubmed/
+schema: → https://schema.org/
+
+**Output**:
+- Valid SPARQL queries only.
+- No explanation, no markdown – just the query text."""
+
+
+GN_FACT_EXTRACTION_PROMPT = """You are a memory extraction system for a genomics and bioinformatics research assistant (GeneNetwork). Your job is to extract ALL meaningful facts, entities, and contextual information from the conversation that would help answer future scientific queries.
+
+Extract facts related to:
+1. **Genes and gene symbols** (e.g., "Gnai3", "Apoe", "Brca1")
+2. **Phenotypes and traits** (e.g., "body weight", "glucose level", "anxiety behavior", trait IDs like "10001")
+3. **Datasets and data resources** (e.g., "BXD", "HMDP", "HCPublish", dataset accession IDs)
+4. **Species and strains** (e.g., "Mus musculus", "Rattus norvegicus", "C57BL/6J", "BXD")
+5. **Genetic markers and genomic locations** (e.g., "rs13478303", "Chr1:45.2 Mb", "D1Mit10")
+6. **QTL / mapping results** (e.g., "LOD score of 5.2", "significant peak on Chr 4", "additive effect of 2.3")
+7. **Publications and PubMed IDs** (e.g., "PMID:12345678")
+8. **Experimental design and methods** (e.g., "GEMMA mapping", "GWAS", "eQTL analysis")
+9. **External databases and tools mentioned** (e.g., "GTEx", "BioGPS", "GeneMANIA", "STRING")
+10. **User preferences or constraints** (e.g., "only mouse data", "exclude BXD", "interested in liver tissue")
+11. **Any scientific conclusions, relationships, or insights** drawn from the data
+
+RULES:
+- Extract facts even if they seem like "general knowledge" — in a genomics context, knowing that "Trees have branches" is irrelevant, but knowing that "Apoe is associated with lipid metabolism" or "BXD is a mouse recombinant inbred panel" is highly valuable.
+- Be specific. Prefer "User asked about QTL mapping for body weight in BXD mice" over "User asked about genetics".
+- Organize information chronologically using timestamps to facilitate retrieval based on time. The last interaction should the last in your collection.
+- Preserve identifiers (gene symbols, trait IDs, marker names, PubMed IDs) exactly as they appear.
+- If the input is a simple greeting or purely conversational filler with no scientific content, return an empty array.
+- If the input contains actionable scientific information, extract it.
+
+Return facts in JSON format: {"facts": ["fact1", "fact2", ...]}
+
+EXAMPLES:
+
+Input: Hi there.
+Output: {"facts": []}
+
+Input: What QTLs are associated with body weight in BXD mice?
+Output: {"facts": ["User is interested in QTL mapping for body weight", "User is querying the BXD mouse panel", "Trait of interest: body weight"]}
+
+Input: The gene Apoe on Chr 7 at 45.2 Mb has a LOD score of 6.3 for cholesterol levels in the HMDP dataset.
+Output: {"facts": ["Gene Apoe located on Chr 7 at 45.2 Mb", "LOD score of 6.3 for cholesterol levels linked to Apoe", "Dataset: HMDP", "Trait: cholesterol levels"]}
+
+Input: Can you find probesets for Gnai3 in the Hippocampus dataset?
+Output: {"facts": ["User is searching for probesets targeting gene Gnai3", "Tissue/dataset context: Hippocampus", "Gene of interest: Gnai3"]}
+
+Input: I only care about rat data. Show me phenotype traits for Rattus norvegicus.
+Output: {"facts": ["User prefers rat data only", "Species constraint: Rattus norvegicus", "User is interested in phenotype traits"]}
+
+Input: There are branches in trees.
+Output: {"facts": []}
+
+Input: The publication PMID:15234567 mentions a significant QTL on Chr 1 for anxiety behavior in the BXD panel.
+Output: {"facts": ["Publication PMID:15234567 mentions significant QTL on Chr 1 for anxiety behavior", "Trait: anxiety behavior", "Panel: BXD", "Chromosomal location: Chr 1"]}"""
+
+GN_UPDATE_MEMORY_PROMPT = """You are a memory manager for a genomics research assistant (GeneNetwork). Compare the new facts below with the existing memories and determine the appropriate action for EACH new fact.
+
+ACTIONS:
+- ADD: The new fact contains information not present in existing memories. This includes:
+  - Different gene symbols, trait names, dataset names, or marker IDs
+  - Different chromosomal locations or genomic coordinates
+  - Different species or strains
+  - Different publications or PubMed IDs
+  - Different QTL results, LOD scores, p-values, or statistical findings
+  - Different experimental methods or databases
+  - Expanded or additional context about a related topic
+- UPDATE: The new fact provides additional detail, a correction, or a more complete version of an existing memory (e.g., adding a LOD score to a gene-trait association already in memory).
+- NONE: The new fact is truly identical to an existing memory. Only use this if the fact conveys exactly the same information with no new details. Note: in genomics, related but different entities (e.g., "gene Apoe" vs "gene Apoa1", "Chr 1" vs "Chr 2", "BXD" vs "HMDP") are DIFFERENT and should be ADDED, not treated as identical.
+- DELETE: The new fact directly contradicts an existing memory (e.g., a corrected gene location, a retracted finding).
+
+CRITICAL RULES FOR GENOMICS DATA:
+1. **Gene symbols are case-sensitive and distinct.** "Apoe" and "APOE" may refer to the same gene (mouse vs human ortholog), but "Apoe" and "Apoa1" are completely different. When in doubt, ADD.
+2. **Trait names are distinct.** "body weight" and "liver weight" are different traits. ADD both.
+3. **Dataset/panel names are distinct.** "BXD", "HMDP", "Collaborative Cross" are different. ADD each.
+4. **Chromosomal locations are distinct.** "Chr 1:45.2 Mb" and "Chr 2:45.2 Mb" are different. ADD both.
+5. **QTL/statistical results are additive.** If memory says "Apoe linked to cholesterol" and new fact says "Apoe has LOD 6.3 for cholesterol", this is an UPDATE (adds quantitative detail). If new fact says "Apoe has LOD 6.3 for glucose", this is an ADD (different trait).
+6. **Publications are distinct by PMID.** Different PMIDs always ADD.
+7. **User preferences evolve.** If user previously said "only mouse data" and now says "include rat data too", UPDATE the preference to "mouse and rat data".
+
+Existing memories:
+{existing_memories}
+
+New facts:
+{new_memories}
+
+Return a JSON object in the following format:
+{{
+    "tool_calls": [
+        {{
+            "tool_name": "add_memory" | "update_memory" | "delete_memory" | "none",
+            "data": "new or updated memory text",
+            "memory_id": "id of the memory to update or delete (if applicable)"
+        }}
+    ]
+}}
+
+For each new fact, decide independently. Be more permissive with ADD than the default behavior — missing a useful genomics fact is worse than a minor duplication."""
diff --git a/aisearch/src/gnais/search/rag.py b/aisearch/src/gnais/search/rag.py
new file mode 100644
index 0000000..7366e26
--- /dev/null
+++ b/aisearch/src/gnais/search/rag.py
@@ -0,0 +1,93 @@
+"""Module with RAG system for AI search in GeneNetwork"""
+
+import asyncio
+from typing import Any
+
+import dspy
+from gnais.search.prompts import GENERAL_SYSTEM_PROMPT
+from gnais.search.tools import route_model, with_memory
+
+
+class RAG(dspy.Signature):
+    input_text: str = dspy.InputField(desc="Query and instructions")
+    chat_history: list = dspy.InputField(desc="History of conversation")
+    context: list = dspy.InputField(desc="Background information")
+    feedback: str = dspy.OutputField(
+        desc="""System response to the query — answer ONLY from the context and chat history provided.
+HTML answer. Link rules:
+- ONLY use <a href> for full web URLs that literally appear in the context.
+- NEVER invent RDF/IRI links.
+- For entities (datasets, genes, traits, strains) use PLAIN TEXT or <strong> tags.
+- If a URL is not in the context, do NOT create a link for it.
+- Always fully expand prefixes (.e.g. gn:87e8288b-697c-5b77-a944-bc27d89b19c3 -> https://rdf.genenetwork.org/v1/id/87e8288b-697c-5b77-a944-bc27d89b19c3)
+
+Fully expand prefixes according to the following map:
+
+PREFIX dcat: <http://www.w3.org/ns/dcat#>
+PREFIX dct: <http://purl.org/dc/terms/>
+PREFIX ex: <http://example.org/stuff/1.0/>
+PREFIX fabio: <http://purl.org/spar/fabio/>
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+PREFIX generif: <http://www.ncbi.nlm.nih.gov/gene?cmd=Retrieve&dopt=Graphics&list_uids=>
+PREFIX genotype: <http://rdf.genenetwork.org/v1/genotype/>
+PREFIX gn: <http://rdf.genenetwork.org/v1/id/>
+PREFIX gnc: <http://rdf.genenetwork.org/v1/category/>
+PREFIX gnt: <http://rdf.genenetwork.org/v1/term/>
+PREFIX owl: <http://www.w3.org/2002/07/owl#>
+PREFIX phenotype: <http://rdf.genenetwork.org/v1/phenotype/>
+PREFIX prism: <http://prismstandard.org/namespaces/basic/2.0/>
+PREFIX publication: <http://rdf.genenetwork.org/v1/publication/>
+PREFIX pubmed: <http://rdf.ncbi.nlm.nih.gov/pubmed/>
+PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
+PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
+PREFIX taxon: <https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Info&id=>
+PREFIX up: <http://purl.uniprot.org/core/>
+PREFIX xkos: <http://rdf-vocabulary.ddialliance.org/xkos#>
+PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
+"""
+    )
+
+
+def _make_rag_stream():
+    routed = route_model()(dspy.Predict(RAG))
+    chosen_model = routed.choose_model()
+    print(f"Choice made: {chosen_model} for RAG")
+    options = routed.options
+    rag = routed.module
+    rag.set_lm(options[chosen_model])
+
+    return dspy.streamify(
+        rag,
+        stream_listeners=[
+            dspy.streaming.StreamListener(
+                signature_field_name="feedback",
+                allow_reuse=True,
+            )
+        ],
+        include_final_prediction_in_output_stream=True,
+    )
+
+
+@with_memory(memory_type="rag")
+async def rag_search(
+    query: str,
+    retriever: Any,
+    system_prompt: str = GENERAL_SYSTEM_PROMPT,
+    user_id: str = "default_user",
+    memory=None,
+    chat_history: list = [],
+):
+    prompt = f"{system_prompt}\nQuery: {query}"
+    yield {"status": "Fetching context…"}
+    context = await asyncio.to_thread(retriever.invoke, query)
+    yield {"status": "Streaming response…"}
+    async for value in _make_rag_stream()(
+        input_text=prompt,
+        chat_history=chat_history,
+        context=context,
+    ):
+        if isinstance(value, dspy.Prediction):
+            yield {"final": value.feedback}
+        else:
+            yield getattr(value, "chunk", str(value))
diff --git a/aisearch/src/gnais/search/ragent.py b/aisearch/src/gnais/search/ragent.py
new file mode 100644
index 0000000..249c305
--- /dev/null
+++ b/aisearch/src/gnais/search/ragent.py
@@ -0,0 +1,216 @@
+"""Hybrid system of RAG and Agent for GeneNetwork's AI Search"""
+
+__all__ = (
+    "hybrid_search",
+    "Synthesis",
+    "StreamEvent",
+)
+
+import asyncio
+import time
+from functools import lru_cache, partial
+from typing import Any
+
+import dspy
+from gnais.config import Config
+from gnais.search.agent import agent_search
+from gnais.search.classification import classify_search
+from gnais.search.corpus import create_ensemble_retriever, get_chroma_db, get_docs
+from gnais.search.grag import graph_rag_search
+from gnais.search.rag import rag_search
+from gnais.search.tools import LLM_EXECUTOR, route_model
+from typing_extensions import TypedDict
+
+
+class StreamEvent(TypedDict):
+    """SSE-ready event emitted by the hybrid search stream."""
+
+    source: str
+    kind: str
+    content: str
+
+
+class Synthesis(dspy.Signature):
+    """Synthesize the final response from all search components.
+    Make sure the final synthesis is in line with the query.
+
+    Format your entire response as valid HTML. Use tags such as
+    <p>, <ul>, <li>, <a>, <strong>, <em>, <h3>, and <br>.
+    Do not wrap the response in markdown code blocks.
+
+    CRITICAL – URL SAFETY RULES (prevents hallucination):
+    1. NEVER invent, guess, or construct any URL.
+    2. Only use URLs that appear verbatim in the `all_generation` input.
+    3. Before outputting, verify: every `href` attribute value must be an exact substring
+       of the `all_generation` string. If you cannot verify, omit the link.
+    4. Do not append, prepend, or modify URL fragments (e.g., no `#`, `?`, or path changes).
+
+    Structure the HTML as follows:
+    - A short status banner (<div class="status-success"> or "status-partial">).
+    - A summary paragraph (<p class="summary">).
+    - For each group of results, an <h3> heading with the type
+      (e.g. "Genes", "Traits", "Datasets") followed by a <ul class="card-list">.
+      Each item should be a <li class="card-item"> containing:
+        - <div class="card-title"><a href="URL">Name</a></div>
+        - <div class="card-description">Description</div> (optional)
+        - <div class="card-meta">Extra info</div> (optional)
+    - If there's need for a table, add a table with a <table class="data">
+    - If no results were found, a <div class="note-box"> explaining why.
+    - Any additional notes in a <div class="note-box"> at the end.
+
+    """
+
+    original_query: str = dspy.InputField()
+    all_generation: list[str] = dspy.InputField()
+    feedback: str = dspy.OutputField(
+        desc="Final synthesized response formatted as valid HTML"
+    )
+
+
+_synthesize = dspy.streamify(
+    route_model()(dspy.Predict(Synthesis)),
+    stream_listeners=[
+        dspy.streaming.StreamListener(signature_field_name="feedback", allow_reuse=True)
+    ],
+    include_final_prediction_in_output_stream=True,
+)
+
+
+# KLUDGE: Creating the retrievers takes the longest time.  Here, we
+# create them at module time once.  XXXX: Find a better way to remove
+# these globals!
+_CHROMA_DB = get_chroma_db(
+    chroma_host="localhost",
+    chroma_port=8000,
+    embed_model="Qwen/Qwen3-Embedding-0.6B",
+)
+_DOCS = get_docs(Config.CORPUS_PATH)
+_KW_RETRIEVER = create_ensemble_retriever(
+    chroma_db=_CHROMA_DB, docs=_DOCS, keyword_weight=0.7
+)
+_SEM_RETRIEVER = create_ensemble_retriever(
+    chroma_db=_CHROMA_DB, docs=_DOCS, keyword_weight=0.5
+)
+
+
+async def _rag_search(query: str, user_id: str = "default_user", memory=None):
+    yield {"status": "Classifying search type…"}
+    loop = asyncio.get_running_loop()
+    classification = await loop.run_in_executor(
+        LLM_EXECUTOR, classify_search, query
+    )
+    yield {"status": f"Search type is: '{classification.get('decision')}'"}
+    retriever = (
+        _KW_RETRIEVER if classification.get("decision") == "keyword" else _SEM_RETRIEVER
+    )
+    yield {"status": "Retrieving documents…"}
+    async for item in rag_search(
+        query=query, retriever=retriever, user_id=user_id, memory=memory
+    ):
+        yield item
+
+
+async def _grag_search(query: str, user_id: str = "default_user", memory=None):
+    async for chunk in graph_rag_search(
+        query=query, sparql_url=Config.SPARQL_ENDPOINT, memory=memory, user_id=user_id
+    ):
+        yield chunk
+
+
+_agent_search = partial(agent_search, sparql_url=Config.SPARQL_ENDPOINT)
+
+
+async def _stream_component(
+    source: str, search_func: Any, queue: asyncio.Queue, **kwargs
+) -> None:
+    start = time.monotonic()
+    try:
+        async for chunk in search_func(**kwargs):
+            if isinstance(chunk, dict) and "final" in chunk:
+                await queue.put(
+                    StreamEvent(source=source, kind="final", content=chunk["final"])
+                )
+            elif isinstance(chunk, dict) and "status" in chunk:
+                await queue.put(
+                    StreamEvent(source=source, kind="status", content=chunk["status"])
+                )
+            else:
+                await queue.put(
+                    StreamEvent(source=source, kind="chunk", content=str(chunk))
+                )
+    except Exception as exc:
+        await queue.put(StreamEvent(source=source, kind="error", content=str(exc)))
+    finally:
+        elapsed = time.monotonic() - start
+        await queue.put(
+            StreamEvent(source=source, kind="timing", content=f"{elapsed:.2f}s")
+        )
+        await queue.put(StreamEvent(source=source, kind="done", content=""))
+
+
+async def hybrid_search(query: str, user_id: str = "default_user", memory=None):
+    """Run hybrid search with concurrent RAG, GraphRAG, and Agent.
+
+    Yields :class:`StreamEvent` dicts for progress from each component,
+    followed by a final synthesis event with ``source="hybrid"``.
+    """
+    total_start = time.monotonic()
+    queue: asyncio.Queue = asyncio.Queue()
+    combined_outputs = {"rag": "", "grag": "", "agent": ""}
+
+    async with asyncio.TaskGroup() as tg:
+        tg.create_task(
+            _stream_component(
+                "rag", _rag_search, queue, query=query, user_id=user_id, memory=memory
+            )
+        )
+        tg.create_task(
+            _stream_component(
+                "grag", _grag_search, queue, query=query, user_id=user_id, memory=memory
+            )
+        )
+        tg.create_task(
+            _stream_component(
+                "agent",
+                _agent_search,
+                queue,
+                query=query,
+                user_id=user_id,
+                memory=memory,
+            )
+        )
+
+        remaining = 3
+        while remaining:
+            event = await queue.get()
+            yield event
+
+            if event["kind"] == "final":
+                combined_outputs[event["source"]] = event["content"]
+            elif event["kind"] == "done":
+                remaining -= 1
+
+    messages = [
+        query,
+        combined_outputs["rag"],
+        combined_outputs["grag"],
+        combined_outputs["agent"],
+    ]
+
+    synthesis_text = ""
+    has_chunks = False
+    async for value in _synthesize(original_query=query, all_generation=messages):
+        if isinstance(value, dspy.Prediction):
+            synthesis_text = value.feedback
+        else:
+            chunk = getattr(value, "chunk", str(value))
+            synthesis_text += chunk
+            has_chunks = True
+            yield StreamEvent(source="synthesis", kind="chunk", content=chunk)
+
+    if not has_chunks and synthesis_text:
+        yield StreamEvent(source="synthesis", kind="chunk", content=synthesis_text)
+
+    total_elapsed = time.monotonic() - total_start
+    yield StreamEvent(source="hybrid", kind="timing", content=f"{total_elapsed:.2f}s")
+    yield StreamEvent(source="hybrid", kind="final", content=synthesis_text)
diff --git a/aisearch/src/gnais/search/tools.py b/aisearch/src/gnais/search/tools.py
new file mode 100644
index 0000000..5ac7a0a
--- /dev/null
+++ b/aisearch/src/gnais/search/tools.py
@@ -0,0 +1,717 @@
+import asyncio
+import concurrent.futures
+import functools
+import logging
+import os
+import random
+from typing import Any
+
+import dspy
+import httpx
+import redis
+from gnais.config import Config
+
+_redis = redis.Redis(host="localhost", port=6379, decode_responses=True)
+
+# Dedicated thread pool for LLM inference so heavy model calls don't
+# saturate the default asyncio executor used for lighter I/O work.
+LLM_EXECUTOR = concurrent.futures.ThreadPoolExecutor(
+    max_workers=os.cpu_count() // 2, thread_name_prefix="llm-worker"
+)
+
+# mem0's internal history store can spew sqlite transaction warnings;
+# suppress them so they don't clutter CLI output.
+for _mem0_logger in ("mem0", "mem0.memory", "mem0.memory.main"):
+    logging.getLogger(_mem0_logger).addFilter(
+        lambda record: "Failed to add history" not in record.getMessage()
+    )
+
+
+async def _exec_sparql(
+    sparql_uri: str,
+    query: str,
+    max_retries: int = 3,
+    base_delay: float = 1,
+) -> dict:
+    """Execute a single SPARQL query with retry + exponential jitter via httpx."""
+    client = httpx.AsyncClient(
+        timeout=httpx.Timeout(180.0, connect=5.0),
+        limits=httpx.Limits(max_connections=20, max_keepalive_connections=10),
+    )
+    for attempt in range(max_retries):
+        try:
+            resp = await client.post(
+                sparql_uri,
+                data={"query": query},
+                headers={"Accept": "application/sparql-results+json"},
+            )
+            resp.raise_for_status()
+            return resp.json()
+        except httpx.HTTPStatusError as e:
+            if e.response.status_code in (504, 503, 502) and attempt < max_retries - 1:
+                await asyncio.sleep(base_delay * (2**attempt) + random.uniform(0, 1))
+                continue
+            raise
+    return {}
+
+
+def build_schema_hint(sparql_uri: str) -> str:
+    """Build a compact schema hint from the live Virtuoso endpoint.
+
+    Cached in Redis for 1 week so multiple workers / restarts share it.
+    """
+    cache_key = f"gn:schema_hint:{sparql_uri}"
+    cached = _redis.get(cache_key)
+    if cached:
+        return cached
+
+    hint = """
+=== HIGH LEVEL STRUCTURE OF THE GENENETWORK RDF GRAPH ===
+```mermaid
+graph TD
+    A[gnc:resource_classification_scheme<br/>a skos:ConceptScheme] -->|xkos:levels| B[gnc:taxonomic_family<br/>a xkos:ClassificationLevel<br/>xkos:depth 1]
+    A -->|xkos:levels| C[gnc:species<br/>a xkos:ClassificationLevel<br/>xkos:depth 2]
+    A -->|xkos:levels| D[gnc:population_category<br/>a xkos:ClassificationLevel<br/>xkos:depth 3]
+    A -->|xkos:levels| E[gnc:set<br/>a xkos:ClassificationLevel<br/>xkos:depth 4]
+
+    B -->|xkos:nextLevel| C
+    C -->|xkos:nextLevel| D
+    D -->|xkos:nextLevel| E
+
+    B -->|gnt:has_taxonomic_family| F[gni:family_*<br/>no explicit rdf:type]
+    C -->|skos:member| G[gni:species_*<br/>a gnc:species]
+    D -->|gnt:has_reference_population| H[gni:population_*<br/>a gnc:reference_population]
+    E -->|skos:member| I[gni:set_*<br/>a gnc:set]
+
+    F -->|gnt:has_species| G
+    G -->|gnt:has_strain| I
+    H -->|gnt:has_strain| I
+
+    I -->|gnt:has_species| G
+    I -->|gnt:has_reference_population| H
+
+    G -->|gnt:has_uniprot_taxon_id| J[uniprot taxon ID]
+
+    I -->|gnt:has_genotype_data| K[gni:dataset_*]
+    I -->|gnt:has_phenotype_data| K
+    I -->|gnt:has_probeset_data| K
+    I -->|gnt:uses_mapping_method| L[gni:mapping_method_*]
+
+    K -->|gnt:has_molecular_trait| M[gni:trait_*]
+    K -->|gnt:has_phenotype_trait| M
+    K -->|gnt:uses_genechip| N[gni:platform_*]
+    K -->|gnt:uses_normalization_method| O[gni:avg_method_*]
+    K -->|gnt:has_strain| I
+```
+
+=== DETAILED SCHEMA OF GENENETWORK RDF DATABASE ===
+```turtle
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
+@prefix xkos: <http://rdf-vocabulary.ddialliance.org/xkos#> .
+@prefix dcterms: <http://purl.org/dc/terms/> .
+@prefix dcat: <http://www.w3.org/ns/dcat#> .
+@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
+@prefix owl: <http://www.w3.org/2002/07/owl#> .
+@prefix gnc: <http://rdf.genenetwork.org/v1/category/> .
+@prefix gni: <http://rdf.genenetwork.org/v1/id/> .
+@prefix gnt: <http://rdf.genenetwork.org/v1/term/> .
+@prefix uniprot: <http://purl.uniprot.org/taxonomy/> .
+
+# -------------------------------------------------
+# Schema and its four levels
+# -------------------------------------------------
+
+gnc:resource_classification_scheme a skos:ConceptScheme ;
+    rdfs:label "GeneNetwork Resource Classification Scheme" ;
+    skos:prefLabel "GeneNetwork Resource Classification Scheme" ;
+    skos:definition "A hierarchical classification scheme for organizing GeneNetwork resources by dataset type, resource set (inbredset group), or species." ;
+    xkos:numberOfLevels 4 ;
+    xkos:levels gnc:taxonomic_family , gnc:species , gnc:population_category , gnc:set .
+
+# Level containers. Each is an xkos:ClassificationLevel and also a class for its members.
+gnc:taxonomic_family a xkos:ClassificationLevel , rdfs:Class ;
+    rdfs:label "Family" ;
+    skos:prefLabel "Family" ;
+    skos:definition "An organizational classification level used in GeneNetwork to group resources into families." ;
+    xkos:depth 1 ;
+    xkos:nextLevel gnc:species ;
+    skos:inScheme gnc:resource_classification_scheme .
+
+gnc:species a xkos:ClassificationLevel , rdfs:Class ;
+    rdfs:label "Species" ;
+    skos:prefLabel "Species" ;
+    skos:definition "A classification level that associates a given resource to a species in GeneNetwork." ;
+    xkos:depth 2 ;
+    xkos:previousLevel gnc:taxonomic_family ;
+    xkos:nextLevel gnc:population_category ;
+    skos:inScheme gnc:resource_classification_scheme .
+
+gnc:population_category a xkos:ClassificationLevel , rdfs:Class ;
+    rdfs:label "Population Category" ;
+    skos:prefLabel "Population Category" ;
+    xkos:depth 3 ;
+    xkos:previousLevel gnc:species ;
+    xkos:nextLevel gnc:set ;
+    skos:inScheme gnc:resource_classification_scheme .
+
+gnc:set a xkos:ClassificationLevel , rdfs:Class ;
+    rdfs:label "Set" ;
+    skos:prefLabel "Set" ;
+    skos:definition "A category representing groups of genetically related strains or individuals (inbred sets, recombinant inbred lines, etc.)." ;
+    xkos:depth 4 ;
+    xkos:previousLevel gnc:population_category ;
+    skos:inScheme gnc:resource_classification_scheme .
+
+# -------------------------------------------------
+# Classes for non-level member instances
+# -------------------------------------------------
+
+gnc:reference_population a rdfs:Class ;
+    rdfs:label "Reference Population" ;
+    rdfs:comment "Class of the entities linked from gnc:population_category via gnt:has_reference_population." .
+
+# Note: gni:family_* entities are linked from gnc:taxonomic_family but do not carry an explicit rdf:type in the source data.
+
+# -------------------------------------------------
+# Properties
+# -------------------------------------------------
+
+# XKOS / SKOS structural properties
+xkos:levels a rdf:Property ;
+    rdfs:label "levels" ;
+    rdfs:domain skos:ConceptScheme ;
+    rdfs:range xkos:ClassificationLevel .
+
+xkos:numberOfLevels a rdf:Property ;
+    rdfs:label "number of levels" ;
+    rdfs:domain skos:ConceptScheme ;
+    rdfs:range xsd:integer .
+
+xkos:nextLevel a rdf:Property ;
+    rdfs:label "next level" ;
+    rdfs:domain xkos:ClassificationLevel ;
+    rdfs:range xkos:ClassificationLevel .
+
+xkos:previousLevel a rdf:Property ;
+    rdfs:label "previous level" ;
+    rdfs:domain xkos:ClassificationLevel ;
+    rdfs:range xkos:ClassificationLevel .
+
+xkos:depth a rdf:Property ;
+    rdfs:label "depth" ;
+    rdfs:domain xkos:ClassificationLevel ;
+    rdfs:range xsd:integer .
+
+skos:member a rdf:Property ;
+    rdfs:label "member" ;
+    rdfs:domain skos:Collection ;
+    rdfs:range skos:Concept ;
+    rdfs:comment "Used here by gnc:species and gnc:set to link to their member instances." .
+
+# GeneNetwork-specific membership properties
+
+gnt:has_taxonomic_family a rdf:Property ;
+    rdfs:label "has taxonomic family" ;
+    rdfs:domain gnc:taxonomic_family ;
+    rdfs:range rdfs:Resource ;
+    rdfs:comment "Links the taxonomic_family level to gni:family_* entities." .
+
+gnt:has_reference_population a rdf:Property ;
+    rdfs:label "has reference population" ;
+    rdfs:domain [ a rdfs:Class ; owl:unionOf ( gnc:population_category gnc:set ) ] ;
+    rdfs:range gnc:reference_population ;
+    rdfs:comment "Links the population_category level to gni:population_* entities; also used from gni:set_* back to populations." .
+
+gnt:has_species a rdf:Property ;
+    rdfs:label "has species" ;
+    rdfs:domain [ a rdfs:Class ; owl:unionOf ( gnc:taxonomic_family gnc:set ) ] ;
+    rdfs:range gnc:species ;
+    rdfs:comment "Links family entities to species, and set entities back to species." .
+
+gnt:has_strain a rdf:Property ;
+    rdfs:label "has strain" ;
+    rdfs:domain [ a rdfs:Class ; owl:unionOf ( gnc:species gnc:reference_population ) ] ;
+    rdfs:range gnc:set ;
+    rdfs:comment "Links species or reference populations to gni:set_* entities." .
+
+# GeneNetwork-specific descriptive properties on instances
+
+gnt:has_uniprot_taxon_id a rdf:Property ;
+    rdfs:label "has UniProt taxon ID" ;
+    rdfs:domain gnc:species ;
+    rdfs:range uniprot:Taxon .
+
+gnt:short_name a rdf:Property ;
+    rdfs:label "short name" ;
+    rdfs:domain gnc:species ;
+    rdfs:range xsd:string .
+
+gnt:has_family_order_id a rdf:Property ;
+    rdfs:label "has family order ID" ;
+    rdfs:domain rdfs:Resource ;
+    rdfs:range xsd:integer ;
+    rdfs:comment "Used on gni:family_* entities for ordering." .
+
+gnt:has_population_order_id a rdf:Property ;
+    rdfs:label "has population order ID" ;
+    rdfs:domain gnc:reference_population ;
+    rdfs:range xsd:integer .
+
+gnt:has_set_code a rdf:Property ;
+    rdfs:label "has set code" ;
+    rdfs:domain gnc:set ;
+    rdfs:range xsd:string .
+
+# Dataset linkage properties from gni:set_* entities
+
+gnt:has_genotype_data a rdf:Property ;
+    rdfs:label "has genotype data" ;
+    rdfs:domain gnc:set ;
+    rdfs:range dcat:Dataset .
+
+gnt:has_phenotype_data a rdf:Property ;
+    rdfs:label "has phenotype data" ;
+    rdfs:domain gnc:set ;
+    rdfs:range dcat:Dataset .
+
+gnt:has_probeset_data a rdf:Property ;
+    rdfs:label "has probeset data" ;
+    rdfs:domain gnc:set ;
+    rdfs:range dcat:Dataset .
+
+gnt:uses_mapping_method a rdf:Property ;
+    rdfs:label "uses mapping method" ;
+    rdfs:domain gnc:set ;
+    rdfs:range skos:Concept ;
+    rdfs:comment "Refers to concepts in the gnc:mapping_method SKOS scheme." .
+
+# Dataset-level properties
+
+gnt:has_molecular_trait a rdf:Property ;
+    rdfs:label "has molecular trait" ;
+    rdfs:domain dcat:Dataset ;
+    rdfs:range skos:Concept ;
+    rdfs:comment "Refers to concepts in the gnc:molecular_trait collection." .
+
+gnt:has_phenotype_trait a rdf:Property ;
+    rdfs:label "has phenotype trait" ;
+    rdfs:domain dcat:Dataset ;
+    rdfs:range skos:Concept ;
+    rdfs:comment "Used by phenotype datasets." .
+
+gnt:uses_genechip a rdf:Property ;
+    rdfs:label "uses genechip" ;
+    rdfs:domain dcat:Dataset ;
+    rdfs:range rdfs:Resource ;
+    rdfs:comment "Refers to gni:platform_* entities." .
+
+gnt:uses_normalization_method a rdf:Property ;
+    rdfs:label "uses normalization method" ;
+    rdfs:domain dcat:Dataset ;
+    rdfs:range skos:Concept ;
+    rdfs:comment "Refers to concepts in the gnc:avg_method SKOS scheme." .
+
+# Common annotation properties
+
+rdfs:label a rdf:Property ;
+    rdfs:label "label" ;
+    rdfs:domain rdfs:Resource ;
+    rdfs:range xsd:string .
+
+skos:prefLabel a rdf:Property ;
+    rdfs:label "preferred label" ;
+    rdfs:domain rdfs:Resource ;
+    rdfs:range xsd:string .
+
+skos:altLabel a rdf:Property ;
+    rdfs:label "alternative label" ;
+    rdfs:domain rdfs:Resource ;
+    rdfs:range xsd:string .
+
+skos:definition a rdf:Property ;
+    rdfs:label "definition" ;
+    rdfs:domain rdfs:Resource ;
+    rdfs:range xsd:string .
+
+dcterms:description a rdf:Property ;
+    rdfs:label "description" ;
+    rdfs:domain rdfs:Resource ;
+    rdfs:range rdf:HTML ;
+    rdfs:comment "On gni:set_* entities this is often an rdf:HTML literal." .
+
+dcterms:created a rdf:Property ;
+    rdfs:label "created date" ;
+    rdfs:domain dcat:Dataset ;
+    rdfs:range xsd:datetime .
+
+dcterms:identifier a rdf:Property ;
+    rdfs:label "identifier" ;
+    rdfs:domain dcat:Dataset ;
+    rdfs:range xsd:string .
+
+dcterms:title a rdf:Property ;
+    rdfs:label "title" ;
+    rdfs:domain dcat:Dataset ;
+    rdfs:range xsd:string .
+```
+
+=== EXAMPLES OF SUCCESSFUL QUERIES WITH KEY QUERY PATTERNS ===
+
+Question: List all classification levels in order
+SPARQL Query:
+```sparql
+SELECT ?level ?depth ?label
+WHERE {
+  gnc:resource_classification_scheme xkos:levels ?level .
+  ?level xkos:depth ?depth ;
+         skos:prefLabel ?label .
+}
+ORDER BY ?depth
+```
+
+Question: Find all sets belonging to a species
+SPARQL Query:
+```sparql
+SELECT ?species ?set
+WHERE {
+  ?species a gnc:species ;
+           gnt:has_strain ?set .
+}
+```
+
+Question: Find all datasets reachable from a species (through sets)
+SPARQL Query:
+```sparql
+SELECT ?species ?set ?dataset
+WHERE {
+  ?species a gnc:species ;
+           gnt:has_strain ?set .
+  ?set gnt:has_genotype_data|gnt:has_phenotype_data|gnt:has_probeset_data ?dataset .
+}
+```
+
+Question: Find all sets for a given reference population
+SPARQL Query:
+```sparql
+SELECT ?population ?set
+WHERE {
+  ?population a gnc:reference_population ;
+              gnt:has_strain ?set .
+}
+```
+
+Question: Traverse family → species → set → dataset
+SPARQL Query:
+```sparql
+SELECT ?family ?species ?set ?dataset
+WHERE {
+  gnc:taxonomic_family gnt:has_taxonomic_family ?family .
+  ?family gnt:has_species ?species .
+  ?species gnt:has_strain ?set .
+  ?set gnt:has_genotype_data|gnt:has_phenotype_data|gnt:has_probeset_data ?dataset .
+}
+```
+    """
+    _redis.setex(cache_key, 604800, hint)  # 1 week
+    return hint
+
+
+class QueryTranslation(dspy.Signature):
+    """Use schema hints to generate valid SPARQL SELECT queries that can retrieve relevant information for the query.
+
+    CRITICAL SCHEMA RULES (derived from GeneNetwork RDF schema):
+    1. Every query MUST start with the PREFIX declarations. Only use declared prefixes.
+    2. ALWAYS include `FROM <http://rdf.genenetwork.org/v1>` between SELECT and WHERE.
+    3. Strain URIs: use gn:set_BXD, gn:set_B6D2F1, gn:set_HMDP — NEVER gn:BXD, gn:B6D2F1, gn:HMDP.
+    4. gnc:phenotype is metadata only (abbreviation, description, lab_code, submitter, contributor). Phenotype TRAITS (with mean, locus, lod_score, additive, sequence, has_trait_page) are gnc:phenotype_trait.
+    5. Probesets and DNA markers use gnt:chr for chromosome. Genes use gnt:chromosome.
+    6. gnt:has_uniprot_id, gnt:has_homologene_id, gnt:has_kegg_id, gnt:has_omim_id, gnt:has_chebi_id, gnt:has_pub_chem_id exist on gnc:probeset, NOT on gnc:gene.
+    7. gnt:has_align_id, gnt:has_protein_id, gnt:has_rgd_id, gnc:has_kg_id, gnc:has_unigen_id exist on gnc:gene, NOT on gnc:dna_marker. Markers use skos:prefLabel or skos:altLabel for names.
+    8. gnt:locus on phenotype_trait contains chromosome positions (e.g. "1-59904011"), NOT phenotype names. Never FILTER gnt:locus for trait names like "liver_weight". Instead link trait -> has_phenotype -> phenotype and filter gnt:abbreviation.
+    9. gnt:symbol is for probesets. gnt:gene_symbol is for genes and strains.
+    10. Datasets are dcat:Dataset. They have dct:title, gnt:has_strain, gnt:has_tissue_info, gnt:has_samples, gnt:has_summary, gnt:has_citation, gnt:has_contributors, gnt:has_case_info, gnt:has_platform_info, gnt:has_specifics, gnt:has_data_processing_info, gnt:has_experiment_design, gnt:has_experiment_type, gnt:has_genotype_files.
+    11. Do NOT use taxon: for species. Use gn:Mus_musculus, gn:Rattus_norvegicus, gn:Homo_sapiens, etc.
+    12. Gene chips / platforms are skos:Concept with skos:inScheme gnc:gene_chip. They have gnt:has_go_tree_value, gnt:has_geo_series_id.
+    13. Mapping methods and averaging methods are skos:Concept in gnc:mapping_method and gnc:avg_method schemes.
+    14. gnt:has_trait_page gives the URL directly. Never build trait URLs manually.
+    15. Leverage as many schema hints as possible.
+    16. To extract information about a specific trait, find the closest matches in SNAPSHOT_OBJECTS and use them to infer the proper RDF subject for the trait.
+    17. To get trait page or URL, use the predicate `gnt:has_trait_page` with the trait as subject. Its object correspond to the URL. Never build trait URLs manually.
+    18. To get publication for a trait, search for object having the predicate `dcterms:references` and the trait as subject.
+    19. To get highest LOD score and corresponding marker and additive effect, use the predicates `gnt:lod_score`, `gnt:locus`, and `gnt:additive`, respectively.
+    20. Mean trait measurements can be accessed by looking up object with predicate gnt:mean and the trait as subject.
+    21. Trait descriptions can be accessed via the predicate `dcterms:description`
+    22. When you have the description of a trait, you should use regex pattern in the object term with the predicate `dcterms:description` to find the corresponding trait (subject).
+
+    When querying SPARQL, prefer fast, efficient SPARQL SELECT queries
+    that avoid Virtuoso timeouts (504 errors).
+
+    CRITICAL PERFORMANCE RULES (to prevent 504s):
+    1. Always add `LIMIT` - start with `LIMIT 50`, increase only if needed. Never omit `LIMIT`.
+    2. Never use `SELECT *` - list only the variables you actually need.
+    3. Avoid expensive operations: no Cartesian products, no cross joins, no full graph scans.
+    4. Use specific FILTER patterns that leverage indexes:
+       - Prefer `STRSTARTS(?label, "prefix")` over `CONTAINS` or regex.
+       - Avoid `FILTER regex(...)` - it disables indexes.
+       - Use `FILTER(?value = "exact")` or `IN` with small lists.
+    5. Prefer property paths over multiple joins when traversing a chain.
+    6. Use VALUES blocks for small sets of constants instead of UNION or OPTIONAL.
+    7. Avoid ORDER BY on large result sets - if needed, combine with `LIMIT` and a narrow `WHERE` clause.
+    8. Never use nested subqueries unless absolutely necessary; flatten them.
+    9. Use `OPTIONAL` only for truly optional patterns – otherwise, use a simple triple pattern.
+    """
+
+    original_query: str = dspy.InputField(desc="User query")
+    schema_hint: str = dspy.InputField(desc="GeneNetwork schema from Virtuoso")
+    translated_queries: list[str] = dspy.OutputField(
+        desc="Top 7 valid most relevant SPARQL queries with the highest likelihood of getting non-empty results."
+    )
+
+
+async def sparql_fetch(
+    sparql_queries: list[str],
+    sparql_uri: str,
+    max_retries: int = 3,
+    base_delay: float = 0.5,
+) -> str:
+    """Execute *sparql_queries* concurrently against *sparql_uri*."""
+    if not sparql_queries:
+        return "No SPARQL queries to run."
+
+    async def _fetch_one(query: str, idx: int) -> str:
+        try:
+            result = await _exec_sparql(sparql_uri, query, max_retries, base_delay)
+            bindings = result.get("results", {}).get("bindings", [])
+            return f"Query {idx} succeeded ({len(bindings)} rows): {bindings}"
+        except Exception as e:
+            return f"Query {idx} failed: {e}\nQuery was:\n{query}"
+
+    tasks = [_fetch_one(q, i) for i, q in enumerate(sparql_queries)]
+    results = await asyncio.gather(*tasks)
+    return "\n\n".join(results)
+
+
+@functools.lru_cache(maxsize=64)
+def make_sparql_fetch_tool(
+    sparql_uri: str, lm: dspy.LM = Config.DEFAULT_LLM
+) -> dspy.Tool:
+    def _fetch(query: str) -> Any:
+        schema_hint = build_schema_hint(sparql_uri)
+        pred = dspy.Predict(QueryTranslation)
+        pred.set_lm(lm)
+        pred = pred(
+            original_query=query,
+            schema_hint=schema_hint,
+        )
+        sparql_queries = pred.get("translated_queries") if pred else []
+        if not sparql_queries:
+            return "No SPARQL queries generated."
+        future = LLM_EXECUTOR.submit(
+            asyncio.run, sparql_fetch(sparql_queries, sparql_uri)
+        )
+        return future.result()
+
+    return dspy.Tool(
+        name="fetch_data",
+        desc="Fetch RDF data around GeneNetwork data through SPARQL",
+        args={
+            "query": {
+                "type": "string",
+                "desc": "SPARQL query to run to fetch relevant data",
+            },
+        },
+        func=_fetch,
+    )
+
+
+def _check_link(url: str) -> str:
+    """Check whether a URL is reachable.
+
+    Returns a short status string suitable for feeding back to the LLM.
+    """
+    try:
+        response = httpx.Client(follow_redirects=True, timeout=10).head(url)
+        ok = response.is_success
+    except Exception:
+        ok = False
+
+    return f"{'Valid URL' if ok else 'Invalid URL'} URL: {url}"
+
+
+check_link = dspy.Tool(
+    name="check_link",
+    desc="Check whether a URL is valid and reachable. Call this BEFORE putting any URL in an <a href>. Returns 'Valid URL: ...' or 'Invalid URL: ...'.",
+    args={
+        "url": {
+            "type": "string",
+            "desc": "URL or link to check",
+        },
+    },
+    func=_check_link,
+)
+
+
+# KLUDGE: For now this is lifted from:
+# <https://dspy.ai/tutorials/mem0_react_agent/>
+class MemoryTools:
+    """Tools for interacting with the Mem0 memory system."""
+
+    def __init__(self, memory):
+        self.memory = memory
+
+    def store_memory(
+        self, content: str, user_id: str, run_id: str, metadata: dict = {}
+    ) -> str:
+        """Store information in memory."""
+        try:
+            self.memory.add(
+                content, user_id=user_id, run_id=run_id, metadata=metadata, infer=True
+            )
+            return f"Stored memory: {content}"
+        except Exception as e:
+            return f"Error storing memory: {str(e)}"
+
+    def search_memories(
+        self, query: str, user_id: str, run_id: str, limit: int = 10
+    ) -> str:
+        """Search for relevant memories."""
+        results = self.memory.search(
+            query, filters={"user_id": user_id, "run_id": run_id}, top_k=limit
+        )
+        chat_history = ""
+        if results and (memories := results.get("results")):
+            for memory in memories:
+                if query := memory.get("metadata", {}).get("query"):
+                    chat_history += f"Query: {query}\n"
+                chat_history += (
+                    f"\nMemory({memory['updated_at']}): {memory['memory']}\n"
+                )
+        return chat_history.strip()
+
+    def get_all_memories(self, user_id: str, run_id: str, filters: dict = {}) -> str:
+        """Get all memories for a user."""
+        try:
+            results = self.memory.get_all(
+                filters={"user_id": user_id, "run_id": run_id}
+            )
+            if not results or not results.get("results"):
+                return "No memories found for this user."
+
+            memory_text = "All memories for user:\n"
+            for i, result in enumerate(results["results"]):
+                memory_text += f"{i}. {result['memory']}\n"
+            return memory_text
+        except Exception as e:
+            return f"Error retrieving memories: {str(e)}"
+
+    def update_memory(self, memory_id: str, new_content: str) -> str:
+        """Update an existing memory."""
+        try:
+            self.memory.update(memory_id, new_content)
+            return f"Updated memory with new content: {new_content}"
+        except Exception as e:
+            return f"Error updating memory: {str(e)}"
+
+    def delete_memory(self, memory_id: str) -> str:
+        """Delete a specific memory."""
+        try:
+            self.memory.delete(memory_id)
+            return "Memory deleted successfully."
+        except Exception as e:
+            return f"Error deleting memory: {str(e)}"
+
+
+def with_memory(memory_type: str = "interaction"):
+    """Decorator factory that injects chat_history from mem0 and persists the interaction after streaming."""
+
+    def decorator(func):
+        @functools.wraps(func)
+        async def wrapper(*args, **kwargs):
+            query = kwargs.get("query")
+            memory = kwargs.get("memory")
+            user_id = kwargs.get("user_id")
+
+            # Pre: search memories and build chat_history
+            chat_history = []
+            memory_tools = None
+            if memory is not None:
+                memory_tools = MemoryTools(memory)
+                memories = memory_tools.search_memories(
+                    query, user_id=user_id, run_id=memory_type
+                )
+                if memories:
+                    chat_history = [memories]
+
+            kwargs["chat_history"] = chat_history
+            async for value in func(*args, **kwargs):
+                if isinstance(value, dict):
+                    feedback = str(value.get("final"))
+                    if memory_tools and feedback:
+                        # Fire-and-forget: don't block the stream on SQLite writes
+                        asyncio.create_task(
+                            asyncio.to_thread(
+                                memory_tools.store_memory,
+                                feedback,
+                                user_id=user_id,
+                                run_id=memory_type,
+                                metadata={"query": query},
+                            )
+                        )
+                yield value
+
+        return wrapper
+
+    return decorator
+
+
+class Route(dspy.Signature):
+    """
+    Choose the most efficient model to handle the task assigned to the program between the LLMs available.
+    The most efficient model is the LLM that can follow the instructions defined in the program with good fidelity while saving on unneccessary cost.
+    The length and complexity of instructions in the task description should orient choice.
+    Tasks with very complex instructions should be handled by the bigger model while those relatively simple can be offloaded to the smaller model.
+    """
+
+    task: str = dspy.InputField(desc="The task of the DSPy program")
+    models: list[str] = dspy.InputField(desc="The 2 models or LLMs available")
+    best_model: str = dspy.OutputField(desc="The most efficient model for the task")
+
+
+class RoutedModule(dspy.Module):
+    def __init__(self, module: dspy.Module, options: dict[str, dspy.LM]):
+        super().__init__()
+        self.module = module
+        self.options = options
+        self.router = dspy.Predict(Route)
+
+    def choose_model(self) -> str:
+        task = str(self.module.signature)
+        models = list(self.options.keys())
+        return self.router(task=task, models=models).get("best_model")
+
+    def forward(self, **kwargs):
+        chosen_model = self.choose_model()
+        print(
+            f"Choice made: {chosen_model} for {self.module.__dict__['signature'].__name__}"
+        )
+        self.module.set_lm(self.options[chosen_model])
+        return self.module(**kwargs)
+
+    def get(self, field_name, default=None):
+        return getattr(self.module, field_name, default)
+
+
+def route_model(
+    options: dict[str, dspy.LM] = {
+        Config.DEFAULT_MODEL: Config.DEFAULT_LLM,
+        Config.ALTERNATIVE_MODEL: Config.ALTERNATIVE_LLM,
+    }
+):
+    """Decorator that returns a dspy.Module wrapping the routed module."""
+
+    def decorator(module: dspy.Module):
+        return RoutedModule(module, options)
+
+    return decorator
diff --git a/aisearch/src/web/app.py b/aisearch/src/web/app.py
new file mode 100644
index 0000000..68d4a3d
--- /dev/null
+++ b/aisearch/src/web/app.py
@@ -0,0 +1,348 @@
+import asyncio
+import concurrent.futures
+import os
+import uuid
+
+import dspy
+import quart
+import quart_flask_patch  # noqa: F401
+import torch
+from flask_caching import Cache
+from flask_limiter import Limiter
+from flask_limiter.util import get_remote_address
+from gnais.config import Config
+from gnais.search.prompts import GN_FACT_EXTRACTION_PROMPT, GN_UPDATE_MEMORY_PROMPT
+from gnais.search.ragent import hybrid_search
+from markupsafe import escape
+from mem0 import Memory
+from mem0.configs.base import MemoryConfig
+from quart import (
+    Quart,
+    Response,
+    flash,
+    jsonify,
+    redirect,
+    render_template,
+    request,
+    session,
+    url_for,
+)
+from quart.typing import ResponseReturnValue
+from quart_auth import (
+    AuthUser,
+    QuartAuth,
+    Unauthorized,
+    current_user,
+    login_required,
+    login_user,
+    logout_user,
+)
+
+app = Quart(__name__)
+app.config.from_object(Config)
+
+# Basic Login
+app.secret_key = Config.SECRET_KEY
+QuartAuth(app)
+
+# Set up template and static directories
+template_dir = os.path.join(os.path.dirname(__file__), "templates")
+app.template_folder = template_dir
+static_dir = os.path.join(os.path.dirname(__file__), "static")
+app.static_folder = static_dir
+
+
+limiter = Limiter(
+    get_remote_address,
+    app=app,
+    default_limits=["100 per minute"],
+    storage_uri="redis://localhost:6379",
+    strategy="fixed-window",
+)
+
+cache = Cache(config={"CACHE_TYPE": "RedisCache"})
+cache.init_app(app)
+
+#  Bootstrapping our model
+#  NOTE: The backend serving localhost:7501 should also run on the GPU
+#  (e.g. vLLM with --device cuda) for maximum inference throughput.
+torch.manual_seed(app.config["SEED"])
+if torch.cuda.is_available():
+    torch.cuda.manual_seed_all(app.config["SEED"])
+
+
+dspy.configure(lm=app.config["DEFAULT_LLM"])
+
+
+@app.before_serving
+async def _set_default_executor():
+    loop = asyncio.get_running_loop()
+    loop.set_default_executor(concurrent.futures.ThreadPoolExecutor(max_workers=64))
+
+
+#  Shared mem0 memory instance (loaded once at server startup)
+_MEMORY = Memory(
+    config=MemoryConfig(
+        custom_fact_extraction_prompt=GN_FACT_EXTRACTION_PROMPT,
+        custom_update_extraction_prompt=GN_UPDATE_MEMORY_PROMPT,
+        llm={
+            "provider": "litellm",
+            "config": {
+                "model": Config.MEMORY_MODEL,
+                "temperature": 0.5,
+                "max_tokens": 100_000,
+                "api_key": Config.API_KEY,
+            },
+        },
+        embedder={
+            "provider": "huggingface",
+            "config": {
+                "model": "Qwen/Qwen3-Embedding-0.6B",
+                "embedding_dims": 1024,
+                "model_kwargs": {
+                    "trust_remote_code": True,
+                    "device": "cuda" if torch.cuda.is_available() else "cpu",
+                },
+            },
+        },
+        vector_store={
+            "provider": "chroma",
+            "config": {
+                "collection_name": "mem0",
+                "host": "localhost",
+                "port": 8001,
+            },
+        },
+    )
+)
+
+
+def _format_sse(event: str, data: str) -> str:
+    lines = data.splitlines() or [""]
+    payload = [f"event: {event}"]
+    payload.extend(f"data: {line}" for line in lines)
+    payload.append("")
+    return "\n".join(payload) + "\n"
+
+
+def _stream_chunk_markup(content: str, error: bool = False) -> str:
+    css_class = "stream-fragment stream-fragment-error" if error else "stream-fragment"
+    return f"<span class='{css_class}'>{escape(content)}</span>"
+
+
+def _stream_status_markup(label: str, tone: str = "working") -> str:
+    return f"<span class='stream-stage-badge is-{tone}'>{escape(label)}</span>"
+
+
+def _stream_final_status_markup(message: str, tone: str = "waiting") -> str:
+    icon = "🔍" if tone == "waiting" else "✨"
+    return (
+        f"<div class='stream-final-status is-{tone}'>"
+        f"<div class='stream-final-status-icon'>{icon}</div>"
+        f"<div class='stream-final-status-text'>{escape(message)}</div>"
+        "</div>"
+    )
+
+
+@app.route("/login", methods=["GET", "POST"])
+async def login():
+    # KLUDGE: Set a static password for now to prevent token abuse
+    # from non-GN testers.
+    users = {"test@balg-qa.genenetwork.org": os.environ.get("USER_PASS")}
+    if quart.request.method == "GET":
+        return await render_template("login.html")
+    form = await request.form
+    email, password = form.get("email"), form.get("password")
+
+    if email in users and users[email] == password:
+        login_user(AuthUser(email))
+        return redirect(url_for("index"))
+
+    flash("Set the correct user/password")
+    return redirect(url_for("login"))
+
+
+@app.route("/logout")
+async def logout():
+    logout_user()
+    return "Logged out"
+
+
+@app.errorhandler(Unauthorized)
+async def redirect_to_login(*_: Exception) -> ResponseReturnValue:
+    return redirect(url_for("login"))
+
+
+@app.route("/")
+@login_required
+async def index():
+    """Serve the AI search web interface."""
+    return await render_template("index.html")
+
+
+@app.route("/search/stream-shell", methods=["GET"])
+@limiter.limit("300 per day")
+@login_required
+async def search_stream_shell():
+    """Shell endpoint that returns the streaming message container."""
+    query = request.args.get("q", "").strip()
+    if not query:
+        return "<div class='error-message'>Missing query parameter 'q'</div>", 400
+    if len(query) > 1000:
+        return "<div class='error-message'>Query too long</div>", 400
+    if "user_id" not in session:
+        session["user_id"] = str(uuid.uuid4())
+    return await render_template("partials/stream_shell.html", query=query)
+
+
+@app.route("/search/stream", methods=["GET"])
+@limiter.limit("300 per day")
+@login_required
+async def search_stream():
+    """SSE endpoint for live hybrid search updates."""
+    query = request.args.get("q", "").strip()
+    if not query:
+        return Response(
+            _format_sse(
+                "final_html",
+                "<div class='error-message'>Missing query parameter 'q'</div>",
+            ),
+            mimetype="text/event-stream",
+        )
+    if len(query) > 1000:
+        return Response(
+            _format_sse(
+                "final_html", "<div class='error-message'>Query too long</div>"
+            ),
+            mimetype="text/event-stream",
+        )
+
+    user_id = session.get("user_id")
+    if not user_id:
+        session["user_id"] = str(uuid.uuid4())
+        user_id = session["user_id"]
+
+    async def event_stream():
+        completed = set()
+        final_sent = False
+
+        yield _format_sse("search_state", _stream_status_markup("Streaming", "working"))
+        yield _format_sse(
+            "final_html",
+            _stream_final_status_markup("Waiting for searches to complete…", "waiting"),
+        )
+        try:
+            async for event in hybrid_search(query, user_id=user_id, memory=_MEMORY):
+                if final_sent:
+                    break
+
+                source = event["source"]
+                kind = event["kind"]
+                content = event["content"]
+
+                if source in {"rag", "grag", "agent"} and kind == "status":
+                    yield _format_sse(
+                        f"{source}_chunk",
+                        f"<div class='stream-status-msg'>{escape(content)}</div>",
+                    )
+                    continue
+
+                if source in {"rag", "grag", "agent"} and kind == "chunk":
+                    yield _format_sse(
+                        f"{source}_chunk",
+                        _stream_chunk_markup(content),
+                    )
+                    continue
+
+                if source in {"rag", "grag", "agent"} and kind == "final":
+                    yield _format_sse(
+                        f"{source}_final_html",
+                        str(content),
+                    )
+                    continue
+
+                if source in {"rag", "grag", "agent"} and kind == "timing":
+                    yield _format_sse(
+                        f"{source}_timing",
+                        escape(content),
+                    )
+                    continue
+
+                if source in {"rag", "grag", "agent"} and kind == "error":
+                    yield _format_sse(
+                        f"{source}_chunk",
+                        _stream_chunk_markup(content, error=True),
+                    )
+                    yield _format_sse(
+                        f"{source}_done",
+                        _stream_status_markup("Error", "error"),
+                    )
+                    completed.add(source)
+                    if len(completed) == 3 and not final_sent:
+                        yield _format_sse(
+                            "search_state",
+                            _stream_status_markup("Synthesizing", "working"),
+                        )
+                        yield _format_sse(
+                            "final_html",
+                            "<div class='synthesis-stream' sse-swap='synthesis_chunk' hx-swap='beforeend'></div>",
+                        )
+                    continue
+
+                if source in {"rag", "grag", "agent"} and kind == "done":
+                    yield _format_sse(
+                        f"{source}_done",
+                        _stream_status_markup("Complete", "complete"),
+                    )
+                    completed.add(source)
+                    if len(completed) == 3 and not final_sent:
+                        yield _format_sse(
+                            "search_state",
+                            _stream_status_markup("Synthesizing", "working"),
+                        )
+                        yield _format_sse(
+                            "final_html",
+                            "<div class='synthesis-stream' sse-swap='synthesis_chunk' hx-swap='beforeend'></div>",
+                        )
+                    continue
+
+                if source == "synthesis" and kind == "chunk":
+                    yield _format_sse("synthesis_chunk", content)
+                    continue
+
+                if source == "hybrid" and kind == "timing":
+                    yield _format_sse(
+                        "total_timing",
+                        f"Total: {escape(content)}",
+                    )
+                    continue
+
+                if source == "hybrid" and kind == "final":
+                    final_sent = True
+                    # await asyncio.to_thread(cache.set, cache_key, content, timeout=3600)
+                    yield _format_sse("final_html", content)
+                    yield _format_sse(
+                        "search_state", _stream_status_markup("Complete", "complete")
+                    )
+                    yield _format_sse("stream_end", "<div></div>")
+                    return
+        except Exception as exc:
+            error_html = f"<div class='error-display'>{escape(str(exc))}</div>"
+            yield _format_sse("final_html", error_html)
+            yield _format_sse("stream_end", "<div></div>")
+
+    response = Response(
+        event_stream(),
+        mimetype="text/event-stream",
+        headers={
+            "Cache-Control": "no-cache",
+            "Connection": "keep-alive",
+            "X-Accel-Buffering": "no",
+        },
+    )
+    response.timeout = None
+    return response
+
+
+if __name__ == "__main__":
+    app.run(host="0.0.0.0", port=5001, debug=True)
diff --git a/aisearch/src/web/auth.py b/aisearch/src/web/auth.py
new file mode 100644
index 0000000..af15505
--- /dev/null
+++ b/aisearch/src/web/auth.py
@@ -0,0 +1,167 @@
+"""Authentication and authorization utilities for ai-search.
+
+This module provides JWT token validation using JWKS from the auth server.
+"""
+
+import logging
+from datetime import datetime
+from functools import wraps
+from typing import Optional
+from urllib.parse import urljoin
+
+import requests
+from authlib.jose import JsonWebKey, JsonWebToken, KeySet
+from authlib.jose.errors import BadSignatureError
+from quart import request, jsonify, current_app
+
+logger = logging.getLogger(__name__)
+
+
+class TokenValidationError(Exception):
+    """Raised when token validation fails."""
+
+    pass
+
+
+def fetch_jwks(auth_server_url: str, timeout: int = 30) -> KeySet:
+    """Fetch JSON Web Keys from the auth server."""
+    jwks_uri = urljoin(auth_server_url, "auth/public-jwks")
+    response = requests.get(jwks_uri, timeout=timeout)
+    response.raise_for_status()
+    jwks_data = response.json()
+    keys = [
+        JsonWebKey.import_key(key) for key in jwks_data.get("jwks", {}).get("keys", [])
+    ]
+    if not keys:
+        raise TokenValidationError("No keys found in JWKS response")
+    return KeySet(keys)
+
+
+def get_cached_jwks(auth_server_url: str) -> KeySet:
+    """Get cached JWKS from Redis, fetching if necessary."""
+    cache = current_app.extensions["cache"]
+    cached = cache.get("auth_server_jwks")
+
+    if cached is not None:
+        return KeySet([JsonWebKey.import_key(key) for key in cached])
+
+    # Fetch and cache for 2 hours
+    jwks = fetch_jwks(auth_server_url)
+    key_data = [key.export() for key in jwks.keys]
+    cache.set("auth_server_jwks", key_data, timeout=7200)
+    return jwks
+
+
+def validate_token(token: str, keys: KeySet) -> dict:
+    """Validate a JWT token against the provided keys.
+
+    Args:
+        token: The JWT token string
+        keys: KeySet containing valid public keys
+
+    Returns:
+        Decoded JWT payload
+
+    Raises:
+        TokenValidationError: If token is invalid or cannot be verified
+    """
+    for key in keys.keys:
+        try:
+            jwt = JsonWebToken(["RS256"])
+            payload = jwt.decode(token, key)
+
+            # Check expiration
+            if payload.get("exp") and datetime.now().timestamp() > payload["exp"]:
+                raise TokenValidationError("Token has expired")
+
+            logger.debug("Token validated successfully")
+            return payload
+        except BadSignatureError:
+            continue
+
+    raise TokenValidationError("Token signature could not be verified")
+
+
+def require_token(func):
+    """Decorator that requires a valid bearer token for the endpoint.
+
+    Expects the Authorization header with a Bearer token.
+    Adds `auth_token` kwarg to the decorated function with the decoded JWT payload.
+
+    Example:
+        @app.route("/api/v1/search")
+        @require_token
+        async def search(auth_token=None):
+            # auth_token contains the decoded JWT payload
+            user_id = auth_token.get("sub")
+            ...
+    """
+
+    @wraps(func)
+    async def wrapper(*args, **kwargs):
+        auth_server_url = current_app.config.get("AUTH_SERVER_URL")
+
+        if not auth_server_url:
+            logger.error("AUTH_SERVER_URL not configured")
+            return jsonify(
+                {
+                    "error": "ConfigurationError",
+                    "description": "Authentication not properly configured",
+                }
+            ), 500
+
+        # Extract bearer token from Authorization header
+        bearer = request.headers.get("Authorization", "")
+
+        if not bearer.startswith("Bearer "):
+            logger.debug("No bearer token provided")
+            return jsonify(
+                {
+                    "error": "AuthenticationError",
+                    "description": "Expected 'Authorization: Bearer <token>' header",
+                }
+            ), 401
+
+        try:
+            _, token = bearer.split(" ", 1)
+            token = token.strip()
+
+            # Get cached JWKS and validate token
+            jwks = get_cached_jwks(auth_server_url)
+            jwt_payload = validate_token(token, jwks)
+
+            # Add auth token info to kwargs
+            kwargs["auth_token"] = {"access_token": token, "jwt": jwt_payload}
+
+            return await func(*args, **kwargs)
+
+        except TokenValidationError as exc:
+            logger.debug("Token validation failed: %s", exc)
+            return jsonify(
+                {"error": "TokenValidationError", "description": str(exc)}
+            ), 401
+        except Exception:
+            logger.error("Unexpected error during token validation", exc_info=True)
+            return jsonify(
+                {
+                    "error": "InternalError",
+                    "description": "Failed to validate authentication token",
+                }
+            ), 500
+
+    return wrapper
+
+
+def get_user_id(auth_token: Optional[dict] = None) -> Optional[str]:
+    """Extract user ID from auth token.
+
+    Args:
+        auth_token: The auth token dict with 'jwt' key containing the payload
+
+    Returns:
+        User ID (subject claim) or None if not available
+    """
+    if auth_token is None:
+        return None
+    jwt_payload = auth_token.get("jwt", {})
+    return jwt_payload.get("sub")
diff --git a/aisearch/src/web/static/css/style.css b/aisearch/src/web/static/css/style.css
new file mode 100644
index 0000000..38fc5a5
--- /dev/null
+++ b/aisearch/src/web/static/css/style.css
@@ -0,0 +1,962 @@
+:root {
+    --bg: #fff;
+    --bg-accent: #f5f5f5;
+    --surface: #fff;
+    --surface-strong: #fff;
+    --surface-muted: #f9f9f9;
+    --line: #ddd;
+    --line-strong: #ccc;
+    --text: #000;
+    --muted: #777;
+    --accent: #3071a9;
+    --accent-strong: #245269;
+    --blue: #3071a9;
+    --success: #3c763d;
+    --warning: #8a6d3b;
+    --error: #a94442;
+    --shadow: none;
+    --shadow-soft: none;
+    --radius-xl: 24px;
+    --radius-lg: 18px;
+    --radius-md: 14px;
+}
+
+* {
+    margin: 0;
+    padding: 0;
+    box-sizing: border-box;
+}
+
+html,
+body {
+    min-height: 100%;
+}
+
+body {
+    font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+    background: #fff;
+    color: var(--text);
+    height: 100vh;
+    display: flex;
+    flex-direction: column;
+}
+
+.header {
+    padding: 1rem 1.4rem;
+    display: flex;
+    align-items: center;
+    gap: 1rem;
+    background: #222;
+    border-bottom: 1px solid #080808;
+    position: sticky;
+    top: 0;
+    z-index: 20;
+}
+
+.header-logo {
+    width: 38px;
+    height: 38px;
+    border-radius: 14px;
+    background: #3071a9;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    color: white;
+    font-weight: 700;
+    letter-spacing: 0.04em;
+}
+
+.header-title {
+    font-size: 1.05rem;
+    font-weight: 700;
+    letter-spacing: 0.02em;
+    color: #fff;
+}
+
+.chat-container {
+    flex: 1;
+    min-height: 0;
+    overflow-y: auto;
+    padding: 1.5rem 1rem 2rem;
+    max-width: 1100px;
+    margin: 0 auto;
+    width: 100%;
+    scrollbar-width: none;
+}
+
+.chat-container::-webkit-scrollbar {
+    display: none;
+}
+
+#messages {
+    display: flex;
+    flex-direction: column;
+    gap: 1.4rem;
+}
+
+.empty-state {
+    text-align: center;
+    padding: 4rem 1rem 2rem;
+    color: var(--muted);
+    max-width: 820px;
+    margin: 0 auto;
+}
+
+.empty-state h1 {
+    font-family: Georgia, "Times New Roman", serif;
+    font-size: clamp(2.6rem, 5vw, 4.5rem);
+    line-height: 0.96;
+    letter-spacing: -0.03em;
+    margin-bottom: 1rem;
+    color: var(--text);
+}
+
+.empty-state p {
+    font-size: 1.02rem;
+    margin-bottom: 0.5rem;
+}
+
+.examples {
+    margin-top: 2rem;
+    display: grid;
+    gap: 0.85rem;
+    text-align: left;
+}
+
+.example-btn {
+    background: #f9f9f9;
+    border: 1px solid #ddd;
+    border-radius: 16px;
+    padding: 1rem 1.1rem;
+    cursor: pointer;
+    color: #333;
+    font-size: 0.96rem;
+    text-align: left;
+    transition: transform 0.2s ease, border-color 0.2s ease, background 0.2s ease;
+}
+
+.example-btn:hover {
+    transform: translateY(-1px);
+    border-color: #3071a9;
+    background: #fff;
+}
+
+.message {
+    animation: fadeIn 0.28s ease;
+}
+
+@keyframes fadeIn {
+    from {
+        opacity: 0;
+        transform: translateY(8px);
+    }
+    to {
+        opacity: 1;
+        transform: translateY(0);
+    }
+}
+
+.message-user {
+    display: flex;
+    justify-content: flex-end;
+}
+
+.message-user .message-content {
+    max-width: min(78%, 760px);
+    background: #3071a9;
+    color: white;
+    border-radius: 22px 22px 8px 22px;
+    padding: 1rem 1.3rem;
+    line-height: 1.65;
+}
+
+.message-ai {
+    display: flex;
+    gap: 1rem;
+    position: relative;
+}
+
+.avatar {
+    width: 40px;
+    height: 40px;
+    border-radius: 14px;
+    background: #222;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    color: white;
+    font-weight: 700;
+    flex-shrink: 0;
+}
+
+.message-ai .message-content {
+    flex: 1;
+    background: var(--surface);
+    border: 1px solid #ddd;
+    border-radius: var(--radius-xl);
+    padding: 1.25rem;
+    line-height: 1.75;
+}
+
+.message-ai-stream .message-content {
+    padding: 1.35rem;
+}
+
+.stream-shell {
+    display: grid;
+    gap: 1rem;
+}
+
+.stream-shell-header {
+    display: flex;
+    justify-content: space-between;
+    gap: 1rem;
+    align-items: start;
+}
+
+.stream-shell-kicker,
+.stream-final-kicker {
+    text-transform: uppercase;
+    letter-spacing: 0.12em;
+    font-size: 0.72rem;
+    font-weight: 700;
+    color: var(--accent);
+}
+
+.stream-shell-header h2 {
+    margin-top: 0.25rem;
+    font-size: 1.15rem;
+    line-height: 1.2;
+}
+
+.stream-shell-intro {
+    color: var(--muted);
+    font-size: 0.96rem;
+}
+
+.stream-stage-grid {
+    display: grid;
+    grid-template-columns: repeat(3, minmax(0, 1fr));
+    gap: 0.9rem;
+}
+
+.stream-stage-card {
+    background: #fafafa;
+    border: 1px solid var(--line);
+    border-radius: var(--radius-lg);
+    padding: 1rem;
+    min-height: 220px;
+    display: flex;
+    flex-direction: column;
+    gap: 0.8rem;
+}
+
+.stream-stage-card.is-muted {
+    opacity: 0.75;
+}
+
+.stream-stage-head,
+.stream-final-head {
+    display: flex;
+    justify-content: space-between;
+    gap: 0.8rem;
+    align-items: center;
+}
+
+.stream-stage-meta,
+.stream-final-meta {
+    display: flex;
+    align-items: center;
+    gap: 0.6rem;
+}
+
+.stream-stage-timing,
+.stream-final-timing {
+    font-family: "SFMono-Regular", "Menlo", "Consolas", monospace;
+    font-size: 0.75rem;
+    color: var(--muted);
+    white-space: nowrap;
+}
+
+.stream-final-timing {
+    font-weight: 600;
+    color: var(--accent);
+}
+
+.stream-stage-name {
+    font-size: 0.95rem;
+    font-weight: 700;
+}
+
+.stream-stage-badge {
+    display: inline-flex;
+    align-items: center;
+    gap: 0.35rem;
+    font-size: 0.74rem;
+    border-radius: 999px;
+    padding: 0.28rem 0.58rem;
+    border: 1px solid transparent;
+    white-space: nowrap;
+}
+
+.stream-stage-badge.is-working {
+    color: #31708f;
+    background: #d9edf7;
+    border-color: #bce8f1;
+}
+
+.stream-stage-badge.is-working::before {
+    content: "";
+    display: inline-block;
+    width: 0.75em;
+    height: 0.75em;
+    border: 2px solid currentColor;
+    border-right-color: transparent;
+    border-radius: 50%;
+    animation: spin 1s linear infinite;
+}
+
+.stream-stage-badge.is-complete {
+    color: var(--success);
+    background: #dff0d8;
+    border-color: #d6e9c6;
+}
+
+.stream-stage-badge.is-complete::before {
+    content: "✓";
+    display: inline-flex;
+    align-items: center;
+    justify-content: center;
+    width: 1em;
+    height: 1em;
+    font-size: 0.7em;
+    font-weight: 700;
+    background: var(--success);
+    color: white;
+    border-radius: 50%;
+}
+
+.stream-stage-badge.is-error {
+    color: var(--error);
+    background: #f2dede;
+    border-color: #ebccd1;
+}
+
+.stream-stage-body-scroll {
+    flex: 1;
+    min-height: 0;
+    max-height: 220px;
+    overflow-y: auto;
+    padding-right: 0.4rem;
+    overscroll-behavior: contain;
+}
+
+.stream-stage-body-scroll::-webkit-scrollbar {
+    width: 6px;
+}
+
+.stream-stage-body-scroll::-webkit-scrollbar-thumb {
+    background: #ccc;
+    border-radius: 999px;
+}
+
+.stream-stage-body {
+    font-family: "SFMono-Regular", "Menlo", "Consolas", monospace;
+    font-size: 0.84rem;
+    line-height: 1.65;
+    color: #555;
+    white-space: pre-wrap;
+    word-break: break-word;
+}
+
+.stream-init-msg {
+    font-family: "SFMono-Regular", "Menlo", "Consolas", monospace;
+    font-size: 0.84rem;
+    color: var(--muted);
+    font-style: italic;
+    padding: 0.25rem 0;
+    animation: blink 1.4s infinite ease-in-out;
+}
+
+.stream-stage-body-scroll:has(.stream-stage-body:not(:empty)) .stream-init-msg {
+    display: none;
+}
+
+@keyframes blink {
+    0%, 100% {
+        opacity: 1;
+    }
+    50% {
+        opacity: 0.35;
+    }
+}
+
+.stream-fragment {
+    display: inline;
+}
+
+.stream-fragment-error {
+    color: var(--error);
+}
+
+.stream-status-msg {
+    font-family: "SFMono-Regular", "Menlo", "Consolas", monospace;
+    font-size: 0.84rem;
+    color: var(--muted);
+    font-style: italic;
+    padding: 0.15rem 0;
+    border-left: 2px solid var(--accent);
+    padding-left: 0.5rem;
+    margin: 0.2rem 0;
+    animation: fadeIn 0.3s ease;
+}
+
+.stream-final-card {
+    background: #fff;
+    border: 1px solid var(--line);
+    border-radius: calc(var(--radius-xl) + 2px);
+    padding: 1.2rem 1.25rem;
+}
+
+.stream-final-note {
+    font-size: 0.82rem;
+    color: var(--muted);
+}
+
+.stream-final-body {
+    margin-top: 1rem;
+}
+
+.stream-final-status {
+    border: 1px dashed #ccc;
+    border-radius: var(--radius-md);
+    padding: 1.25rem;
+    color: var(--muted);
+    background: #f9f9f9;
+    text-align: center;
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    gap: 0.5rem;
+}
+
+.stream-final-status.is-waiting {
+    animation: pulse-soft 2s infinite ease-in-out;
+}
+
+.stream-final-status.is-synthesizing {
+    animation: pulse-soft 1.5s infinite ease-in-out;
+}
+
+.stream-final-status-icon {
+    font-size: 1.5rem;
+    line-height: 1;
+    opacity: 0.85;
+}
+
+.stream-final-status-text {
+    font-size: 0.95rem;
+    font-weight: 500;
+}
+
+@keyframes pulse-soft {
+    0%, 100% {
+        opacity: 1;
+    }
+    50% {
+        opacity: 0.55;
+    }
+}
+
+.ai-response h3 {
+    color: var(--text);
+    margin: 1.5rem 0 0.8rem;
+    font-size: 1.08rem;
+    border-bottom: 1px solid var(--line);
+    padding-bottom: 0.45rem;
+}
+
+.ai-response h3:first-child {
+    margin-top: 0;
+}
+
+.summary {
+    background: #d9edf7;
+    border-left: 4px solid var(--accent);
+    padding: 1rem 1.1rem;
+    margin: 0.85rem 0 1.2rem;
+    border-radius: 0 12px 12px 0;
+    color: #245269;
+}
+
+.status-success,
+.status-partial,
+.status-error {
+    display: inline-flex;
+    align-items: center;
+    gap: 0.45rem;
+    font-size: 0.88rem;
+    margin-bottom: 0.8rem;
+    font-weight: 600;
+}
+
+.status-success {
+    color: var(--success);
+}
+
+.status-partial {
+    color: var(--warning);
+}
+
+.status-error {
+    color: var(--error);
+}
+
+.status-success::before,
+.status-partial::before,
+.status-error::before {
+    width: 18px;
+    height: 18px;
+    border-radius: 999px;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    color: white;
+    font-size: 11px;
+}
+
+.status-success::before {
+    content: "✓";
+    background: var(--success);
+}
+
+.status-partial::before {
+    content: "!";
+    background: var(--warning);
+}
+
+.status-error::before {
+    content: "✕";
+    background: var(--error);
+}
+
+.card-list {
+    list-style: none;
+    display: grid;
+    gap: 0.9rem;
+    margin-top: 0.9rem;
+}
+
+.card-item {
+    background: #fff;
+    border: 1px solid var(--line);
+    border-radius: var(--radius-md);
+    padding: 1rem;
+    transition: transform 0.2s ease, border-color 0.2s ease;
+}
+
+.card-item:hover {
+    transform: translateY(-1px);
+    border-color: #3071a9;
+}
+
+.card-title {
+    font-weight: 700;
+    color: var(--blue);
+    font-size: 1.03rem;
+    margin-bottom: 0.45rem;
+}
+
+.card-title a {
+    color: inherit;
+    text-decoration: none;
+}
+
+.card-title a:hover {
+    text-decoration: underline;
+}
+
+.card-description {
+    color: #333;
+    font-size: 0.95rem;
+    line-height: 1.6;
+    margin-bottom: 0.45rem;
+}
+
+.card-meta {
+    color: var(--muted);
+    font-size: 0.88rem;
+    font-style: italic;
+}
+
+.note-box,
+.access-note,
+.error-display,
+.raw-response {
+    border-radius: var(--radius-md);
+    padding: 1rem;
+    margin-top: 1rem;
+}
+
+.note-box {
+    background: #fcf8e3;
+    border: 1px solid #faebcc;
+    color: #8a6d3b;
+}
+
+.access-note {
+    background: #dff0d8;
+    border: 1px solid #d6e9c6;
+    color: #3c763d;
+}
+
+.error-display {
+    background: #f2dede;
+    border: 1px solid #ebccd1;
+    color: #a94442;
+}
+
+.raw-response {
+    background: #f5f5f5;
+    border: 1px solid var(--line);
+    font-family: "SFMono-Regular", "Menlo", "Consolas", monospace;
+    font-size: 0.85rem;
+    white-space: pre-wrap;
+    word-wrap: break-word;
+    max-height: 200px;
+    overflow-y: auto;
+}
+
+.loading-message {
+    display: flex;
+    align-items: center;
+    gap: 1rem;
+    color: var(--muted);
+    padding: 0.8rem 0.2rem;
+}
+
+.loading-dots {
+    display: flex;
+    gap: 4px;
+}
+
+.loading-dots span {
+    width: 8px;
+    height: 8px;
+    background: var(--accent);
+    border-radius: 50%;
+    animation: bounce 1.4s infinite ease-in-out;
+}
+
+.loading-dots span:nth-child(2) {
+    animation-delay: 0.2s;
+}
+
+.loading-dots span:nth-child(3) {
+    animation-delay: 0.4s;
+}
+
+@keyframes spin {
+    to {
+        transform: rotate(360deg);
+    }
+}
+
+@keyframes bounce {
+    0%, 80%, 100% {
+        transform: scale(0.2);
+        opacity: 0.5;
+    }
+    40% {
+        transform: scale(1);
+        opacity: 1;
+    }
+}
+
+.htmx-indicator {
+    display: none !important;
+}
+
+.htmx-request.htmx-indicator {
+    display: flex !important;
+}
+
+.input-container {
+    padding: 1rem 1rem 1.5rem;
+    background: #fff;
+    border-top: 1px solid #f0f0f0;
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    gap: 0.25rem;
+}
+
+.input-wrapper {
+    max-width: 768px;
+    width: 100%;
+    margin: 0 auto;
+    display: flex;
+    align-items: flex-end;
+    gap: 0.5rem;
+    background: #fff;
+    border: 1px solid #e5e5e5;
+    border-radius: 24px;
+    padding: 0.5rem 0.6rem 0.5rem 1.25rem;
+    box-shadow:
+        0 2px 12px rgba(0, 0, 0, 0.06),
+        0 0 0 1px rgba(0, 0, 0, 0.02);
+    transition: box-shadow 0.2s ease, border-color 0.2s ease, transform 0.2s ease;
+}
+
+.input-wrapper:focus-within {
+    box-shadow:
+        0 2px 16px rgba(48, 113, 169, 0.10),
+        0 0 0 1px rgba(48, 113, 169, 0.15);
+    border-color: rgba(48, 113, 169, 0.25);
+}
+
+.textarea-box {
+    flex: 1;
+    position: relative;
+    min-width: 0;
+    display: flex;
+    align-items: flex-start;
+}
+
+.search-input {
+    width: 100%;
+    min-height: 44px;
+    max-height: 200px;
+    padding: 10px 28px 10px 0;
+    font-size: 16px;
+    line-height: 1.5;
+    border: none;
+    background: transparent;
+    outline: none;
+    color: #222;
+    resize: none;
+    overflow-y: hidden;
+    font-family: inherit;
+}
+
+.search-input::placeholder {
+    color: #999;
+}
+
+.search-input:disabled {
+    opacity: 0.5;
+    cursor: not-allowed;
+}
+
+.clear-btn {
+    position: absolute;
+    right: 0;
+    top: 8px;
+    background: none;
+    border: none;
+    color: #999;
+    font-size: 1.2rem;
+    line-height: 1;
+    cursor: pointer;
+    width: 24px;
+    height: 24px;
+    display: none;
+    align-items: center;
+    justify-content: center;
+    border-radius: 50%;
+    transition: color 0.15s ease, background 0.15s ease;
+}
+
+.clear-btn:hover {
+    color: #666;
+    background: #f0f0f0;
+}
+
+.search-button,
+.stop-button {
+    width: 36px;
+    height: 36px;
+    border-radius: 50%;
+    border: none;
+    color: white;
+    cursor: pointer;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    flex-shrink: 0;
+    transition: transform 0.15s ease, background 0.15s ease, opacity 0.15s ease;
+    margin-bottom: 4px;
+}
+
+.search-button {
+    background: #3071a9;
+    font-size: 1.1rem;
+}
+
+.search-button:hover {
+    transform: scale(1.06);
+    background: #245269;
+}
+
+.search-button:disabled {
+    opacity: 0.4;
+    cursor: not-allowed;
+}
+
+.stop-button {
+    background: #e53935;
+    display: none;
+}
+
+.stop-button:hover {
+    transform: scale(1.06);
+    background: #c62828;
+}
+
+.stop-button:active {
+    transform: scale(0.95);
+    background: #b71c1c;
+}
+
+.char-counter {
+    max-width: 768px;
+    width: 100%;
+    margin: 0 auto;
+    text-align: right;
+    font-size: 0.75rem;
+    color: #999;
+    padding-right: 0.75rem;
+    min-height: 1.2em;
+}
+
+.char-counter.caution {
+    color: #f5a623;
+}
+
+.char-counter.warning {
+    color: #e53935;
+}
+
+.copy-btn {
+    position: absolute;
+    top: 0.75rem;
+    right: 0.75rem;
+    background: #fff;
+    border: 1px solid var(--line);
+    border-radius: 999px;
+    padding: 0.36rem 0.7rem;
+    font-size: 0.78rem;
+    color: var(--muted);
+    cursor: pointer;
+    opacity: 0;
+    transition: opacity 0.2s ease, background 0.2s ease;
+}
+
+.message-ai:hover .copy-btn {
+    opacity: 1;
+}
+
+.copy-btn:hover {
+    background: white;
+}
+
+.scroll-to-bottom-btn {
+    position: fixed;
+    bottom: 100px;
+    right: 24px;
+    z-index: 100;
+    background: var(--accent);
+    color: white;
+    border: none;
+    border-radius: 50%;
+    width: 44px;
+    height: 44px;
+    font-size: 1.2rem;
+    line-height: 1;
+    cursor: pointer;
+    opacity: 0;
+    pointer-events: none;
+    transition: opacity 0.2s ease, transform 0.2s ease, background 0.2s ease;
+    box-shadow: 0 2px 10px rgba(0, 0, 0, 0.18);
+    display: flex;
+    align-items: center;
+    justify-content: center;
+}
+
+.scroll-to-bottom-btn.visible {
+    opacity: 1;
+    pointer-events: auto;
+}
+
+.scroll-to-bottom-btn:hover {
+    transform: scale(1.06);
+    background: var(--accent-strong);
+}
+
+.error-message {
+    background: #f2dede;
+    border: 1px solid #ebccd1;
+    border-radius: var(--radius-md);
+    padding: 1rem;
+    color: #a94442;
+}
+
+@media (max-width: 920px) {
+    .stream-stage-grid {
+        grid-template-columns: 1fr;
+    }
+
+    .stream-stage-body-scroll {
+        max-height: 180px;
+    }
+
+    .message-user .message-content {
+        max-width: 92%;
+    }
+}
+
+@media (max-width: 640px) {
+    .header {
+        padding: 0.85rem 1rem;
+    }
+
+    .chat-container {
+        padding: 1rem 0.75rem 1.5rem;
+    }
+
+    .message-ai {
+        gap: 0.75rem;
+    }
+
+    .avatar {
+        width: 34px;
+        height: 34px;
+        border-radius: 12px;
+    }
+
+    .message-ai .message-content,
+    .message-user .message-content {
+        padding: 1rem;
+    }
+
+    .stream-shell-header,
+    .stream-final-head {
+        flex-direction: column;
+        align-items: flex-start;
+    }
+
+    .copy-btn {
+        position: static;
+        opacity: 1;
+        margin-top: 0.75rem;
+    }
+
+    .input-container {
+        padding: 0.75rem;
+    }
+
+    .input-wrapper {
+        border-radius: 20px;
+        padding: 0.4rem 0.5rem 0.4rem 1rem;
+    }
+}
diff --git a/aisearch/src/web/templates/index.html b/aisearch/src/web/templates/index.html
new file mode 100644
index 0000000..02d61f1
--- /dev/null
+++ b/aisearch/src/web/templates/index.html
@@ -0,0 +1,233 @@
+<!DOCTYPE html>
+<html lang="en">
+    <head>
+        <meta charset="utf-8">
+        <meta name="viewport" content="width=device-width, initial-scale=1.0">
+        <title>GeneNetwork AI Search</title>
+        <script src="https://unpkg.com/htmx.org@2.0.8"></script>
+        <script src="https://unpkg.com/htmx-ext-sse@2.2.4/sse.js"></script>
+        <script src="https://unpkg.com/hyperscript.org@0.9.12"></script>
+        <link rel="stylesheet"
+              href="{{ url_for('static', filename='css/style.css') }}">
+    </head>
+    <body>
+        <div class="header">
+            <div class="header-logo">GN</div>
+            <div class="header-title">GeneNetwork AI Search</div>
+        </div>
+        <div class="chat-container" id="chat-container">
+            <div class="empty-state" id="empty-state">
+                <h1>GeneNetwork AI Search</h1>
+                <p>Ask natural language questions about genes, traits, and datasets</p>
+                <p style="font-size: 0.9em; color: #999;">Powered by Hybrid RAG + Agent</p>
+                <div class="examples">
+                    <button type="button"
+                            class="example-btn"
+                            _="on click set #search-input.value to 'What genes are associated with diabetes?' then call #search-input.focus()">
+                        What genes are associated with diabetes?
+                    </button>
+                    <button type="button"
+                            class="example-btn"
+                            _="on click set #search-input.value to 'Find datasets related to obesity' then call #search-input.focus()">
+                        Find datasets related to obesity
+                    </button>
+                    <button type="button"
+                            class="example-btn"
+                            _="on click set #search-input.value to 'How does the mosaic structure of BXD recombinant inbred mice genomes affect mapping resolution across generations?' then call #search-input.focus()">
+                        How does the mosaic structure of BXD recombinant inbred mice genomes affect mapping resolution across generations?
+                    </button>
+                </div>
+            </div>
+            <!-- messages wrapper -->
+            <div id="messages"></div>
+            <!-- indicator sits after messages so it appears at the bottom -->
+            <div id="loading-indicator" class="message message-ai htmx-indicator">
+                <div class="avatar">AI</div>
+                <div class="message-content">
+                    <div class="loading-message">
+                        <div class="loading-dots">
+                            <span></span><span></span><span></span>
+                        </div>
+                        <span>Searching GeneNetwork data...</span>
+                    </div>
+                </div>
+            </div>
+        </div>
+        <div class="input-container">
+            <form class="input-wrapper"
+                  id="search-form"
+                  hx-get="/search/stream-shell"
+                  hx-target="#messages"
+                  hx-swap="beforeend"
+                  hx-indicator="#loading-indicator"
+                  _="on htmx:beforeRequest
+                         if no #search-input.value.trim() halt end
+                         put 'none' into #empty-state's style.display
+                         on htmx:afterRequest
+                         put '' into #search-input.value
+                         call #search-input.focus()">
+                <div class="textarea-box">
+                    <textarea name="q"
+                              id="search-input"
+                              class="search-input"
+                              placeholder="Ask a question about genes, traits, or datasets... (Shift+Enter for new line)"
+                              rows="1"
+                              autocomplete="off"
+                              autofocus></textarea>
+                    <button type="button"
+                            class="clear-btn"
+                            id="clear-btn"
+                            title="Clear message"
+                            aria-label="Clear message">×</button>
+                </div>
+                <button type="submit"
+                        class="search-button"
+                        id="search-btn"
+                        aria-label="Send"
+                        title="Send">↑</button>
+                <button type="button"
+                        class="stop-button"
+                        id="stop-btn"
+                        aria-label="Stop generating"
+                        title="Stop generating">
+                    <svg viewBox="0 0 24 24" width="16" height="16" fill="currentColor" aria-hidden="true">
+                        <rect x="6" y="6" width="12" height="12" rx="2" />
+                    </svg>
+                </button>
+            </form>
+            <div class="char-counter" id="char-counter">0 / 1000</div>
+        </div>
+        <button id="scroll-to-bottom" class="scroll-to-bottom-btn" title="Scroll to bottom">↓</button>
+
+        <script type="text/hyperscript">
+         def focusInput()
+         call #search-input.focus()
+         end
+        </script>
+        <script>
+        (function(){
+            const chat = document.getElementById('chat-container');
+            const btn = document.getElementById('scroll-to-bottom');
+            const messages = document.getElementById('messages');
+            if (!chat || !btn || !messages) return;
+            const THRESHOLD = 80;
+            let atBottom = true;
+
+            function check() {
+                atBottom = chat.scrollHeight - chat.scrollTop <= chat.clientHeight + THRESHOLD;
+                btn.classList.toggle('visible', !atBottom);
+            }
+
+            chat.addEventListener('scroll', check);
+            window.addEventListener('resize', check);
+
+            new MutationObserver(function(muts) {
+                var hit = muts.some(function(m) {
+                    return m.target.closest && m.target.closest('.stream-final-body');
+                });
+                if (!hit) return;
+                if (atBottom) chat.scrollTop = chat.scrollHeight;
+                else btn.classList.add('visible');
+            }).observe(messages, { childList: true, subtree: true });
+
+            btn.addEventListener('click', function(){
+                chat.scrollTop = chat.scrollHeight;
+                atBottom = true;
+                btn.classList.remove('visible');
+            });
+        })();
+        </script>
+        <script>
+        (function(){
+            const input = document.getElementById('search-input');
+            const sendBtn = document.getElementById('search-btn');
+            const stopBtn = document.getElementById('stop-btn');
+            const clearBtn = document.getElementById('clear-btn');
+            const charCounter = document.getElementById('char-counter');
+            const messages = document.getElementById('messages');
+            const form = document.getElementById('search-form');
+            if (!input || !sendBtn || !stopBtn || !messages || !form) return;
+            const MAX_HEIGHT = 200;
+            const MAX_CHARS = 1000;
+
+            function autoGrow() {
+                input.style.height = 'auto';
+                var h = Math.min(input.scrollHeight, MAX_HEIGHT);
+                input.style.height = h + 'px';
+                input.style.overflowY = input.scrollHeight > MAX_HEIGHT ? 'auto' : 'hidden';
+            }
+
+            function updateUI() {
+                var len = input.value.length;
+                if (charCounter) {
+                    charCounter.textContent = len + ' / ' + MAX_CHARS;
+                    charCounter.classList.remove('warning', 'caution');
+                    if (len > 900) charCounter.classList.add('warning');
+                    else if (len > 700) charCounter.classList.add('caution');
+                }
+                if (clearBtn) clearBtn.style.display = len > 0 ? 'flex' : 'none';
+            }
+
+            function setStreaming(v) {
+                input.disabled = v;
+                sendBtn.style.display = v ? 'none' : 'flex';
+                stopBtn.style.display = v ? 'flex' : 'none';
+                if (!v) {
+                    input.focus();
+                    autoGrow();
+                    updateUI();
+                }
+            }
+
+            window.onStreamEnd = function() { setStreaming(false); };
+
+            form.addEventListener('htmx:afterRequest', function(evt) {
+                if (evt.detail.successful) setStreaming(true);
+                setTimeout(function() {
+                    autoGrow();
+                    updateUI();
+                }, 0);
+            });
+
+            stopBtn.addEventListener('click', function() {
+                var lastMsg = messages.lastElementChild;
+                if (lastMsg) lastMsg.remove();
+                setStreaming(false);
+            });
+
+            if (clearBtn) {
+                clearBtn.addEventListener('click', function() {
+                    input.value = '';
+                    input.focus();
+                    autoGrow();
+                    updateUI();
+                });
+            }
+
+            input.addEventListener('input', function() {
+                autoGrow();
+                updateUI();
+            });
+
+            input.addEventListener('keydown', function(e) {
+                if (e.key !== 'Enter') return;
+                if (e.shiftKey) return;
+                e.preventDefault();
+                if (!input.disabled && input.value.trim()) {
+                    form.requestSubmit();
+                }
+            });
+
+            input.addEventListener('paste', function() {
+                setTimeout(function() {
+                    autoGrow();
+                    updateUI();
+                }, 0);
+            });
+
+            autoGrow();
+            updateUI();
+        })();
+        </script>
+    </body>
+</html>
diff --git a/aisearch/src/web/templates/login.html b/aisearch/src/web/templates/login.html
new file mode 100644
index 0000000..1b87523
--- /dev/null
+++ b/aisearch/src/web/templates/login.html
@@ -0,0 +1,187 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>GeneNetwork AI Search – Sign In</title>
+    <script src="https://unpkg.com/htmx.org@2.0.8"></script>
+    <script src="https://unpkg.com/htmx-ext-sse@2.2.4/sse.js"></script>
+    <script src="https://unpkg.com/hyperscript.org@0.9.12"></script>
+    <link rel="stylesheet" href="{{ url_for('static', filename='css/style.css') }}">
+    <style>
+        /* === Login Page Specific Styles === */
+        * {
+            margin: 0;
+            padding: 0;
+            box-sizing: border-box;
+        }
+
+        body {
+            font-family: system-ui, -apple-system, 'Segoe UI', Roboto, 'Helvetica Neue', sans-serif;
+            background: linear-gradient(135deg, #0f172a 0%, #1e293b 100%);
+            min-height: 100vh;
+            display: flex;
+            align-items: center;
+            justify-content: center;
+            padding: 1.5rem;
+        }
+
+        .login-container {
+            width: 100%;
+            max-width: 440px;
+        }
+
+        .login-card {
+            background: rgba(255, 255, 255, 0.05);
+            backdrop-filter: blur(10px);
+            border-radius: 2rem;
+            padding: 2rem 1.8rem 2.2rem 1.8rem;
+            box-shadow: 0 25px 45px -12px rgba(0, 0, 0, 0.4), 0 0 0 1px rgba(255, 255, 255, 0.08);
+            transition: transform 0.2s ease;
+        }
+
+        .login-card:hover {
+            transform: translateY(-3px);
+        }
+
+        .brand {
+            text-align: center;
+            margin-bottom: 2rem;
+        }
+
+        .brand h1 {
+            font-size: 1.9rem;
+            font-weight: 600;
+            background: linear-gradient(120deg, #c084fc, #60a5fa);
+            background-clip: text;
+            -webkit-background-clip: text;
+            color: transparent;
+            letter-spacing: -0.3px;
+        }
+
+        .brand p {
+            color: #94a3b8;
+            font-size: 0.9rem;
+            margin-top: 0.5rem;
+        }
+
+        .input-group {
+            margin-bottom: 1.5rem;
+        }
+
+        .input-group label {
+            display: block;
+            color: #e2e8f0;
+            font-size: 0.85rem;
+            font-weight: 500;
+            margin-bottom: 0.4rem;
+            letter-spacing: 0.3px;
+        }
+
+        .input-group input {
+            width: 100%;
+            padding: 0.85rem 1rem;
+            background: rgba(15, 23, 42, 0.6);
+            border: 1px solid #334155;
+            border-radius: 1rem;
+            font-size: 1rem;
+            color: #f1f5f9;
+            transition: all 0.2s ease;
+            outline: none;
+        }
+
+        .input-group input:focus {
+            border-color: #818cf8;
+            box-shadow: 0 0 0 3px rgba(129, 140, 248, 0.3);
+            background: rgba(30, 41, 59, 0.8);
+        }
+
+        .input-group input::placeholder {
+            color: #64748b;
+            font-size: 0.9rem;
+        }
+
+        button {
+            width: 100%;
+            background: linear-gradient(95deg, #4f46e5, #7c3aed);
+            border: none;
+            padding: 0.85rem;
+            border-radius: 1.5rem;
+            font-weight: 600;
+            font-size: 1rem;
+            color: white;
+            cursor: pointer;
+            transition: all 0.2s ease;
+            margin-top: 0.5rem;
+            box-shadow: 0 4px 12px rgba(79, 70, 229, 0.3);
+        }
+
+        button:hover {
+            background: linear-gradient(95deg, #6366f1, #8b5cf6);
+            transform: scale(0.98);
+            box-shadow: 0 6px 16px rgba(79, 70, 229, 0.4);
+        }
+
+        .footer-note {
+            text-align: center;
+            margin-top: 1.8rem;
+            font-size: 0.75rem;
+            color: #64748b;
+        }
+
+        .footer-note a {
+            color: #a78bfa;
+            text-decoration: none;
+        }
+
+        .footer-note a:hover {
+            text-decoration: underline;
+        }
+
+        /* Simple error styling (if any flash messages appear) */
+        .error-message {
+            background: rgba(239, 68, 68, 0.15);
+            border-left: 4px solid #ef4444;
+            padding: 0.75rem;
+            border-radius: 0.75rem;
+            margin-bottom: 1.5rem;
+            color: #fecaca;
+            font-size: 0.85rem;
+        }
+    </style>
+</head>
+<body>
+    <div class="login-container">
+        <div class="login-card">
+            <div class="brand">
+                <h1> GeneNetwork</h1>
+                <p>AI-powered genomics search</p>
+            </div>
+
+            <!-- Show error if provided via query param or flash (optional) -->
+            {% with messages = get_flashed_messages(with_categories=true) %}
+                {% if messages %}
+                    {% for category, message in messages %}
+                        <div class="error-message">{{ message }}</div>
+                    {% endfor %}
+                {% endif %}
+            {% endwith %}
+
+            <form method="post" action="/login">
+                <div class="input-group">
+                    <label>📧 Email address</label>
+                    <input type="email" name="email" placeholder="you@example.org" required autofocus>
+                </div>
+                <div class="input-group">
+                    <label>🔒 Password</label>
+                    <input type="password" name="password" placeholder="••••••••" required>
+                </div>
+                <button type="submit">Sign in →</button>
+            </form>
+            <div class="footer-note">
+                Secure access · <a href="#">Request access</a>
+            </div>
+        </div>
+    </div>
+</body>
+</html>
diff --git a/aisearch/src/web/templates/partials/response.html b/aisearch/src/web/templates/partials/response.html
new file mode 100644
index 0000000..cc8a0fd
--- /dev/null
+++ b/aisearch/src/web/templates/partials/response.html
@@ -0,0 +1,12 @@
+<div class="message message-ai">
+    <div class="avatar">AI</div>
+    <div class="message-content">
+        {% include "partials/response_body.html" %}
+    </div>
+    <button type="button" class="copy-btn"
+            _="on click
+                writeText(my previousElementSibling's innerText) on navigator.clipboard
+                put 'Copied!' into me
+                wait 1.5s
+                put 'Copy' into me">Copy</button>
+</div>
diff --git a/aisearch/src/web/templates/partials/response_body.html b/aisearch/src/web/templates/partials/response_body.html
new file mode 100644
index 0000000..4b305b5
--- /dev/null
+++ b/aisearch/src/web/templates/partials/response_body.html
@@ -0,0 +1,55 @@
+<div class="ai-response">
+    {% if data.error or data.status == 'error' %}
+        <div class="status-error">Search encountered an error</div>
+        <div class="error-display">{{ data.error or data.message or 'Unknown error' }}</div>
+        {% if data.raw_response %}
+            <div class="raw-response">
+                <strong>Raw Response:</strong><br>
+                {{ data.raw_response }}
+            </div>
+        {% endif %}
+    {% else %}
+        {% if data.status == 'success' %}
+            <div class="status-success">Search completed successfully</div>
+        {% elif data.status == 'partial_success' %}
+            <div class="status-partial">Partial success - some data sources unavailable</div>
+        {% endif %}
+
+        {% if data.summary %}
+            <div class="summary">{{ data.summary }}</div>
+        {% endif %}
+
+        {% if grouped and grouped|length > 0 %}
+            {% for type, items in grouped.items() %}
+                {% set display_type = type[0]|upper + type[1:] + 's' %}
+                <h3>{{ display_type }} ({{ items|length }})</h3>
+                <ul class="card-list">
+                    {% for item in items %}
+                        {% set url = item.url or item.link or '#' %}
+                        {% set name = item.name or item.symbol or item.trait_name or 'Unnamed' %}
+                        <li class="card-item">
+                            <div class="card-title">
+                                <a href="{{ url }}" target="_blank">{{ name }}</a>
+                            </div>
+                            {% if item.description %}
+                                <div class="card-description">{{ item.description }}</div>
+                            {% endif %}
+                            {% if item.extra %}
+                                <div class="card-meta">{{ item.extra }}</div>
+                            {% endif %}
+                        </li>
+                    {% endfor %}
+                </ul>
+            {% endfor %}
+        {% else %}
+            <div class="note-box">No specific results were found for this query.</div>
+        {% endif %}
+
+        {% if data.note %}
+            <div class="note-box">{{ data.note }}</div>
+        {% endif %}
+        {% if data.access_note %}
+            <div class="access-note">{{ data.access_note }}</div>
+        {% endif %}
+    {% endif %}
+</div>
diff --git a/aisearch/src/web/templates/partials/stream_shell.html b/aisearch/src/web/templates/partials/stream_shell.html
new file mode 100644
index 0000000..24d4ffe
--- /dev/null
+++ b/aisearch/src/web/templates/partials/stream_shell.html
@@ -0,0 +1,95 @@
+<div class="message message-user">
+    <div class="message-content">
+        {{ query }}
+    </div>
+</div>
+
+<div class="message message-ai message-ai-stream">
+    <div class="avatar">AI</div>
+    <div class="message-content stream-shell"
+         hx-ext="sse"
+         sse-connect="{{ url_for('search_stream') }}?q={{ query|urlencode }}"
+         sse-close="stream_end">
+        <div class="stream-shell-header">
+            <div>
+                <div class="stream-shell-kicker">Live Research Trace</div>
+                <h2>Cross-checking RAG, GraphRAG, and AgentSearch</h2>
+            </div>
+            <div class="stream-shell-state" sse-swap="search_state" hx-swap="innerHTML">
+                <span class="stream-stage-badge is-working">Starting</span>
+            </div>
+        </div>
+
+        <div class="stream-shell-intro">
+            Intermediate system output stays muted while the final answer is assembled.
+        </div>
+
+        <div class="stream-stage-grid">
+            <section class="stream-stage-card is-muted">
+                <div class="stream-stage-head">
+                    <span class="stream-stage-name">RAGSearch</span>
+                    <div class="stream-stage-meta">
+                        <span class="stream-stage-timing" sse-swap="rag_timing" hx-swap="innerHTML"></span>
+                        <span class="stream-stage-state" sse-swap="rag_done" hx-swap="innerHTML" hx-on:htmx:after-swap="this.closest('.stream-stage-card').querySelector('.stream-init-msg').style.display='none'">
+                            <span class="stream-stage-badge is-working">Running</span>
+                        </span>
+                    </div>
+                </div>
+                <div class="stream-stage-body-scroll" sse-swap="rag_final_html" hx-target="find .stream-stage-body" hx-swap="innerHTML" hx-on:htmx:after-swap="this.scrollTop = this.scrollHeight">
+                    <div class="stream-init-msg">Initializing search…</div>
+                    <div class="stream-stage-body" sse-swap="rag_chunk" hx-target="this" hx-swap="beforeend" hx-on:htmx:after-swap="this.parentElement.scrollTop = this.parentElement.scrollHeight"></div>
+                </div>
+            </section>
+
+            <section class="stream-stage-card is-muted">
+                <div class="stream-stage-head">
+                    <span class="stream-stage-name">GraphRAGSearch</span>
+                    <div class="stream-stage-meta">
+                        <span class="stream-stage-timing" sse-swap="grag_timing" hx-swap="innerHTML"></span>
+                        <span class="stream-stage-state" sse-swap="grag_done" hx-swap="innerHTML" hx-on:htmx:after-swap="this.closest('.stream-stage-card').querySelector('.stream-init-msg').style.display='none'">
+                            <span class="stream-stage-badge is-working">Running</span>
+                        </span>
+                    </div>
+                </div>
+                <div class="stream-stage-body-scroll" sse-swap="grag_final_html" hx-target="find .stream-stage-body" hx-swap="innerHTML" hx-on:htmx:after-swap="this.scrollTop = this.scrollHeight">
+                    <div class="stream-init-msg">Initializing search…</div>
+                    <div class="stream-stage-body" sse-swap="grag_chunk" hx-target="this" hx-swap="beforeend" hx-on:htmx:after-swap="this.parentElement.scrollTop = this.parentElement.scrollHeight"></div>
+                </div>
+            </section>
+
+            <section class="stream-stage-card is-muted">
+                <div class="stream-stage-head">
+                    <span class="stream-stage-name">AgentSearch</span>
+                    <div class="stream-stage-meta">
+                        <span class="stream-stage-timing" sse-swap="agent_timing" hx-swap="innerHTML"></span>
+                        <span class="stream-stage-state" sse-swap="agent_done" hx-swap="innerHTML" hx-on:htmx:after-swap="this.closest('.stream-stage-card').querySelector('.stream-init-msg').style.display='none'">
+                            <span class="stream-stage-badge is-working">Running</span>
+                        </span>
+                    </div>
+                </div>
+                <div class="stream-stage-body-scroll" sse-swap="agent_final_html" hx-target="find .stream-stage-body" hx-swap="innerHTML" hx-on:htmx:after-swap="this.scrollTop = this.scrollHeight">
+                    <div class="stream-init-msg">Initializing search…</div>
+                    <div class="stream-stage-body" sse-swap="agent_chunk" hx-target="this" hx-swap="beforeend" hx-on:htmx:after-swap="this.parentElement.scrollTop = this.parentElement.scrollHeight"></div>
+                </div>
+            </section>
+        </div>
+
+        <section class="stream-final-card">
+            <div class="stream-final-head">
+                <span class="stream-final-kicker">Final Answer</span>
+                <div class="stream-final-meta">
+                    <span class="stream-final-timing" sse-swap="total_timing" hx-swap="innerHTML"></span>
+                    <span class="stream-final-note">Rendered after the three systems complete</span>
+                </div>
+            </div>
+            <div class="stream-final-body" sse-swap="final_html" hx-swap="innerHTML">
+                <div class="stream-final-status is-waiting">
+                    <div class="stream-final-status-icon">🔍</div>
+                    <div class="stream-final-status-text">Waiting for searches to complete…</div>
+                </div>
+            </div>
+        </section>
+
+        <div hidden sse-swap="stream_end" hx-on:htmx:after-swap="window.onStreamEnd()"></div>
+    </div>
+</div>