Evaluation code for paper 1

1 files changed, 14 insertions, 0 deletions

diff --git a/gnqa/paper1_eval/src/data/queries/query_generation_prompt.md b/gnqa/paper1_eval/src/data/queries/query_generation_prompt.md
new file mode 100644
index 0000000..5b09832
--- /dev/null
+++ b/gnqa/paper1_eval/src/data/queries/query_generation_prompt.md
@@ -0,0 +1,14 @@
+# OpenAI gpt4o Query Generation prompt
+
+## System settings	
+There is a retrieval augmented generation system, called GNQA, that holds a corpus of 3000 research documents. The documents span the topics of research related to genenetwork.org, research about the genetics and genomics of diabetes and aging. The systems topics will be referred to as GN, aging, and sugah. Two types of individuals query GNQA, citizen scientists and domain experts. A citizen scientist is someone with no more than undergraduate level understanding of biology and is someone who did not major or minor in biology. A domain expert has studied advanced biology and has a graduate degree in a type of biology or majored in biology for undergraduate school.
+	
+## User messages
+
+Generate 20 questions, for GNQA, about research on GN from the perspective of a citizen scientist.
+Generate 20 questions, for GNQA, about research on GN from the perspective of a domain expert.
+Generate 20 questions, for GNQA, about research on aging from the perspective of a domain expert.
+Generate 20 questions, for GNQA, about research on aging from the perspective of a citizen scientist.
+Generate 20 questions, for GNQA, about research on sugah from the perspective of domain expert.
+Generate 20 questions, for GNQA, about research on sugah from the perspective of citizen scientist.
+