From edb4668735704b01b2f974066e6d0515fa9738af Mon Sep 17 00:00:00 2001 From: Arun Isaac Date: Thu, 1 Jun 2023 23:18:18 +0100 Subject: issues: Rebuild xapian index conditionally. --- ...pian-index-rebuild-conditional-on-database-checksums.gmi | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 issues/make-xapian-index-rebuild-conditional-on-database-checksums.gmi diff --git a/issues/make-xapian-index-rebuild-conditional-on-database-checksums.gmi b/issues/make-xapian-index-rebuild-conditional-on-database-checksums.gmi new file mode 100644 index 0000000..decc02e --- /dev/null +++ b/issues/make-xapian-index-rebuild-conditional-on-database-checksums.gmi @@ -0,0 +1,13 @@ +# Make xapian index rebuild conditional on database checksums + +* assigned: arun + +Currently, we unconditionally rebuild the xapian index once every day regardless of whether the database has actually changed over the last day. Not only is this computationally wasteful, but it also means that we cannot respond to database updates quickly enough. + +We need to be rebuilding the xapian index only when the database changes. Here's how: + +* [x] Lock the xapian index being built so that two build jobs do not run simultaneously. +* [x] Register database table checksums in the xapian index. +* [ ] Trigger an index rebuild only when the checksums have changed. + +Only task 3 remains. Most of the code for task 3 is done. But, it depends on guile-dbi, and unfortunately, the Guix guile-dbi package only builds with guile-2.2; it does not build with guile-3.0. Hopefully, the fix is simple. Fingers crossed. -- cgit v1.2.3