summaryrefslogtreecommitdiff
path: root/issues
diff options
context:
space:
mode:
authorPjotr Prins2021-11-20 12:45:16 +0100
committerPjotr Prins2021-11-20 12:45:16 +0100
commit1d1d0f4f448a2b0e15f10a3a7030d93f955cb421 (patch)
treeeda94f6f67f627747a9920fd7bc887b1c5417b3d /issues
parent78143cb46fb98b2152d9e40d5f885e5eea2414c3 (diff)
downloadgn-gemtext-1d1d0f4f448a2b0e15f10a3a7030d93f955cb421.tar.gz
gemma-wrapper: add lock support
Diffstat (limited to 'issues')
-rw-r--r--issues/gemma/gemma-wrapper-has-incomplete-files.gmi4
1 files changed, 4 insertions, 0 deletions
diff --git a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi
index 4bea71d..3c9a7ad 100644
--- a/issues/gemma/gemma-wrapper-has-incomplete-files.gmi
+++ b/issues/gemma/gemma-wrapper-has-incomplete-files.gmi
@@ -17,6 +17,10 @@ GNU parallel can fail, but does not tell how individual processes did. Need to c
Turns out GNU parallel can keep track of jobs in a job log - and even rerun the ones missing. The last we don't need because we are using a cache. But we can use the log file to remove any incomplete output files!
+## Dealing with locks
+
There is another parallel issue (pun intended) where gemma-wrapper is invoked twice for the same job. This is quite possible when people get impatient waiting for a first job to finish.
One solution is to write a lock file using the inputs as a hash. The lock file can contain a PID and we can check if that is still alive. I should do the same for sheepdog locks(!)
+
+=> https://github.com/genetics-statistics/gemma-wrapper/commit/e7e516ec5a6ffc5b398302fa204685a40e76e171 Added locking support