summaryrefslogtreecommitdiff
path: root/doc/merge.txt
diff options
context:
space:
mode:
Diffstat (limited to 'doc/merge.txt')
-rw-r--r--doc/merge.txt31
1 files changed, 31 insertions, 0 deletions
diff --git a/doc/merge.txt b/doc/merge.txt
new file mode 100644
index 0000000..28757a7
--- /dev/null
+++ b/doc/merge.txt
@@ -0,0 +1,31 @@
+-*- markdown -*-
+
+The merge process
+=================
+
+- merge-fetch maintains a single file 'fetched' referring to a given
+ entry in 'logorder', indicating which entries are fetched and
+ sequenced so far.
+
+- merge-backup reads 'fetched' and pushes these entries to secondary
+ merge nodes, maintaining one file per secondary,
+ 'backup.<secondary>', indicating how many entries that have been
+ copied to and verified at the secondary in question.
+
+- merge-sth writes a new 'sth' file by reading the
+ 'backup.<secondary>' files into a list, picking a new tree size by
+ sorting the list (in falling order) and indexing it with the
+ 'backupquorum' config option. If the new tree size is smaller than
+ what the old 'sth' file says, no new STH is created.
+
+- merge-dist distributes 'sth' and missing entries to frontend nodes.
+
+TODO
+====
+
+- Run the three pieces in parallell.
+
+- Improve merge-fetch by parallellising it using one process per
+storage node writing to a "queue info" file (storage-node, hash) and a
+single "queue handling process" reading queue files and writing to the
+'fetched' file.