Details
-
Task
-
Resolution: Obsolete
-
Neutral
-
None
-
None
-
None
-
-
Empty show more show less
-
Empty show more show less
Description
Goal
Clarify expectations, goals and options wrt. keeping the ES index consistent.
Problem
The SearchIndex interface does not give us consistency guarantees beyond those that we have for the integrated Lucene index in Jackrabbit. After a server / JVM crash the index can still go out of sync, requiring a full re-index.
Solution sketch
To improve on index resilience we would need to:
- Put changes on a persisted indexing queue before they are stored in JCR
- Process the changes from the indexing queue after they have been stored in JCR:
- if storing was successful, send the changes to ES and remove them from the indexing queue
- if storing was not successful, remove the changes from the indexing queue
- A non empty indexing queue on a server restart indicates there was an abnormal termination and index recovery is needed: determine whether the changes in the indexing queue are present in JCR and only send them to ES if not.
Implementation considerations
- Option 1: wrap the configured persistent manager and override the store method to intercept the change log before and after changes are stored to JCR.
- Option 2: indexing could work like a passive cluster node following the cluster journal. (See ClusterNode, Journal, UpdateEventChannel)
Checklists
Acceptance criteria