Uploaded image for project: 'External Enterprise Search'
  1. External Enterprise Search
  2. ESRCH-45

Index consistency on JVM crash

    XMLWordPrintable

Details

    • Task
    • Resolution: Obsolete
    • Neutral
    • None
    • None
    • None

    Description

      Goal

      Clarify expectations, goals and options wrt. keeping the ES index consistent.

      Problem

      The SearchIndex interface does not give us consistency guarantees beyond those that we have for the integrated Lucene index in Jackrabbit. After a server / JVM crash the index can still go out of sync, requiring a full re-index.

      Solution sketch

      To improve on index resilience we would need to:

      • Put changes on a persisted indexing queue before they are stored in JCR
      • Process the changes from the indexing queue after they have been stored in JCR:
        • if storing was successful, send the changes to ES and remove them from the indexing queue
        • if storing was not successful, remove the changes from the indexing queue
        • A non empty indexing queue on a server restart indicates there was an abnormal termination and index recovery is needed: determine whether the changes in the indexing queue are present in JCR and only send them to ES if not.

      Implementation considerations

      Checklists

        Acceptance criteria

        Attachments

          Activity

            People

              mduerig Michael Duerig
              mduerig Michael Duerig
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Checklists

                  Task DoR