Uploaded image for project: 'Magnolia'
  1. Magnolia
  2. MAGNOLIA-6775

Filesystem of public instances keeps growing exponentially

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • None
    • None
    • None
    • None

    Description

      Situation:
      As of Thursday, ~08:00 AM our three public instances started growing rapidly.
      As you can see from pbwvl-ascms01.filesystem.png, the filesystem grew several GB until wednesday, where the instances acutally stopped working.
      We extended the filesystem by another 30GB and rebooted all three instances.
      After the reboot, the use of the filesystem dropped back to a reasonable level.
      However, the filesystem started to grow again and keeps growing exponentially.
      We rebooted all three instances again and the use of the filesystem dropped again.
      This problem only occurs on the public instances.
      We are running Magnolia 5.3.10 Enterprise Edition - Professional on a Tomcat 8.0.28 ("About Magnolia" is attached to this ticket)

      Additional information from our system admins:
      The datastore inside the repository folder seems to be the one growing unnaturally, but this is yet to be confirmed.

      27G /opt/tomcat/instances/ascms03/magnolia/repositories/public/magnolia/repository

      27G /opt/tomcat/instances/ascms03/magnolia/repositories/public/magnolia/repository/datastore
      12K /opt/tomcat/instances/ascms03/magnolia/repositories/public/magnolia/repository/namespaces
      4.0K /opt/tomcat/instances/ascms03/magnolia/repositories/public/magnolia/repository/privileges
      36K /opt/tomcat/instances/ascms03/magnolia/repositories/public/magnolia/repository/nodetypes
      8.0K /opt/tomcat/instances/ascms03/magnolia/repositories/public/magnolia/repository/meta
      27G /opt/tomcat/instances/ascms03/magnolia/repositories/public/magnolia/repository

      We also attached our current catalina.out and magnolia-error.log from one of our public instances.

      Expectations:

      • We need immediate help
      • We need to find out, what is growing so fast
      • We need to understand, why it is growing so fast
      • We need to know, what we can do about it, because as the situation is right now, we won't make it through the weekend and we cannot keep rebooting our public instances frequently
      • Feel free to contact Steffen Brand (+49 5139 898 - 1725) or Andreas Christmann (+49 5139 898 - 5134) via phone
      • Use entwicklungcms@rossmann.de for email communication

      Checklists

        Acceptance criteria

        Attachments

          Activity

            People

              Unassigned Unassigned
              rossmann Dirk Rossmann GmbH
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Checklists

                  Bug DoR
                  Task DoD