Uploaded image for project: 'Magnolia'
  1. Magnolia
  2. MAGNOLIA-8760

Usage of PostgreSQL to publish heavy data loads increases toast tables size too much

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • None
    • 6.2.28
    • None

    Description

      It has been reported that using PostgreSQL and trying to publish any high amount of data/number of sites or pages will make the toast table increase in size up to occupying the whole available DB.

      This issue has been reported on PostgreSQL 11.9 when the user tried to publish 8000 sites and ended up filling 15GB of DB available size when trying to publish 120MB of data.

      You can find the used configuration parameters attached to this ticket.

      There are some workarounds that can be used, like performing an export/import of the data for a first heavy load publication instead of performing a normal publish, executing a vacuum of the toast table after the publish or directly increasing the DB size so it's not filled during the publication.

      Checklists

        Acceptance criteria

        Attachments

          1. 4_5k.png
            4_5k.png
            275 kB
          2. after_vacuum.png
            after_vacuum.png
            76 kB
          3. BatchPublicationCommand.java
            3 kB
          4. before_vacuum.png
            before_vacuum.png
            36 kB
          5. chart.png
            chart.png
            269 kB
          6. image-2023-02-27-15-30-37-729.png
            image-2023-02-27-15-30-37-729.png
            36 kB
          7. image-2023-02-27-15-31-26-772.png
            image-2023-02-27-15-31-26-772.png
            20 kB
          8. image-2023-02-28-10-15-31-357.png
            image-2023-02-28-10-15-31-357.png
            374 kB
          9. image-2023-02-28-10-16-05-662.png
            image-2023-02-28-10-16-05-662.png
            77 kB
          10. image-2023-03-01-17-22-16-421.png
            image-2023-03-01-17-22-16-421.png
            580 kB
          11. pg_config_parameters_public.txt
            37 kB
          12. pom.xml
            6 kB
          13. postgres-repo-conf.xml
            5 kB
          14. repositories.xml
            2 kB

          Issue Links

            Activity

              People

                dalonso Daniel Alonso
                rgaona Roberto Gaona
                Christian Menzel, Fabian Mangold, Jan Haderka
                Nucleus
                Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:
                  Work Started:

                  Checklists

                    Bug DoR
                    Task DoD