Uploaded image for project: 'Magnolia'
  1. Magnolia
  2. MAGNOLIA-8760

Usage of PostgreSQL to publish heavy data loads increases toast tables size too much

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • None
    • 6.2.28
    • None

      It has been reported that using PostgreSQL and trying to publish any high amount of data/number of sites or pages will make the toast table increase in size up to occupying the whole available DB.

      This issue has been reported on PostgreSQL 11.9 when the user tried to publish 8000 sites and ended up filling 15GB of DB available size when trying to publish 120MB of data.

      You can find the used configuration parameters attached to this ticket.

      There are some workarounds that can be used, like performing an export/import of the data for a first heavy load publication instead of performing a normal publish, executing a vacuum of the toast table after the publish or directly increasing the DB size so it's not filled during the publication.

        Acceptance criteria

          1. 4_5k.png
            4_5k.png
            275 kB
          2. after_vacuum.png
            after_vacuum.png
            76 kB
          3. BatchPublicationCommand.java
            3 kB
          4. before_vacuum.png
            before_vacuum.png
            36 kB
          5. chart.png
            chart.png
            269 kB
          6. image-2023-02-27-15-30-37-729.png
            image-2023-02-27-15-30-37-729.png
            36 kB
          7. image-2023-02-27-15-31-26-772.png
            image-2023-02-27-15-31-26-772.png
            20 kB
          8. image-2023-02-28-10-15-31-357.png
            image-2023-02-28-10-15-31-357.png
            374 kB
          9. image-2023-02-28-10-16-05-662.png
            image-2023-02-28-10-16-05-662.png
            77 kB
          10. image-2023-03-01-17-22-16-421.png
            image-2023-03-01-17-22-16-421.png
            580 kB
          11. pg_config_parameters_public.txt
            37 kB
          12. pom.xml
            6 kB
          13. postgres-repo-conf.xml
            5 kB
          14. repositories.xml
            2 kB

              dalonso Daniel Alonso
              rgaona Roberto Gaona
              Christian Menzel, Fabian Mangold, Jan Haderka
              Nucleus
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Created:
                Updated:
                Resolved:
                Work Started:

                  Bug DoR
                  Task DoD