Uploaded image for project: 'Magnolia Test Framework'
  1. Magnolia Test Framework
  2. MGNLTEST-50

Resolve timing-related failures in new UI tests

    XMLWordPrintable

Details

    • Task
    • Resolution: Not an issue
    • Neutral
    • None
    • None
    • None

    Description

      Some page object self tests in the new UI test framework seem to fail rather frequently, especially on Jenkins, namely

      info.magnolia.testframework.selftest.ContentAppSelfTest#unselectRowByPath

      info.magnolia.testframework.selftest.MediaEditorSelfTest#mediaEditorPerformsAction

      info.magnolia.testframework.selftest.AlertNotificationSelfTest#alertConfirmAndNotification

      See also https://jenkins.magnolia-cms.com/blue/organizations/jenkins/ce/detail/MGNLCE-166-page-extraction

      Increasing the wait timeout for some possibly long actions, such as when hitting the "save" or "commit" button in forms or alerts only partially helps.

      Locally, I observed that sometimes such tests fail the first time but pass the second time around, if one repeats them. Would be nice to have a @RetryOnFailure annotation also for JUnit5. There actually is a native @Repeated one but it doesn't take failures into account.

      Also tried some 3rd party JUnit extensions as suggested here https://www.swtestacademy.com/junit-5-how-to-repeat-failed-test/ but sadly they didn't work that great.

      Checklists

        Acceptance criteria

        Attachments

          Issue Links

            Activity

              People

                Unassigned Unassigned
                fgrilli Federico Grilli
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  Checklists

                    Task DoR