Uploaded image for project: 'Magnolia Test Framework'
  1. Magnolia Test Framework
  2. MGNLTEST-47

New mgnl page objects may have "timing issues" - analyze and fix if required

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • 1.0
    • None
    • None

      Context

      While writing tests, it turned out that "we" (actually me, chm) have a lot of issues which seem related with timing. (Typically tests were failing with ElementNotVisibleException or NoSuchElementException)
      While I 1st thought, that this is all related to my env. (docker on mac), some posts on slack from Rico let me think, that it could indeed be a probelm within the API.
      I wrote a test to test whether the API has timing issues. (See below)


      Testing the hypothesis

      Test class which creates JCR content, see https://gist.github.com/Watcher24/24a57798641897470b5ff4e489415a0e

      • 2 of the methods create config:/modules/foobar
      • 2 of the methods create config:/modules/foobar/goodies/@beer
      • 2 methods using the API as “expected” in the (+/-) “most fluent way”.
      • 2 of the methods have some additional calls are meant to delay things a bit.

      Test runs: 3x with 10 reps (=30 reps for each method).

      Results: (✔︎/✖︎)

      • #createContent_withArtificalStops
        • 9/1, 9/1, 10/0 -> 28/2
      • #createContent_fullyFluent
        • 10/0, 9/1, 10/0 -> 29/1
      • #createMoreContent_withArtificalStops
        • 8/2, 10/0, 9/1 -> 27/3
      • #createMoreContent_fullyFluent
        • 5/5, 3/7, 3/7 -> 11/19

      (Also see there for more details regarding the test results.)

      Conclusion

      Execution of #createMoreContent_fullyFluent fails much too much (19 failures on 30 runs).
      Since #createMoreContent_withArtificalStops - which aims to do the same thing but using "artifical delays" - is much more successful, I state herby that we do have timing issues.
      Anyhow ... further analysis is required.


      Tasks

      • Further analyze the problem.
      • Add additional self-tests testing such timing issues.
        • Some self-tests should contain multiple calls of "events" which may need a "waiting period"
      • Add additional "waitings" inside page objects where it is required.

      Basically make sure the API can be used the "fluent way" it was built for - without the need of adding "artifical delays".


      Outcome

      • Created an @ImplicitWaitTimeout annotation to increase WebDriver's implicit wait timeout when needed
      • Added an ExpectedCondition called vaadinIsDone() allowing to suspend Selenium client code execution until Vaadin's client-server communication is done (possibly the major cause of intermittent failures requiring some sort of delay)
      • Hardened Grid row selection by using ExpectedConditions.refreshed(..) so to avoid StaleElementReferenceException

        Acceptance criteria

              fgrilli Federico Grilli
              cmeier Christoph Meier
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved:

                  Task DoR