[MGNLTEST-47] New mgnl page objects may have "timing issues" - analyze and fix if required Created: 09/Oct/19 Updated: 08/Jul/20 Resolved: 13/Jan/20 |
|
| Status: | Closed |
| Project: | Magnolia Test Framework |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 1.0 |
| Type: | Task | Priority: | Blocker |
| Reporter: | Christoph Meier | Assignee: | Federico Grilli |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Template: |
|
| Acceptance criteria: |
Empty
|
| Task DoR: |
Empty
|
| Date of First Response: | |
| Epic Link: | core-TF-features-bugs-improvements |
| Story Points: | 8 |
| Description |
ContextWhile writing tests, it turned out that "we" (actually me, chm) have a lot of issues which seem related with timing. (Typically tests were failing with ElementNotVisibleException or NoSuchElementException) Testing the hypothesisTest class which creates JCR content, see https://gist.github.com/Watcher24/24a57798641897470b5ff4e489415a0e
Test runs: 3x with 10 reps (=30 reps for each method). Results: (✔︎/✖︎)
(Also see there for more details regarding the test results.) Conclusion Execution of #createMoreContent_fullyFluent fails much too much (19 failures on 30 runs). Tasks
Basically make sure the API can be used the "fluent way" it was built for - without the need of adding "artifical delays". Outcome
|
| Comments |
| Comment by Christoph Meier [ 18/Oct/19 ] |
|
Here is another example / a few lines of codes, where tests always fail:
contentApp.clickRow(titleInitialVersion)
.hitAction("Publish")
.hitAction("Edit page")
.hitAction("Edit page properties");
The "problem" is publish which will may take some time plus displays a "banner" ("Successfully bla bla ..."). In the given case, I don't know how to avoid the failure (besides an artificial wait via debugger). Not sure what would be "ideal" here. hitAction(String actionLabel, int waitAfterward) Not the most elegant way ... but probably a possibility?
|
| Comment by Federico Grilli [ 18/Oct/19 ] |
|
For the record, I'm reporting here parts of the discussion we had on Slack rico christoph rico christoph |
| Comment by Christoph Meier [ 18/Dec/19 ] |
|
Time boxed for the time being to 5SP. |
| Comment by Federico Grilli [ 02/Jan/20 ] |
|
Some results after the latest hardening contained in the PR. It appears that some flakiness in tests can't be completely and reliably avoided (network issues for instance). 1st run: 55/60 tests passed, 5 failed Methods prone to failure
Surefire may help us here https://maven.apache.org/surefire/maven-surefire-plugin/examples/rerun-failing-tests.html especially since version 3.0.0-M4 and https://issues.apache.org/jira/browse/SUREFIRE-1584 |
| Comment by Christoph Meier [ 13/Jan/20 ] |
|
Follow-up: |