[MGNLTEST-48] Port SeleniumNodesVncSidekickRule to JUnit-5 / new ui-tests-framework Created: 09/Oct/19  Updated: 07/Jul/20  Resolved: 07/Jan/20

Status: Closed
Project: Magnolia Test Framework
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Neutral
Reporter: Christoph Meier Assignee: Federico Grilli
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Date of First Response:
Epic Link: core-TF-features-bugs-improvements
Story Points: 3

 Description   

We have the class info.magnolia.integrationtests.rules.SeleniumNodesVncSidekickRule which makes sure that screen-castings (videos) of failed tests are saved in the directory ce/magnolia-integration-tests/tests/target/failsafe-reports/recordings. (I assume there is also a similar directoy in platform/dx-core/integration-tests/tests too.)

Currently this "rule" is registered in the class info.magnolia.integrationtests.uitest.AbstractMagnoliaUITest - which is the base class for the UI-tests of the "old ui-test-framework".

Task
Make sure that the JUnit-5 ui-tests using the New Magnolia ui-tests-framework also store these recordings in case a ui tests fails.



 Comments   
Comment by Federico Grilli [ 13/Dec/19 ]

In the old testing framework, recordings are complemented by srt files logging test execution events especially in case of elements not found or displayed. Such subtitles can be combined with the videos in order to make it easier spotting the possible cause of a test failure.
Events are logged by a JUnit4 rule using info.magnolia.integrationtests.rules.ExecutionLog.

Such rule won't be ported because events are mostly logged by our own info.magnolia.integrationtests.rules.SeleniumDriver#getElement(org.openqa.selenium.By), whereas in the new framework page objects use plain Selenium WebDriver to find elements.

Video recordings and stack traces are enough in most cases to determine the cause of failure (imho, the subtitles don't add much anyway).

Comment by Christoph Meier [ 16/Dec/19 ]

fgrilli, would it be possible to provide an example of "this"

... recordings are complemented by srt files logging test execution events especially in case of elements not found or displayed. Such subtitles can be combined with the videos in order to make it easier spotting the possible cause of a test failure.

Just to get an idea.

Comment by Federico Grilli [ 16/Dec/19 ]

For instance https://jenkins.magnolia-cms.com/job/ce/job/master/2104/ - if you expand the build artifacts view you'll see that mp4 files usually have a matching srt file (e.g. downloadMultipleLogFiles.flv.mp4 and downloadMultipleLogFiles.srt)

Comment by Christoph Meier [ 17/Dec/19 ]

Alright. Now I have understood and meanwhile also seen a few "srt files".
For me it's fine if we want have these files anymore. Their value is debatable.

However, I propose to use biunique file names for the video files, which currently reflect the method name only.
Such video-file-name could be:

fully_qualified_class_name-methodName
-> example -> 
info_magnolia_testframework_selftest_AppLauncherTest-nonSuperuserCannotAccessSecurityApp.
(Will add that to the PR too.)

Generated at Mon Feb 12 07:45:03 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.