[MGNLTEST-133] Run self tests against multiple magnolia versions Created: 17/Dec/20  Updated: 16/Nov/23

Status: Open
Project: Magnolia Test Framework
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Neutral
Reporter: Rishab Dhar Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: TAE
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Date of First Response:
Epic Link: TAE-build-or-infra-tasks
Team: Foundation

 Description   

The self tests right now run always against the latest snapshot of magnolia. Often times, self tests start failing with newer snapshots. In order to disambiguate the root cause of the failures, consider parameterizing test runs against older release and the newer snapshot on Jenkins.

Implementation notes

This would be easy to do using Jenkins' matrix directive: https://www.jenkins.io/blog/2019/11/22/welcome-to-the-matrix/

If the pipeline was able to automatically resolve the Magnolia bundle version before the current SNAPSHOT, it could create a simple matrix and run tests on both bundles. This would apply to all jobs that use the moduleUiTestsPipeline, but that wouldn't be a problem?

this would make builds way slower, find out when to run it (PR builds?) and not (canonical branches).



 Comments   
Comment by Christoph Meier [ 09/Nov/23 ]

Tp precise tha a bit:
atm - core-TF selft-tests are run against latest 6.2-SNAPSHOT dx-core-test-webapp ... actually inly using the author.

Since we soon split into

  • 2.0-SNAPSHOT, java11, selenium and selenide latest version which require java11
  • 1.4-SNAPSHOT, java8, selenium-, selenide-version which still support java 8

The former we will run against 6.3-SNAPSHOT
The latter we will run against 6.2-SNAPSHOT

Once we have the above mentioned change in place - would that sufficient, rdhar  ?

Generated at Mon Feb 12 07:45:53 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.