[MGNLTEST-132] Parallelize test runs for test framework Created: 17/Dec/20  Updated: 16/Nov/23

Status: Selected
Project: Magnolia Test Framework
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Neutral
Reporter: Rishab Dhar Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: TAE
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
relation
is related to MGNLTEST-31 Enable parallel execution of JUnit 5 ... Closed
Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Date of First Response:
Epic Link: TAE-build-or-infra-tasks
Team: Foundation
Work Started:

 Description   

Currently the test framework utilizes the moduleUiTestsPipeline to run UI tests on Jenkins on a single magnolia instance. The average build times for PRs are now approaching 20 minutes, with more POs in the queue to be added.

  • Consider adding configurable parameters to the Jenkins pipeline to enable multi instance setup to speed up the builds.
  • This will also require enabling the forked VM setup for test framework (for which a parent pom should be updated)
  • Selenium standalone needs to be dropped from docker-compose in favor of selenium hub + node setup (with port range mappings for exposed ports on both magnolia and selenium node containers)

Implementation notes

  • replicate IT setup from non-TF pipelines
  • possible Failsafe / JUnit5 issue to enable parallelism?


 Comments   
Comment by Maxime Michel [ 15/Nov/21 ]

apchelintcev I vaguely remember working on this with you, we were struggling to configure parallelization for the Jupiter engine in the Surefire plugin? (While it went OK with the vintage engine.) Do you recall anything in this regard, could it be that it is no longer a problem by now? Thanks.

Comment by Aleksandr Pchelintcev [ 22/Nov/21 ]

To be completely honest my vague recollection was that we managed to get it working without major issues. I will have to look again though.

Comment by Aleksandr Pchelintcev [ 22/Nov/21 ]

There's a forked execution surefire config available in ce/ee:

  <profiles>
    <!-- This profile will run ui-tests on a real browser - use to run them on your machine -->
    <profile>
      <!--
      Concurrent execution profile kicks in when the corresponding property is specified.
      It configures to run the failsafe plugin to run in a forked mode and augments the
      server host name making it fork number-specific.

      This profile is a logical addition to the ui-tests profile, i.e. it is pointless
      to try to run this one with the latter disabled.
      -->
      <id>forked-test-execution</id>
      <activation>
        <property>
          <name>concurrent.instances</name>
        </property>
      </activation>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <configuration>
              <workingDirectory>${project.build.directory}/test-runner-${surefire.forkNumber}</workingDirectory>
              <forkCount>${concurrent.instances}</forkCount>
              <reuseForks>true</reuseForks>
              <argLine>-Xmx2048m</argLine>
              <systemPropertyVariables>
                <fork.number>$${surefire.forkNumber}</fork.number>
              </systemPropertyVariables>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

  </profiles>

Is it confirmed that it works only for the junit4 tests?

Comment by Maxime Michel [ 22/Nov/21 ]

so I think that forks should work, the parallel test execution (where you run a couple of test cases from same suite) isn’t that required/relevant

Generated at Mon Feb 12 07:45:52 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.