[MGNLTEST-164] RenderedPage not reachable (or too late) Created: 04/May/21  Updated: 14/Jul/21  Resolved: 28/Jun/21

Status: Closed
Project: Magnolia Test Framework
Component/s: None
Affects Version/s: 1.1.0, 2.0-cloud
Fix Version/s: 1.2.1

Type: Bug Priority: Critical
Reporter: Christoph Meier Assignee: Martin Haderka
Resolution: Resolved Votes: 0
Labels: QA&Testing, VN-Maintenance, shaky-test
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to MGNLTEST-165 contentApp.hasRow("yet-another-page")... Closed
Template:
Acceptance criteria:
Empty
Task DoD:
[X]* Doc/release notes changes? Comment present?
[X]* Downstream builds green?
[X]* Solution information and context easily available?
[X]* Tests
[X]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Bug DoR:
[ ]* Steps to reproduce, expected, and actual results filled
[ ]* Affected version filled
Date of First Response:
Epic Link: Fn tests spike 2
Sprint: UI FW 30, UI FW 31
Story Points: 5

 Description   

We have been using the RenderedPage for assertions in several tests in PagesCoreFunctionalTests (and PagesSpaFunctionalTests). While this worked fine for some time, this changed recently.

It's possible that this is related to the fact that we had to change the version of ubuntu on AWS. The ubuntu version 18 we are using now has been reported to be significantly slower than previous versions.

Tests have been "fixed" / adjusted by doing assertions with selenium-independent assertions using the MagnoliaHttpClient (instead of the the RenderedPage).

Acceptance criteria

  • "Adjust" the test-fwk in a way that RenderedPage again can be used as a trustworthy way to make assertions.
  • Make sure there is a self-test checking that the RenderedPage works as expected.

Example test, which worked "before" and now fails, but which should work again from PagesCoreFunctionalTests):

    @Test
    @Fixture(fixtureFile = "website.lorem-ipsum.yaml", repository = "website")
    @Cleanup("website:/lorem-ipsum")
    void addComponent(PageObjects expect, Navigator navigator) {
        loginAndOpenPagesApp(expect, Personas.ERIC);

        pagesApp.clickRow("lorem-ipsum")
                .hitAction("Edit page");

        PageEditor pageEditor = expect.pageEditor("Lorem ipsum");
        // check whether the footer is still "empty"
        assertThrows(NoSuchElementException.class, () -> pageEditor.selectArea("Footer").selectComponent("Text and image"));

        pageEditor.hitAction("Add component");
        expect.form("Add component")
                .setComboBoxOption("Type of component", "Text and image")
                .clickButton("Next");
        expect.form("Text and image")
                .setTextField("Headline (en)", "Do Re Mi")
                .save();

        navigator.to(DIRECT_AUTHOR, "lorem-ipsum");
        assertTrue(expect.renderedPage("Lorem ipsum").hasText("Do Re Mi"));
    }

 

Developers note

The issue simply spoken seems to be a timing issue. 
A fix may likely requires code which checks whether certain condition is already checked/true.
Note that we already do this to a certain extent, not within RenderedPage itself, but on PageObjects#renderedPage(string)

I wonder whether it would be "nicer" to have the conditional logic directly within RenderedPage ... but not yet sure whether that's easy doable.

In case you need to add more "waiting" / "conditional waiting" - consider using the https://github.com/awaitility/awaitility framework.

While it is legit to use the MagnoliaHttpClient to make assertions, RenderedPage must work.
The former for instance is okay to check the content of a page from pages-app in PagesCoreFunctionalTests. But the client cannot used to check the content in the context of SPA. The http response body of a page done with SPA template(s) only contains a bit of JavaScript but not the content to check.

Here's a possible reason why Ubuntu 18 is "slower" than previous versions:
https://github.com/opencontainers/runc/pull/2921 (via Maxime).

 

 



 Comments   
Comment by Christoph Meier [ 04/May/21 ]

mmichel, I remember you told me that the Ubuntu version 18, which we use now, has been reported as being slower than previous working, you even sent me the link to an issue tracker (on GIT) about this.
Unfortunately  cannot find your post about this anymore.
Could you please add that link from Ubuntu issue tracker here?

Comment by Maxime Michel [ 04/May/21 ]

It's the following issue: https://github.com/docker/for-linux/issues/738

However, I have yet to see how much of an impact it has on our test suite. Although people do report performance issues, there is a lot of uncertainty around the actual cause of the problem.

Comment by Roman Kovařík [ 08/Jun/21 ]

This seems suspicious to me:

private void refreshUntil(Supplier<Boolean> condition) {
        eventually(() -> {
            if (condition.get()) {
                return true;
            } else {
                driver.navigate().refresh();
                return false;
            }
        });
    }

Not sure if the refresh is intentional. If there was a timeout issue when rendering the page, isn't there a chance that by refreshing the page we just repeat the same timeout issue over and over again? Should we just wait instead of refresh?

Generated at Mon Feb 12 07:46:12 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.