[MGNLUI-7749] Reduce grid refresh rate Created: 18/Jan/23  Updated: 14/Aug/23  Resolved: 08/Feb/23

Status: Closed
Project: Magnolia UI
Component/s: None
Affects Version/s: 6.2.27
Fix Version/s: 6.2.29

Type: Bug Priority: High
Reporter: Richard Gange Assignee: Sang Ngo Huu
Resolution: Fixed Votes: 0
Labels: cs-bk
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: 0.75d Time Spent: 0.25d
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Attachments: File create-a-lot-of-pages.groovy     File extensiveUiRreshes.groovy     File magnolia-ui-framework-jcr-6.2.25-MGNLUI-7749.jar     Text File scratch_3.txt    
Issue Links:
Relates
relates to MGNLUI-6797 Server push configuration Closed
relates to MAGNOLIA-9050 Improve UI push scalability Open
relation
Sub-Tasks:
Key
Summary
Type
Status
Assignee
MGNLUI-7799 Implementation Sub-task Completed Roman Kovařík  
MGNLUI-7800 Rw Sub-task Completed Quach Hao Thien  
MGNLUI-7801 Preint QA Sub-task Completed Quach Hao Thien  
MGNLUI-7802 QA + PM Sub-task Completed Sang Ngo Huu  
MGNLUI-7823 Test the patch Sub-task Completed Rabie Hayoun  
Template:
Acceptance criteria:
Empty
Task DoD:
[X]* Doc/release notes changes? Comment present?
[X]* Downstream builds green?
[X]* Solution information and context easily available?
[X]* Tests
[X]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Bug DoR:
[X]* Steps to reproduce, expected, and actual results filled
[X]* Affected version filled
Date of First Response:
Epic Link: Datasource observation
Sprint: Nucleus 29
Story Points: 3
Team: Nucleus
Work Started:

 Description   

Steps to reproduce

  1. http://localhost:8080/magnoliaAuthor/.magnolia/admincentral#app:groovy:detail;/untitled:edit
  2. Paste the content of attached script extensiveUiRreshes.groovy
  3. Click run
  4. Open http://localhost:8080/magnoliaAuthor/.magnolia/admincentral#app:pages-app:browser in multiple browser tabs

Expected results

UI is still usable.

Actual results

Server under huge stress, user can't interact with UI, stackoverflow in logs.

Development notes

When lot's of users are working in the same app/workspace the grid refresh can become overwhelmed with requests. This can lead to a stack overflow.

2023-01-17 10:35:21,782 ERROR nfo.magnolia.admincentral.AdmincentralErrorHandler: Admincentral ist auf einen unerwarteten Fehler gestoßen.
java.util.concurrent.ExecutionException: java.lang.StackOverflowError
	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_77]
	at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_77]
	at com.vaadin.server.VaadinSession$FutureAccess.get(VaadinSession.java:114) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.server.VaadinService.runPendingAccessTasks(VaadinService.java:2098) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.ui.UI.push(UI.java:1701) ~[vaadin-server-8.15.2.jar:8.15.2]
	at info.magnolia.admincentral.ResurfaceUI.push(ResurfaceUI.java:125) ~[magnolia-admincentral-6.2.23.jar:?]
	at info.magnolia.ui.contentapp.observation.JcrDataSourceObservation.lambda$null$0(JcrDataSourceObservation.java:80) ~[magnolia-ui-framework-jcr-6.2.25.jar:?]
	at info.magnolia.admincentral.ResurfaceUI.lambda$accessSynchronously$2(ResurfaceUI.java:141) ~[magnolia-admincentral-6.2.23.jar:?]
	at com.vaadin.ui.UI.accessSynchronously(UI.java:1511) ~[vaadin-server-8.15.2.jar:8.15.2]
	at info.magnolia.admincentral.ResurfaceUI.accessSynchronously(ResurfaceUI.java:131) ~[magnolia-admincentral-6.2.23.jar:?]
	at com.vaadin.ui.UI$3.run(UI.java:1575) ~[vaadin-server-8.15.2.jar:8.15.2]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_77]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_77]
	at com.vaadin.server.VaadinService.runPendingAccessTasks(VaadinService.java:2095) ~[vaadin-server-8.15.2.jar:8.15.2]
... 
...
...
Caused by: java.lang.StackOverflowError
	at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3615) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3604) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4155) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:926) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:2051) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3538) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:489) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3240) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2411) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2834) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2838) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2082) ~[mysql-connector-java-5.1.30.jar:?]
	at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1307) ~[mysql-connector-java-5.1.30.jar:?]
	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
	at org.apache.jackrabbit.core.util.db.ConnectionHelper.execute(ConnectionHelper.java:524) ~[jackrabbit-data-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.util.db.ConnectionHelper.reallyExec(ConnectionHelper.java:411) ~[jackrabbit-data-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.util.db.ConnectionHelper$3.call(ConnectionHelper.java:385) ~[jackrabbit-data-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.util.db.ConnectionHelper$3.call(ConnectionHelper.java:381) ~[jackrabbit-data-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.util.db.ConnectionHelper$RetryManager.doTry(ConnectionHelper.java:552) ~[jackrabbit-data-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.util.db.ConnectionHelper.exec(ConnectionHelper.java:388) ~[jackrabbit-data-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager.loadBundle(BundleDbPersistenceManager.java:886) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.getBundleCacheMiss(AbstractBundlePersistenceManager.java:766) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.getBundle(AbstractBundlePersistenceManager.java:750) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager.load(AbstractBundlePersistenceManager.java:490) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.state.SharedItemStateManager.loadItemState(SharedItemStateManager.java:1878) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.state.SharedItemStateManager.getNonVirtualItemState(SharedItemStateManager.java:1798) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.state.SharedItemStateManager.getItemState(SharedItemStateManager.java:271) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.state.LocalItemStateManager.getNodeState(LocalItemStateManager.java:110) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.state.LocalItemStateManager.getItemState(LocalItemStateManager.java:175) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.state.SessionItemStateManager.getItemState(SessionItemStateManager.java:161) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.getItemState(HierarchyManagerImpl.java:154) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(HierarchyManagerImpl.java:285) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.CachingHierarchyManager.buildPath(CachingHierarchyManager.java:232) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(HierarchyManagerImpl.java:287) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.CachingHierarchyManager.buildPath(CachingHierarchyManager.java:232) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(HierarchyManagerImpl.java:287) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.CachingHierarchyManager.buildPath(CachingHierarchyManager.java:232) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(HierarchyManagerImpl.java:287) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.CachingHierarchyManager.buildPath(CachingHierarchyManager.java:232) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(HierarchyManagerImpl.java:287) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.CachingHierarchyManager.buildPath(CachingHierarchyManager.java:232) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.buildPath(HierarchyManagerImpl.java:287) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.CachingHierarchyManager.buildPath(CachingHierarchyManager.java:232) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.HierarchyManagerImpl.getPath(HierarchyManagerImpl.java:402) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.CachingHierarchyManager.getPath(CachingHierarchyManager.java:266) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at info.magnolia.cms.core.DefaultACLBasedPermissions.canRead(DefaultACLBasedPermissions.java:135) ~[magnolia-core-6.2.25.jar:?]
	at org.apache.jackrabbit.core.security.DefaultAccessManager.canRead(DefaultAccessManager.java:265) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.ItemManager.canRead(ItemManager.java:437) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.ItemManager.createItemData(ItemManager.java:847) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.ItemManager.getItemData(ItemManager.java:391) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.ItemManager.getItem(ItemManager.java:328) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.ItemManager.getItem(ItemManager.java:621) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.SessionImpl.getNodeById(SessionImpl.java:538) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.query.lucene.LuceneQueryFactory.execute(LuceneQueryFactory.java:229) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.query.lucene.join.QueryEngine.execute(QueryEngine.java:465) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.query.lucene.join.QueryEngine.execute(QueryEngine.java:126) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.query.lucene.join.QueryEngine.execute(QueryEngine.java:115) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.query.QueryObjectModelImpl$2.perform(QueryObjectModelImpl.java:129) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.query.QueryObjectModelImpl$2.perform(QueryObjectModelImpl.java:124) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.session.SessionState.perform(SessionState.java:216) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at org.apache.jackrabbit.core.query.QueryObjectModelImpl.execute(QueryObjectModelImpl.java:123) ~[jackrabbit-core-2.20.6.jar:2.20.6]
	at info.magnolia.ui.contentapp.JcrDataProvider.fetchFromBackEnd(JcrDataProvider.java:87) ~[magnolia-ui-framework-jcr-6.2.25.jar:?]
	at info.magnolia.ui.contentapp.JcrDataProvider.sizeInBackEnd(JcrDataProvider.java:103) ~[magnolia-ui-framework-jcr-6.2.25.jar:?]
	at com.vaadin.data.provider.AbstractBackEndDataProvider.size(AbstractBackEndDataProvider.java:66) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.data.provider.DataProviderWrapper.size(DataProviderWrapper.java:85) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.data.provider.DataCommunicator.getDataProviderSize(DataCommunicator.java:840) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.data.provider.DataCommunicator.sendDataToClient(DataCommunicator.java:384) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.data.provider.DataCommunicator.beforeClientResponse(DataCommunicator.java:364) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.server.communication.UidlWriter.write(UidlWriter.java:126) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.server.communication.AtmospherePushConnection.push(AtmospherePushConnection.java:168) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.server.communication.AtmospherePushConnection.push(AtmospherePushConnection.java:146) ~[vaadin-server-8.15.2.jar:8.15.2]
	at com.vaadin.ui.UI.push(UI.java:1708) ~[vaadin-server-8.15.2.jar:8.15.2]
	at info.magnolia.admincentral.ResurfaceUI.push(ResurfaceUI.java:125) ~[magnolia-admincentral-6.2.23.jar:?]
	at info.magnolia.ui.contentapp.observation.JcrDataSourceObservation.lambda$null$0(JcrDataSourceObservation.java:80) ~[magnolia-ui-framework-jcr-6.2.25.jar:?]
	at info.magnolia.admincentral.ResurfaceUI.lambda$accessSynchronously$2(ResurfaceUI.java:141) ~[magnolia-admincentral-6.2.23.jar:?]
	at com.vaadin.ui.UI.accessSynchronously(UI.java:1511) ~[vaadin-server-8.15.2.jar:8.15.2]
	at info.magnolia.admincentral.ResurfaceUI.accessSynchronously(ResurfaceUI.java:131) ~[magnolia-admincentral-6.2.23.jar:?]
	at com.vaadin.ui.UI$3.run(UI.java:1575) ~[vaadin-server-8.15.2.jar:8.15.2]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_77]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_77]

Notes
Refresh more intelligently: the first time react immediately, but over time under stress refresh less frequently (e.g. once in 5 seconds)



 Comments   
Comment by Roman Kovařík [ 19/Jan/23 ]

Discovery:

We might try to skip a lot of push calls in JcrDatasourceObservation if the previous haven't completed yet ~

if (!inProgress) {
    ui.access(() -> {
        synchronized {
          inProgress = true;
          actions.forEach(Runnable::run);
          ui.push();
          inProgress = false;
        }
     });
}
Comment by Thomas Comiotto [ 24/Jan/23 ]

Most of all: limit the scope of the observation / push calls!

We run a multisite installation with 500+ sites and up to 1500 concurrent authors. 99% of the refresh action happens because of JCR changes in areas where the current user either doesn't work (item not visible) or isn't concerned with (no edit rights in that particular site or subtree).

Comment by Roman Kovařík [ 26/Jan/23 ]

Hi tcomiotto,

Thanks for your suggestions.

This ticket aims for a short-time fix to avoid stackoverflows and reduce the amount of the operations.

A preliminary version is here https://git.magnolia-cms.com/projects/PLATFORM/repos/ui/pull-requests/2291/overview.

There are also long-term ideas like MGNLUI-6342 (we've added your suggestion and described possible side effects https://jira.magnolia-cms.com/browse/MGNLUI-6342?focusedId=339631&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-339631) or MGNLUI-7762.

 

Best regards

Roman

Comment by Thomas Comiotto [ 06/Feb/23 ]

Hi Roman,

I will look into MGNLUI-6342 tomorrow. I normally don't do Magnolia core development, so I lack the necessary IDE projects (and probably also some build infrastructure). Could you please attach a custom jar with the short-time fix.

Thanks
Thomas

Comment by Roman Kovařík [ 07/Feb/23 ]

Hi Thomas,

 

we've applied the fix to version 6.2.25 of the JCR framework module (according to the support ticket) magnolia-ui-framework-jcr-6.2.25-MGNLUI-7749.jar.

 

Hope that helps

Roman

Generated at Mon Feb 12 09:49:00 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.