[TASKMGMT-41] Pulse performance slows down as number of Task increases Created: 24/Jun/20  Updated: 21/Dec/20  Resolved: 10/Jul/20

Status: Closed
Project: Task Management
Component/s: None
Affects Version/s: 1.2.7
Fix Version/s: 1.2.8

Type: Bug Priority: Neutral
Reporter: Richard Gange Assignee: Richard Gange
Resolution: Fixed Votes: 0
Labels: maintenance, performance, quickwin
Remaining Estimate: Not Specified
Time Spent: 0.5d
Original Estimate: Not Specified

Attachments: Zip Archive info.zip    
Issue Links:
causality
relation
is related to MGNLUI-4335 Pulse rendering on login generate unn... Closed
is related to MAGNOLIA-7847 Targeted indexing configurations Open
is related to MGNLUI-5999 Improve responsivity of the pulse whe... Open
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Bug DoR:
[ ]* Steps to reproduce, expected, and actual results filled
[ ]* Affected version filled
Release notes required:
Yes
Date of First Response:
Sprint: Maintenance 15
Story Points: 2

 Description   

We need to look at ways to improve the performance of the Pulse UI as number of tasks increases. Note: This is a separate issue from the resolved task issue.

Problems:
Over time the pulse slows down as more tasks are added. For example, if you schedule a bunch of tasks (100s) then it becomes more difficult to complete the entire transaction. After clicking "Approve and publish" the transition to the scheduled tab can take a long time as the UI is trying to load everything.

2020-06-24 17:20:55,120 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:20:55,121 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:20:55,123 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:20:55,124 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:20:55,125 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:20:55,126 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:20:55,127 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:20:55,128 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:20:55,130 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:20:55,131 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:20:55,409 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%' order by [mgnl:lastModified] desc] with limit=60 and offset=0
2020-06-24 17:20:55,427 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 18ms

Another problem scenario can be when a lot of new tasks start to build up. Once there is a significant amount of new tasks then after closing the scheduler dialog it can take a while for the system to finish it. Again we have many queries running. I know the time stamps look short here but you have to think about this in a production scenario. All that has happened is I scheduled a task and all of this is running.

2020-06-24 17:48:26,650 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE ((actorId IS NULL AND (groupIds = 'publishers') OR (actorIds = 'superuser')) AND status LIKE '%Created%') OR ((actorId = 'superuser' OR actorIds = 'superuser' ) AND status LIKE '%Failed%') ] with limit=-1 and offset=-1
2020-06-24 17:48:26,656 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 5ms
2020-06-24 17:48:26,658 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,658 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 0ms
2020-06-24 17:48:26,659 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,660 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:48:26,661 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,661 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 0ms
2020-06-24 17:48:26,661 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser') AND status LIKE '%InProgress%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,662 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:48:26,662 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser') AND status LIKE '%Failed%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,662 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 0ms
2020-06-24 17:48:26,662 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser') AND status LIKE '%Scheduled%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,666 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 4ms
2020-06-24 17:48:26,667 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,667 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 0ms
2020-06-24 17:48:26,668 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,668 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 0ms
2020-06-24 17:48:26,669 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,669 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 0ms
2020-06-24 17:48:26,670 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,670 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 0ms
2020-06-24 17:48:26,671 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:48:26,672 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:48:26,718 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Created%') AND status NOT LIKE '%Archived%' order by [mgnl:lastModified] desc] with limit=2 and offset=0
2020-06-24 17:48:26,720 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 2ms

Next let's say I simply open the Pulse and click the schedule tab where I have over 200 scheduled pages. I get the following:

2020-06-24 17:51:58,954 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:51:58,964 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 9ms
2020-06-24 17:51:58,965 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:51:58,966 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:51:58,967 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:51:58,968 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:51:58,969 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:51:58,971 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:51:58,972 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%'] with limit=-1 and offset=-1
2020-06-24 17:51:58,973 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 1ms
2020-06-24 17:51:59,007 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : JCR SQL2 statement is [SELECT * FROM [mgnl:task] WHERE (actorId = 'superuser' OR actorIds = 'superuser' OR groupIds = 'publishers') AND (status LIKE '%Scheduled%') AND status NOT LIKE '%Archived%' order by [mgnl:lastModified] desc] with limit=60 and offset=0
2020-06-24 17:51:59,019 DEBUG info.magnolia.task.persistence.TasksStoreImpl     : Query execution took 11ms

Expected:
I expect the performance to be better and not regress as tasks are added. Under a pressure situation this could become a real issue for users as getting the job done is slowed down by a lot of extra processing. We need to look for ways to streamline things a bit.

Workaround:
Increasing bundle cache in the task workspace.xml could be beneficial but it's not a silver bullet.
The campaign publisher module could be a better way to handle large groups content going out at the same time.

To consider:

  • Can we further subcategorize tasks to make the queries more efficient?
  • Can we cache queries?
  • Can we reduce queries?
  • Can we adjust the search config file for better performance?


 Comments   
Comment by Richard Gange [ 25/Jun/20 ]

In the release notes we need to make sure customers understand they need to reindex the tasks workspace in order to benefit from this change.

Comment by Richard Gange [ 01/Jul/20 ]

Patch: info.zip

Generated at Mon Feb 12 11:02:31 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.