[MGNLMIGRATION-234] Optimize performance for large projects Created: 21/Mar/13  Updated: 29/Mar/13  Resolved: 29/Mar/13

Status: Closed
Project: Migration 4.4 to 4.5 (closed)
Component/s: None
Affects Version/s: None
Fix Version/s: 1.2.2

Type: Improvement Priority: Critical
Reporter: Zdenek Skodik Assignee: Roman Kovařík
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
dependency
depends upon MGNLMIGRATION-237 Make report persistance optional Closed
relation
is related to MGNLMIGRATION-190 Support persisting unfinished migration Closed
Template:
Acceptance criteria:
Empty
Date of First Response:

 Description   

The module works with session based saving. The more nodes to migrate, the slower it is. This is an exponential function. It makes migrating large sets of nodes a neverending job. We should think about workspace based saving/export/import/...

F.e. having about 10 thousands pages and focusing on "update node type" task, the migration is able to perform about 20 thousands of node types changes in first hour, 15 thousands in second hour, 10 thousands in third hour, ..., 3 thousands in 18th hour, etc. Such job would need about 3 days, which can't be acceptable. The linked migration.log is a good reference (covers less then a day), it was running on 8GB RAM and 2 CPUs.



 Comments   
Comment by Roman Kovařík [ 29/Mar/13 ]

UpdateWebsitePrimaryTypeMigrationTaskTest is already part of MGNLMIGRATION-231.

Comment by Jan Haderka [ 29/Mar/13 ]

I would prefer there was an explicit comment around all those session.save() calls saying this is to prevent OOME for gib migrations.

Also Node type of " + countOfNodes + " nodes already changed. message should pbly read "Changed node type of xxx nodes already."

Generated at Mon Feb 12 10:18:26 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.