[MLEARN-3] Store ranking neural network labels in non-flat way in JCR Created: 10/Dec/18  Updated: 06/Feb/19  Resolved: 06/Feb/19

Status: Closed
Project: Machine Learning
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Neutral
Reporter: Cedric Reichenbach Assignee: Unassigned
Resolution: Outdated Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to MGNLPER-87 Investigate: Find Bar results and use... Closed
Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Date of First Response:
Epic Link: Periscope improvements

 Description   

Storing 1000+ nodes or properties on the same level leads to performance issues with JCR/Jackrabbit. Since we potentially store more labels than that (up to 10k by default), we should do so in a hierarchical structure with always less than 1k neighbours on any level.

Current location of labels (on demo): https://demo.magnolia-cms.com/.magnolia/admincentral#app:configuration:browser;/modules/periscope-result-ranker/persistence/labels:treeview:



 Comments   
Comment by Mikaël Geljić [ 08/Jan/19 ]

quoting relevant input from the Slack thread over #pd-foundation:

The JR performance issue is only with large amounts of child-nodes, not properties; mostly because there's no ordering to track for props, so that could still scale up decently
—would be easy to validate w/ a JR unit-test too; e.g. https://git.magnolia-cms.com/users/mgeljic/repos/jcr-tests/browse/src/test/java/me/mkgl/test/jackrabbit

Labels are specific to the output layer of our neural network, the mapping dynamically changes depending on user choices (new labels get added, and if a threshold is reached, old ones forgotten). So order is important and specific to the NN's learning state, meaning we cannot share the list between users.

Labels are nothing (KB) compared to the network itself (MB);

Labels' property numbers are there to indicate an index. All we need is basically an ordered list of labels [...]. A multi-value property would probably do the trick as well.

Generated at Mon Feb 12 02:28:57 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.