[MGNLPER-121] Release NN memory when an user becomes inactive rather than when the 11th user logs in Created: 15/Nov/19  Updated: 17/Mar/20  Resolved: 22/Jan/20

Status: Closed
Project: Periscope
Component/s: None
Affects Version/s: None
Fix Version/s: 1.2

Type: Task Priority: Neutral
Reporter: Maxime Michel Assignee: Maxime Michel
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to MLEARN-18 Adopt deeplearning4j latest beta version Closed
Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Release notes required:
Yes
Documentation update required:
Yes
Date of First Response:
Epic Link: Periscope improvements

 Description   

Research in DEV-1381 showed that:

  • DL4J's memory consumption can be kept under control with JVM arguments
  • the limit of 10 users allowed to have a memory NN is not flexible
    • too big for small instances, too small for big instances
    • also, 10 in-memory networks lead to a lot of processing on top of them

Although releasing users immediately from Magnolia's internal cache doesn't free memory immediately, it is still an improvement over the current solution. We should implement that while at the same time find out what to do with JVM flags. (see linked GitHub issue)



 Comments   
Comment by Maxime Michel [ 17/Jan/20 ]

QA note: the approach implemented in this ticket indeed allows more off-heap memory to be released.

Comment by Aleksandr Pchelintcev [ 22/Jan/20 ]

The solution provided is prone to fail when the destruction is triggered outside of normal Mgnl scope (i.e. as result of Vaadin cleaning-up the dead sessions as a result of consequent failed heartbeat checks). In order to avoid the problem, the related user has to be pre-resolved [during construction] instead of being retreived via static accessor during desctruction itself.

Comment by Maxime Michel [ 17/Mar/20 ]

This ticket introduces logic at UI level to let the machine learning mechanism (DX core only) release users from memory as soon as they log out or their session expires. Before this change, a user would stay in memory until the limit of 10 users was reached.

This helps with memory consumption but DL4J memory management should still be configured to avoid off-heap memory starvation, in particular the following JVM flag: -Dorg.bytedeco.javacpp.maxbytes.

We have been contributing to javacpp so that the process' total memory consumption can be defined with a relative rather than absolute figure. Once this feature is available, we will provide an official recommendation.

For now, what can be said is that Magnolia's use case is not a typical DLF4J setup, and that the total memory for the process should be set to maybe 10-25% of Xmx, rather than 100% of Xmx, which is what happens by default.

Generated at Mon Feb 12 10:28:56 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.