[MLEARN-18] Adopt deeplearning4j latest beta version Created: 09/Oct/19  Updated: 20/Jan/20  Resolved: 16/Oct/19

Status: Closed
Project: Machine Learning
Component/s: None
Affects Version/s: None
Fix Version/s: 1.1.1

Type: Improvement Priority: Neutral
Reporter: Andres Garcia Assignee: Ilgun Ilgun
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to MGNLPER-121 Release NN memory when an user become... Closed
causality
relation
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Release notes required:
Yes
Date of First Response:
Sprint: Add-Ons 22
Story Points: 3

 Description   

Since beta4 they have improved the memory comsuption and performance, these changes might reduce the memory footprint and remove the errors we've noticed when operating magnolia instances with limited memory requirements (<8GB).

 

https://github.com/eclipse/deeplearning4j/releases



 Comments   
Comment by Espen Jervidalo [ 14/Oct/19 ]

ilgun, I haven't looked into details, but is the memory improvement you mentioned targeting the off heap / native memory management? Because the issue we see is not related to jvm heap consumption. It's the off heap memory that causes the jvm to exit.

As for simulation. I hope we can reproduce this on a non cloud environment. Otherwise things will get complex to coordinate. But if you spin up Magnolia on a memory restricted host it should be doable I hope.

Comment by Espen Jervidalo [ 14/Oct/19 ]

For more details, including heap information during the crash, see https://wiki.magnolia-cms.com/display/SRE/2019-09-26+Magnolia+process+killed+and+restarted and the attached tmp file.

There's also more details in the linked SRE ticket.

I doubt that updating the library will fix the problem as such. We'll have to get memory recommendations from you guys for different memory sizes. As somebody else also mentioned, that will also be relevant for on premise installations.

Generated at Mon Feb 12 02:29:06 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.