[MLEARN-18] Adopt deeplearning4j latest beta version Created: 09/Oct/19 Updated: 20/Jan/20 Resolved: 16/Oct/19 |
|
| Status: | Closed |
| Project: | Machine Learning |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 1.1.1 |
| Type: | Improvement | Priority: | Neutral |
| Reporter: | Andres Garcia | Assignee: | Ilgun Ilgun |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Template: |
|
||||||||||||||||
| Acceptance criteria: |
Empty
|
||||||||||||||||
| Task DoD: |
[ ]*
Doc/release notes changes? Comment present?
[ ]*
Downstream builds green?
[ ]*
Solution information and context easily available?
[ ]*
Tests
[ ]*
FixVersion filled and not yet released
[ ] 
Architecture Decision Record (ADR)
|
||||||||||||||||
| Release notes required: |
Yes
|
||||||||||||||||
| Date of First Response: | |||||||||||||||||
| Sprint: | Add-Ons 22 | ||||||||||||||||
| Story Points: | 3 | ||||||||||||||||
| Description |
|
Since beta4 they have improved the memory comsuption and performance, these changes might reduce the memory footprint and remove the errors we've noticed when operating magnolia instances with limited memory requirements (<8GB).
https://github.com/eclipse/deeplearning4j/releases |
| Comments |
| Comment by Espen Jervidalo [ 14/Oct/19 ] |
|
ilgun, I haven't looked into details, but is the memory improvement you mentioned targeting the off heap / native memory management? Because the issue we see is not related to jvm heap consumption. It's the off heap memory that causes the jvm to exit. As for simulation. I hope we can reproduce this on a non cloud environment. Otherwise things will get complex to coordinate. But if you spin up Magnolia on a memory restricted host it should be doable I hope. |
| Comment by Espen Jervidalo [ 14/Oct/19 ] |
|
For more details, including heap information during the crash, see https://wiki.magnolia-cms.com/display/SRE/2019-09-26+Magnolia+process+killed+and+restarted and the attached tmp file. There's also more details in the linked SRE ticket. I doubt that updating the library will fix the problem as such. We'll have to get memory recommendations from you guys for different memory sizes. As somebody else also mentioned, that will also be relevant for on premise installations. |