[MGNLDMS-159] Caching big documents from dms repository causes OutOfMemoryError Created: 17/Mar/09 Updated: 03/Jul/14 Resolved: 01/Dec/10 |
|
| Status: | Closed |
| Project: | Document Management System (closed) |
| Component/s: | None |
| Affects Version/s: | 1.3 |
| Fix Version/s: | 1.5 |
| Type: | Bug | Priority: | Major |
| Reporter: | Henryk Paluch | Assignee: | Jan Haderka |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | cache, outofmemoryerror | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Magnolia 3.6.3 CE |
||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Template: |
|
||||||||||||||||
| Acceptance criteria: |
Empty
|
||||||||||||||||
| Date of First Response: | |||||||||||||||||
| Description |
|
magnolia-module-cache caches on public instance all data into byte[] array which easily causes OutofMemoryError on large repositories. NOTE: this error occures on Public instances only, whose has caching enabled! How to reproduce: java.lang.OutOfMemoryError: Java heap space Woraround: disable caching for large repositories (attachment pending) Possible solution: cache module should stream large files into disk rather than memory (as it did in 3.0.x version). This should prevent OutOfMemory error on large Document and or Website repositories. |
| Comments |
| Comment by Henryk Paluch [ 17/Mar/09 ] |
|
oom_stacktrace.txt - stacktrace of OutOfMemoryError |
| Comment by Henryk Paluch [ 17/Mar/09 ] |
|
config.modules.cache.config.configurations.default.cachePolicy.voters.urls.excludes.dmsNoCache.xml We are using this workaround on our production system: It has about 4GB documents, typical size of one documen is 5MB, some of them are 100MB long. Heap size is 1.5GB |
| Comment by Jan Haderka [ 02/Apr/09 ] |
|
Done as of r24144. |
| Comment by Philipp Bärfuss [ 08/Jun/09 ] |
|
I have reverted the current fix. The reason is that this voter creates a jcr session per request to the dms. This can be very critical you, like we do in STK, store background images in the dms. This results in opening 20 sessions per page, even witch cache switched on. Here are some notes for a potential future solution: a) use a shared session for the read access. Since the voter only reads file sizes we won't run into security issues. b) cache a special entry for big files
c) us a special threshold stream which streams into a temp file after a certain amount of data has bean streamed |
| Comment by Philipp Bärfuss [ 08/Jun/09 ] |
|
If we re add the voter we definitely have to consider |
| Comment by Jan Haderka [ 09/Jun/09 ] |
Stream is not the only problem with this solution. Binary items have to be passed to ehCache in CachedPage as Byte[]. |
| Comment by Philipp Bärfuss [ 01/Dec/10 ] |
|
Fixed by |