[MAGNOLIA-6500] Speed up DirectoryWatcherTest using Oracle JVM internal API Created: 15/Jan/16  Updated: 04/Apr/16  Resolved: 15/Jan/16

Status: Closed
Project: Magnolia
Component/s: None
Affects Version/s: 5.4.4
Fix Version/s: 5.4.5

Type: Improvement Priority: Neutral
Reporter: Tobias Mattsson Assignee: Tobias Mattsson
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: 0d
Time Spent: 0.25d
Original Estimate: Not Specified

Issue Links:
Relates
relates to MAGNOLIA-6623 Speed up DirectoryWatcher on OSX in d... Closed
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Date of First Response:
Sprint: Basel 27
Story Points: 1

 Description   

DirectoryWatcher test is in main/magnolia-resource-loader which currently takes 2 minutes and 20 seconds to build on my machine. This time is spent waiting for file system polling which only happens every 10 seconds. Each test has to wait for these 10 seconds to verify its result.

In the Oracle JVM the 10 second interval is the default and can be changed down to 2 seconds. This reduces the build time to 53 seconds. A reduction by 87 seconds, 62%.

Doing it requires a change to DirectoryWatcher itself, to get the modifiers in there for the register call. I've tried changing just the test by using reflection and proxies to squeeze the extra modifier in there but it's just too much work.



 Comments   
Comment by Christopher Zimmermann [ 18/Jan/16 ]

For development purposes it would be great if there was some way that a dev could choose to impreove the file watching speed.

Comment by Mikaël Geljić [ 18/Jan/16 ]

As far as I remember, it only takes 10s on OS X because there's no native WatchService implementation there, so it falls back to a polling impl; while on linux and windows it listens to native file system events, and is pretty immediate.

Comment by Tobias Mattsson [ 18/Jan/16 ]

That's my understanding as well, it falls back to sun.nio.fs.PollingWatchService on OS X.

Since this is only for tests it will only benefit developers running the tests on their Macs. That's us!

As Christopher points out it could also be helpful for developers using configuration by file and running OS X.

Comment by Federico Grilli [ 21/Jan/16 ]

Pre-integration QA results:
Mac OS X 10.11.2 (El Capitan)

java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

Before
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 110.118 sec - in info.magnolia.dirwatch.DirectoryWatcherTest
After
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.103 sec - in info.magnolia.dirwatch.DirectoryWatcherTest

Some concerns have been expressed regarding the effect of this change on OS where a native watcher implementation exists (e.g. Windows). I ran the above test on Windows7 and JDK7 and that seems to cause no issue at all

Before
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.874 sec - in info.magnolia.dirwatch.DirectoryWatcherTest
After
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312 sec - in info.magnolia.dirwatch.DirectoryWatcherTest

Comment by Tobias Mattsson [ 22/Jan/16 ]

The the native watch services are provided by OpenJDK and all ignore SensitivityWatchEventModifier.

Source:
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/c1e0c10972f4/src/solaris/classes/sun/nio/fs/SolarisWatchService.java
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/c1e0c10972f4/src/solaris/classes/sun/nio/fs/LinuxWatchService.java
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/c1e0c10972f4/src/windows/classes/sun/nio/fs/WindowsWatchService.java

The fallback PollingWatchService is used on OS X, Solaris before version 11, and on AIX. See implementations of java.io.FileSystem.newWatchService.
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/c1e0c10972f4/src/aix/classes/sun/nio/fs/AixFileSystem.java
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/c1e0c10972f4/src/windows/classes/sun/nio/fs/WindowsFileSystem.java
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/c1e0c10972f4/src/solaris/classes/sun/nio/fs/MacOSXFileSystem.java
http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/c1e0c10972f4/src/solaris/classes/sun/nio/fs/LinuxFileSystem.java

These links are to a commit on the development branch of OpenJDK8 dated Thu, 21 Jan 2016 19:21:34 +0000.

Generated at Mon Feb 12 04:15:07 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.