[MGNLIMG-209] Support webp images Created: 12/Sep/18  Updated: 13/Jun/23  Resolved: 24/Sep/21

Status: Closed
Project: Imaging
Component/s: None
Affects Version/s: 3.3.2, 3.4.1
Fix Version/s: 3.5.0

Type: Improvement Priority: Neutral
Reporter: Viet Nguyen Assignee: Jan Haderka
Resolution: Done Votes: 3
Labels: None
Remaining Estimate: 0.25d
Time Spent: 0.75d
Original Estimate: Not Specified

Attachments: XML File config.server.MIMEMapping.webp.xml     File libwebp-imageio.dylib     File webp-imageio-1.0.1-SNAPSHOT-debug.jar     File webp-imageio-1.0.1-SNAPSHOT.jar     File webp-imageio-1.0.1-SNAPSHOT.pom     XML File webp_generator_config.xml     Text File webp_generator_response.txt    
Issue Links:
Cloners
is cloned by BUILD-534 Support webp images Closed
is cloned by MGNLDAM-989 Support webp images Closed
Relates
relates to MGNLIMG-242 Support webp images also on aarch64 Closed
causality
relation
is related to RESPDAM-34 Add WEBP support Closed
Template:
Acceptance criteria:
Empty
Task DoD:
[X]* Doc/release notes changes? Comment present?
[X]* Downstream builds green?
[X]* Solution information and context easily available?
[X]* Tests
[X]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Release notes required:
Yes
Date of First Response:

 Description   

WebP is an image format employing both lossy[6] and lossless compression. It is currently developed by Google, based on technology acquired with the purchase of On2 Technologies.

– wikipedia

WebP lossless images are 26% smaller in size compared to PNGs. WebP lossy images are 25-34% smaller than comparable JPEG images at equivalent SSIM quality index.

Lossless WebP supports transparency (also known as alpha channel) at a cost of just 22% additional bytes. For cases when lossy RGB compression is acceptable, lossy WebP also supports transparency, typically providing 3× smaller file sizes compared to PNG.

https://developers.google.com/speed/webp/

Currently we had this issue when uploading a webp image:

errorlog
SEVERE: Servlet.service() for servlet [default] in context with path [/mgnlwebapp55] threw exception
 com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: info.magnolia.imaging.ImagingException: Could not load image for info.magnolia.imaging.parameters.BinaryNodeParameterProvider@6758e62e
 at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218)
 at com.google.common.cache.LocalCache.get(LocalCache.java:4147)
 at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)
 at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)
 at info.magnolia.imaging.caching.CachingImageStreamer.serveImage(CachingImageStreamer.java:148)
 at info.magnolia.imaging.Imaging.generate(Imaging.java:76)
 at info.magnolia.imaging.ImagingServlet.doGet(ImagingServlet.java:87)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
 at info.magnolia.cms.filters.ServletDispatchingFilter.doFilter(ServletDispatchingFilter.java:148)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.CompositeFilter.doFilter(CompositeFilter.java:65)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.virtualuri.VirtualUriFilter.doFilter(VirtualUriFilter.java:98)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.module.cache.executor.Bypass.processCacheRequest(Bypass.java:58)
 at info.magnolia.module.cache.executor.CompositeExecutor.processCacheRequest(CompositeExecutor.java:67)
 at info.magnolia.module.cache.filter.CacheFilter.doFilter(CacheFilter.java:220)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.i18n.I18nContentSupportFilter.doFilter(I18nContentSupportFilter.java:89)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.RangeSupportFilter.doFilter(RangeSupportFilter.java:77)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.security.BaseSecurityFilter.doFilter(BaseSecurityFilter.java:57)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.multisite.filters.CrossSiteSecurityFilter.doFilter(CrossSiteSecurityFilter.java:104)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.security.SecurityCallbackFilter.doFilter(SecurityCallbackFilter.java:84)
 at info.magnolia.cms.filters.OncePerRequestAbstractMgnlFilter.doFilter(OncePerRequestAbstractMgnlFilter.java:59)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.security.LogoutFilter.doFilter(LogoutFilter.java:94)
 at info.magnolia.cms.filters.OncePerRequestAbstractMgnlFilter.doFilter(OncePerRequestAbstractMgnlFilter.java:59)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.module.site.filters.SiteMergeFilter.doFilter(SiteMergeFilter.java:119)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.multisite.filters.MultiSiteFilter.doFilter(MultiSiteFilter.java:120)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.MultiChannelFilter.doFilter(MultiChannelFilter.java:83)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.security.auth.login.LoginFilter.doFilter(LoginFilter.java:127)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.enterprise.registration.RegistrationFilter.doFilter(RegistrationFilter.java:64)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.CosMultipartRequestFilter.doFilter(CosMultipartRequestFilter.java:111)
 at info.magnolia.cms.filters.OncePerRequestAbstractMgnlFilter.doFilter(OncePerRequestAbstractMgnlFilter.java:59)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.personalization.preview.filter.PreviewFilter.doFilter(PreviewFilter.java:92)
 at info.magnolia.cms.filters.OncePerRequestAbstractMgnlFilter.doFilter(OncePerRequestAbstractMgnlFilter.java:59)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:81)
 at info.magnolia.cms.filters.ContentTypeFilter.doFilter(ContentTypeFilter.java:155)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.ContextFilter.doFilter(ContextFilter.java:128)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.MgnlFilterChain.doFilter(MgnlFilterChain.java:79)
 at info.magnolia.cms.filters.CompositeFilter.doFilter(CompositeFilter.java:65)
 at info.magnolia.cms.filters.AbstractMgnlFilter.doFilter(AbstractMgnlFilter.java:85)
 at info.magnolia.cms.filters.SafeDestroyMgnlFilterWrapper.doFilter(SafeDestroyMgnlFilterWrapper.java:107)
 at info.magnolia.cms.filters.MgnlFilterDispatcher.doDispatch(MgnlFilterDispatcher.java:67)
 at info.magnolia.cms.filters.MgnlMainFilter.doFilter(MgnlMainFilter.java:110)
 at info.magnolia.cms.filters.MgnlMainFilter.doFilter(MgnlMainFilter.java:96)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
 at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
 at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:651)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
 at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:412)
 at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
 at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:754)
 at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1385)
 at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
 at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
 at java.base/java.lang.Thread.run(Thread.java:844)
 Caused by: java.lang.RuntimeException: info.magnolia.imaging.ImagingException: Could not load image for info.magnolia.imaging.parameters.BinaryNodeParameterProvider@6758e62e
 at info.magnolia.imaging.caching.CachingImageStreamer$1.load(CachingImageStreamer.java:135)
 at info.magnolia.imaging.caching.CachingImageStreamer$1.load(CachingImageStreamer.java:127)
 at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708)
 at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416)
 at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299)
 at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212)
 ... 104 more
 Caused by: info.magnolia.imaging.ImagingException: Could not load image for info.magnolia.imaging.parameters.BinaryNodeParameterProvider@6758e62e
 at info.magnolia.imaging.operations.load.AbstractLoader.apply(AbstractLoader.java:74)
 at info.magnolia.imaging.operations.ImageOperationChain.apply(ImageOperationChain.java:81)
 at info.magnolia.imaging.operations.ImageOperationChain.generate(ImageOperationChain.java:74)
 at info.magnolia.imaging.DefaultImageStreamer.serveImage(DefaultImageStreamer.java:61)
 at info.magnolia.imaging.caching.CachingImageStreamer.generateAndStore(CachingImageStreamer.java:226)
 at info.magnolia.imaging.caching.CachingImageStreamer$1.load(CachingImageStreamer.java:132)
 ... 109 more



 Comments   
Comment by Viet Nguyen [ 02/Oct/18 ]

Follow below guidelines then it should work
https://github.com/lonnyj/webp-imageio
A note on Mac OS High Sierra 10.13.2 and current built library as of Oct, 2018
You can copy this file to your java library (-Djava.library.path=<path_to_your_lib_folder>) without make it yourself
libwebp-imageio.dylib
You can use this (compiled by Oracle Java 9) without checking out the source and build it yourself
webp-imageio-1.0.1-SNAPSHOT.jar
Offline Maven artifact installation

mvn install:install-file -Dfile=webp-imageio-1.0.1-SNAPSHOT.jar -DgroupId=com.github.lonnyj -DartifactId=webp-imageio -Dversion=1.0.1-SNAPSHOT -Dpackaging=jar

How to use it in your project

<dependency>
  <artifactId>webp-imageio</artifactId>
  <groupId>com.github.lonnyj</groupId>
  <version>1.0.1-SNAPSHOT</version>
</dependency>

No additional step need to do in Magnolia CMS.
Test example:

public static void main(String[] args) throws IOException {
    // decode
    ImageIO.read(new File("/path-to-your-folder/webp-input-example.webp"));
    
    // signal read successful
    System.out.println("read successful");
    
    // read another image
    BufferedImage image = ImageIO.read(new File("/path-to-your-folder/png-example.png"));
    
    // encode
    ImageIO.write(image, "webp", new File("/path-to-your-folder/webp-output-example.webp"));
    
    // signal write successful
    System.out.println("write successful");
}

Again note that you need to add below JAVA option if you didn't have it configured automatically yet, in Eclipse set your VM arguments when running or debugging instead of Application arguments.
-Djava.library.path=<path_to_your_lib_folder>

Additional info:

  1. Pom configuration of the webp-imageio project webp-imageio-1.0.1-SNAPSHOT.pom
  2. Debug file - just use it when you know what it is webp-imageio-1.0.1-SNAPSHOT-debug.jar
Comment by Viet Nguyen [ 02/Oct/18 ]

Note that this would not work atm https://bitbucket.org/luciad/webp-imageio

Comment by Richard Gange [ 31/Oct/18 ]

I also had to set the MIME type under /server/MIMEMapping

Here is an export of the settings: config.server.MIMEMapping.webp.xml

Comment by Siarhei Ivanou [ 19/Mar/19 ]

Could you please provide webp-jni.dll file for this?

Comment by Viet Nguyen [ 20/Mar/19 ]

Hello s.ivanou,

May I have your Windows version please, currently I am having windows 7 virtual machine and trying to build it locally but I think it would be much better if you can run some simple command from your machine to build a best suite dll for you. Commands are described on their official website like this:

Using CMake

  • Install CMake 2.8 or newer. CMake can be downloaded from www.cmake.org or installed using your systems package manager.
  • Download libwebp 0.4.2 and extract it into the project's directory
  • Run 'cmake .' in the root of directory of the project to generate the build scripts for your system.
  • Build the project using the generated build scripts. If the Java build fails and you previously built with Maven, run 'mvn clean' and try again.
  • The build scripts will generate a number of binaries
    • java/webp-imageio.jar: JAR file containing the Image I/O reader and writer
    • c/libwebp-imageio.so: the JNI library that is required by webp-imageio.jar

Also from their README:

Windows build:
--------------

By running:

nmake /f Makefile.vc CFG=release-static RTLIBCFG=static OBJDIR=output

the directory output\release-static(x64|x86)\bin will contain the tools
cwebp.exe and dwebp.exe. The directory output\release-static(x64|x86)\lib will
contain the libwebp static library.
The target architecture (x86/x64) is detected by Makefile.vc from the Visual
Studio compiler (cl.exe) available in the system path.

Comment by Siarhei Ivanou [ 20/Mar/19 ]

Hello viet.nguyen,

Thanks for the information provided. My Windows version is Win 10 Pro. I have tried CMake and nmake approaches, but both doesn't work for me out of box. 

So I got managed to generate build scripts with CMake, but when I try to build the project with maven it fails:

In file included from /cygdrive/c/Java/jdk1.8.0_161/include/jni.h:45:0,
                 from E:/libwebp-0.4.2/webp-imageio-master/c/com_luciad_imageio_webp_WebP.h:2,
                 from E:/libwebp-0.4.2/webp-imageio-master/c/webp-imageio.c:24:
/cygdrive/c/Java/jdk1.8.0_161/include/win32/jni_md.h:34:9: error: unknown type name '__int64'
 typedef __int64 jlong;
         ^~~~~~~
make[2]: *** [CMakeFiles/webp-imageio.dir/build.make:1554: CMakeFiles/webp-imageio.dir/webp-imageio.c.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:64: CMakeFiles/webp-imageio.dir/all] Error 2
make: *** [Makefile:76: all] Error 2

Building via nmake results with next errors:

        cl.exe /nologo /O2 /DNDEBUG /MT /Isrc /nologo /W3 /EHsc /c /GS /DWIN32 /D_CRT_SECURE_NO_WARNINGS /DWIN32_LEAN_AND_MEAN /DHAVE_WINCODEC_H /DWEBP_USE_THREAD /Fdoutput\release-static\x64\obj\libwebp.pdb /Fooutput\release-static\x64\obj\dec\ src\dec\alpha.c src\dec\buffer.c src\dec\frame.c src\dec\idec.c src\dec\io.c src\dec\quant.c src\dec\tree.c src\dec\vp8.c src\dec\vp8l.c src\dec\webp.c
alpha.c
src\dec\alpha.c(14): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory
buffer.c
src\dec\buffer.c(14): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory
frame.c
src\dec\frame.c(14): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory
idec.c
src\dec\idec.c(14): fatal error C1083: Cannot open include file: 'assert.h': No such file or directory
io.c
src\dec\io.c(14): fatal error C1083: Cannot open include file: 'assert.h': No such file or directory
quant.c
e:\libwebp-0.4.2\libwebp-0.4.2\src\dec\./vp8i.h(17): fatal error C1083: Cannot open include file: 'string.h': No such file or directory
tree.c
e:\libwebp-0.4.2\libwebp-0.4.2\src\dec\./vp8i.h(17): fatal error C1083: Cannot open include file: 'string.h': No such file or directory
vp8.c
src\dec\vp8.c(14): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory
vp8l.c
src\dec\vp8l.c(15): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory
webp.c
src\dec\webp.c(14): fatal error C1083: Cannot open include file: 'stdlib.h': No such file or directory
Comment by Viet Nguyen [ 21/Mar/19 ]

Going through some Google searches, I am seeing that using VS2017 "cl" program may cause your compile errors. Could you please try to use older version VS 2013/ VS 2015 and trying to set Windows SDK Version to 10 for your compiler. I am sorry that our help in building up this kind of library on Windows is quite limited because we had no Windows C expert in house, we just follow guidelines from the library author.

Comment by Chris Jennings [ 22/Mar/21 ]

Further notes on WebP support.
On Magnolia 6.2.6 I have been able to generate WebP output from the Imaging Module. This system has the Responsive DAM incubator extension (v2.0.3) installed which brings the webp-imageio dependency via the org.sejda fork.

I used a default generator supplied by the imaging module and simply changed the `outputFormat` parameter. This created a WebP version of the supplied JPEG image.

See attachments: webp_generator_config.xml, webp_generator_response.txt

Generated at Mon Feb 12 02:13:21 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.