[MAGNOLIA-6338] New template scripts on the classpath are not loaded Created: 12/Aug/15  Updated: 03/Oct/17  Resolved: 01/Feb/16

Status: Closed
Project: Magnolia
Component/s: resource-loader
Affects Version/s: 5.4
Fix Version/s: 5.4.4, 5.4.5

Type: Bug Priority: Critical
Reporter: Christian Ringele Assignee: Ilgun Ilgun
Resolution: Obsolete Votes: 0
Labels: None
Σ Remaining Estimate: 0d Remaining Estimate: 0d
Σ Time Spent: 7h Time Spent: 7h
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Issue Links:
Relates
relates to MAGNOLIA-6223 Provide proper API for resource chang... Closed
duplicate
is duplicated by MAGNOLIA-6389 Re-iterate over the ClasspathScanner Closed
relation
is related to MAGNOLIA-6469 Deleted folders are not detected in C... Closed
is related to MGNLRES-240 ResourceContainer should react to del... Closed
supersession
is superseded by MAGNOLIA-6523 Refactor ClasspathResourceOrigin and ... Closed
Sub-Tasks:
Key
Summary
Type
Status
Assignee
MAGNOLIA-6457 Make observed extensions configurable Sub-task Closed Ilgun Ilgun  
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Bug DoR:
[ ]* Steps to reproduce, expected, and actual results filled
[ ]* Affected version filled
Release notes required:
Yes
Date of First Response:
Epic Link: Resource reloading without restart
Sprint: Basel 24, Basel 31
Story Points: 1

 Description   

Resources on the classpath are not observed, except the YAML ones.
Hence new files, e.g. template-scripts are not found until restart.
As part of this issue, We will also handle newly added and deleted resources.
Also ClasspathResourceOrigin is serving resources from cache, and thus, one would never find any created resource after the system startup.
We will change this and serve resources if they are actually present.


Original description:

Situation:

In a checked out JAR/Maven module in the IDE (Eclipse in this case)
Having there the inner new module structure as:
/src/main/resources/%moduleName%/templates etc.

Behavior in 5.3.x:
  • One could create on the fly a ftl file in:
    /src/main/resources/foo/test.ftl
    And use it in a template def as: /foo/test.ftl
    Worked perfect
Behavior in 5.4.0:
  • create new ftl in: /src/main/resources/foo/test.ftl
    (old pattern) or /src/main/resources/%moduleName%/templates/foo/test.ftl (new pattern)
    And use it in a template def
    Does not work anymore -> needs server restart
    Also resources app doesn't show it until restart.
  • new yaml files created in: /%moduleName%/templates/...
    are properly found
  • Existing ftl files during startup are loaded correctly on changes.
  • In the directory where magnolia.resources.dir=${magnolia.home} points to new ftl's are found on the fly correctly.
Stack trace:
"2015-08-06 16:59:13,031 INFO  agnolia.config.source.yaml.YamlConfigurationSource: Registered DefinitionMetadataBuilder.DefinitionMetadataImpl(type=TEMPLATE, referenceId=travel-demo:pages/home2, name=home2, module=travel-demo, location=/travel-demo/templates/pages/home2.yaml, relativeLocation=pages/home2) from LayeredResource{path='/travel-demo/templates/pages/home2.yaml', layeredResources=[ClasspathResource{path=/travel-demo/templates/pages/home2.yaml}]}
2015-08-06 16:59:40,800 ERROR ndering.engine.ModeDependentRenderExceptionHandler: Error while rendering [/test2] with template [travel-demo:pages/home2] for URI [/test2.html?mgnlPreview=false&mgnlChannel=desktop]:
RenderException: java.io.FileNotFoundException: Template "/travel-demo/templates/pages/main2.ftl" not found. The quoted name was interpreted by this template loader: info.magnolia.freemarker.FreemarkerHelper$ConfigDelegatingTemplateLoader@d0bb7d8.
info.magnolia.rendering.engine.RenderException: java.io.FileNotFoundException: Template "/travel-demo/templates/pages/main2.ftl" not found. The quoted name was interpreted by this template loader: info.magnolia.freemarker.FreemarkerHelper$ConfigDelegatingTemplateLoader@d0bb7d8.
        at info.magnolia.rendering.renderer.FreemarkerRenderer.onRender(FreemarkerRenderer.java:100)
        at info.magnolia.rendering.renderer.AbstractRenderer.render(AbstractRenderer.java:155)
        at info.magnolia.rendering.engine.DefaultRenderingEngine.render(DefaultRenderingEngine.java:118):

With proposed solution;

  • Introduced magnolia.resources.classpath.observation.pattern property. With this property, we gave user a way to handle which files they would like us to observe for them. Given a user wants ftl files to be observed, they can just add it to the observation pattern.
  • Detection of added/deleted files and handling the properly
  • Deleted files are reflected in the resources-app properly {with help of MGNLRES-240}

    as well.

  • Serving files upon request if they are present.


 Comments   
Comment by Christian Ringele [ 13/Aug/15 ]

Work around:
Point in the property in the 'magnolia.properties' file:

# The directory to expose file system resources from
magnolia.resources.dir=${magnolia.home}

to the directory within you checked out char module to: /src/main/resources
Then its observed and picked up by the file based resource loading.
(Default is located in 'webapp/WEB-INF/config/default/magnolia.properties')

(Or a soft link in Linux to /src/main/resources, and then point to the soft link with the property)

Comment by Mikaël Geljić [ 13/Aug/15 ]

To shed some details: ClasspathResourceOrigin internally uses the Reflections library, does an initial scan upon startup.
The only resources we're actually interested to observe are the YAML files, because configuration then needs to be reloaded.
However for template scripts, they are served "upon request".

Now because we cache the tree structure of classpath resources, we never detect new files which are not YAML.
And triggering observation for all classpath resources is a performance no-go. Instead it would be nice to simply rely on IDEs' hot deployment as you would expect (at least for the non-YAML ones).

And indeed, using the file-system resource origin is a valid workaround. Except I'd rather make this explicit by keeping this directory around the webapp, rather than hard-pointing the magnolia.resources.dir property to one specific module (it would already be a dead end if you want to work on multiple modules at the same time).

Comment by Aleksandr Pchelintcev [ 02/Dec/15 ]

commented the PR

Comment by Federico Grilli [ 24/Dec/15 ]

QA was done as follows

  • set log level for info.magnolia.classpathwatch.ClasspathScanner to DEBUG
  • one project open in my IDE (eclipse) using FTL and YAML config files under src/main/resources

TEST classpath scanning on

  • magnolia.develop=true at magnolia.properties file so to trigger cp scanning
  • created a new ftl file on the fly within the IDE and used it in a config
    • changes are picked up
    • in the logs one can read DEBUG info.magnolia.classpathwatch.ClasspathScanner : Found resource change at mtk/templates/components/baz.ftl, triggering callback.
  • modified an existing ftl file (called html.ftl) on the fly within the IDE and reloaded page where used
    • changes are picked up
    • in the logs one can read DEBUG info.magnolia.classpathwatch.ClasspathScanner : Found resource change at mtk/templates/components/html.ftl, triggering callback.

TEST classpath scanning on and no observation of FTLs

  • magnolia.develop=true at magnolia.properties file so to trigger cp scanning
  • magnolia.resources.classpath.observation.pattern=.*
    .(yaml)$
    at magnolia.properties
    • creating a new ftl or modifying an existing one doesn't show any resource change found, according to logs.

TEST classpath scanning off

  • magnolia.develop=false (default value) at magnolia.properties
    • creating a new ftl or modifying an existing one doesn't show any resource change found, according to logs.
Comment by Michael Mühlebach [ 01/Feb/16 ]

This issue should be solved by MAGNOLIA-6523.

Generated at Mon Feb 12 04:13:34 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.