[MGNLCI-22] Bootstrapping/importing content with CT defined workspaces Created: 18/Nov/19  Updated: 04/Feb/21  Resolved: 25/Jun/20

Status: Closed
Project: Content Importer
Component/s: None
Affects Version/s: 1.1
Fix Version/s: 1.0.3

Type: Improvement Priority: Major
Reporter: Christopher Zimmermann Assignee: Robert Šiška
Resolution: Fixed Votes: 2
Labels: None
Remaining Estimate: 0d
Time Spent: 1d
Original Estimate: Not Specified

Issue Links:
Relates
relates to MGNLCI-24 Default configuration should be to au... Open
causality
is causing MGNLCI-25 Automatic bootstrapping and configura... Closed
is causing MGNLCI-23 DOC: Bootstrapping content from light... Closed
relation
is related to NPMCLI-238 'mgnl jumpstart' command should also ... Open
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Release notes required:
Yes
Documentation update required:
Yes
Date of First Response:
Epic Link: LD improvements
Sprint: HL & LD 4, HL & LD 5
Story Points: 5

 Description   

Problem:

If you use the Content Types feature to define workspaces and node types, and you have JCR content bootstrap files that rely on those Content Type definitions (typically in a modules 'mgnl-bootstrap' directory, or in the WEB-INF/bootraps/common directory), then the content will not be correctly installed at install time. When the system tries to bootstrap the content, it has not yet processed the Content Types and so the workspace does not exist yet.

Implemented in this ticket:

To bootstrap content, when using Content Types, use the Content Importer feature. Do not use the " WEB-INF/bootraps/common" feature.

The following capabilities have been added to the Content Importer feature.

  • In addition to magnolia.content.bootstrap.dir property, this ticket introduces magnolia.content.bootstrap.pattern which specifies wildcard pattern (glob). When not used, the behavior of content-importer is identical (the pattern defaults to *. {yaml,xml})
  • Property magnolia.content.bootstrap.initial=true (false by default) makes all files matching the pattern bootstrap at the startup. If the content with same path already exists, the import task is created instead.

Note - if you use the bootstrap.initial property, then you will recieve additional import tasks at every system restart. You can workaround  this by setting the property to false after the initial import. Or by removing the bootstrap files from the configured directory once they are imported.
This behaviour is addressed in https://jira.magnolia-cms.com/browse/MGNLCI-25

 

 

 

Copied from a documentation comment from uscheidegger 

If you have bootstrap data using that namespace and workspace, (defined in a CT yaml definiitons)  things will fail miserably.

After manually adding a workspace definition to the module descriptor, Magnolia only complains about the unknown namespace. 

So it seems like Magnolia should read the contentTypes definitions first, before bootstrapping anything. Of course this will hardly be possible, if the contentTypes are defined in JCR. 

IMO, the Magnolia startup code needs to be adapted:

  • When the repo config is read from the module descriptor files, Magnolia should also check for content type definitions files.
  • OR at least name space definitions in module descriptors should be supported.
    • However this would be a bit of a compromise as it would force us to keep the workspace and namespace definition in the module descriptor and  the content type definition.


 Comments   
Comment by Robert Šiška [ 15/Jun/20 ]

This issue implements:

  • In addition to magnolia.content.bootstrap.dir property, this PR introduces magnolia.content.bootstrap.pattern which specifies wildcard pattern (glob). When not used, the behavior of content-importer is identical (the pattern defaults to *. {yaml,xml})

.

  • Property magnolia.content.bootstrap.initial=true (false by default) makes all files matching the pattern bootstrap at the startup. If the content with same path already exists, the import task is created instead.
Comment by Christopher Zimmermann [ 15/Jun/20 ]

And is something changed such that content imports succeed now when the workspace is created by a content-type?

Comment by Robert Šiška [ 15/Jun/20 ]

Yes, content-importer has an optional dependency on content-type module, so when it starts, all repositories are already registered.

Comment by Christopher Zimmermann [ 16/Jun/20 ]

So all changes are only relevant to the  {{magnolia.content.bootstrap.dir }}directory (content importer), but not to the 'magnolia.bootstrap.dir' (usually 'WEB-INF/bootstrap/common')?

Comment by Christopher Zimmermann [ 16/Jun/20 ]

So when magnolia.content.boostrap.initial=true is used - then on install, no Tasks are created, no human intervention required? (unless path already exists?)

Comment by Christopher Zimmermann [ 22/Jun/20 ]

rsiska - When 'magnolia.content.bootstrap.initial=true' is used - does this happen at every startup? Or only at install as the 'WEB-INF/bootraps/common' feautre does?
(If at every startup, I think this will cause problems with all of the files that anyone adds to the importer running at every startup and generating tasks everytime.)

Comment by Jaroslav Simak [ 25/Jun/20 ]

Closing this ticket because module has been released even without proper final QA.

Comment by Robert Šiška [ 29/Jun/20 ]

So all changes are only relevant to the {{magnolia.content.bootstrap.dir }}directory (content importer), but not to the 'magnolia.bootstrap.dir' (usually 'WEB-INF/bootstrap/common')?

Yes.

So when magnolia.content.bootstrap.initial=true is used - then on install, no Tasks are created, no human intervention required? (unless path already exists?)

Also yes.

When 'magnolia.content.bootstrap.initial=true' is used - does this happen at every startup? Or only at install as the 'WEB-INF/bootraps/common' > feautre does?When 'magnolia.content.bootstrap.initial=true' is used - does this happen at every startup? Or only at install as the 'WEB-INF/bootraps/common' > feautre does?

Yes, at every start-up.

(If at every startup, I think this will cause problems with all of the files that anyone adds to the importer running at every startup and generating tasks everytime.)

Yes, that's a problem but I'd say it's only a symptom of another problem that tasks are created even when there are no actual changes (see MGNLCI-14)

jsimak changing same import file leads into same name siblings when task is approved

That happens only with yaml files, right? There's an issue with DataTransporter ignoring Import Behaviour when importing yaml files...

Comment by Jaroslav Simak [ 29/Jun/20 ]

rsiska
Also happened with XML files. And yes, i came to the same conclusion when debugging, it's the DataTransporter issue.

Generated at Mon Feb 12 00:22:39 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.