[NPMCLI-120] Jumpstart can download and use webapps instead of bundles Created: 20/Mar/17  Updated: 28/Feb/18  Resolved: 30/Jan/18

Status: Closed
Project: Magnolia CLI
Component/s: None
Affects Version/s: None
Fix Version/s: 3.0.0

Type: Story Priority: Neutral
Reporter: Christopher Zimmermann Assignee: Milan Divilek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screen Shot 2018-01-23 at 15.33.39.png    
Issue Links:
causality
is causing NPMCLI-166 webapp chooser does not show all webapps Closed
is causing NPMCLI-167 update jumpstart to use new cloud web... Closed
dependency
is depended upon by NPMCLI-61 Jumpstart should allow getting bundle... Closed
relation
is related to MGNLTOMCAT-1 Supply a downloadable magnolia precon... Closed
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Date of First Response:
Epic Link: CLI v3
Sprint: Kromeriz 126, Kromeriz 130, Kromeriz 131, Kromeriz 132
Story Points: 8

 Description   

Jumpstart should support grabbing a magnolia webapp and a tomcat and putting them together in a working setup that functions the same as a bundle.

Reason:

  • When starting a real project, a demo project often has to be removed which can be complicated.
  • Magnolia provides more useful webapps than it provides bundles.
    For example: several of these webapps are more useful for starting actual projects than the bundles because they do not include demo projects.

Info on existing bundles and webapps:
https://documentation.magnolia-cms.com/display/DOCS/Bundles+and+webapps

Proposed options:

Propsal 1

Behaviour with no options

mgnl jumpstart Default becomes:

  • It prompts you on which edition you want (empty, community, enterprise-edition-standard, enterprise-edition-pro, cloud. Community is the default.)
  • If you choose community, ee-std, ee-pro, then It prompts you with or without demo (y/n) n is the default

Additional options

Specify the full webapp or bundle name:
mgnl jumpstart --webapp <webapp name>
mgnl jumpstart --bundle <bundle name>

ie
mgnl jumpstart --webapp magnolia‑enterprise‑standard‑webapp

Proposal 2

Same as above - but
mgnl jumpstart Default becomes:
It prompts you which one you want- from these options:

  • cloud
  • empty
  • community (default)
  • community-demo
  • enterprise-standard
  • enterprise-standard-demo
  • enterprise-pro
  • enterprise-pro-demo


 Comments   
Comment by Federico Grilli [ 05/Apr/17 ]

I'd rather not have tons of options for available webapps cause I'm afraid that would get messy, also considering that the offer might be increased in the future. I like the idea of a --list option to display available web app flavours.

Comment by Robert Kowalski [ 06/Apr/17 ]

Lets work in an iterative way on this and split this up.

  • Step 1: change jumpstart under the hood so that mgnl jumpstart is creating the current standard installation on its own. For the user nothing should change. This is already a huge step forward and can be merged / deployed as standalone improvement
  • Step 2: using the new helper methods and infrastructure which was created in Step 1 we can assemble all kinds of webapps. For this step we still need to define how a user decides for webapps.
  • Step 3: if decided, we can also seperately add the --list feature or other ways to discover available options. If small enough, this may also fit good into Step 2

This way we get several benefits:

  • The size of a PR is smaller, they are easier to review, test and reason about
  • We get some time to argue how step 2 or 3 could work
  • We can deliver the first improvement (slightly faster installation time) immediately and create value for our users early
Comment by Robert Kowalski [ 06/Apr/17 ]

Set Storypoints to 3, have to readjust as soon as we know how we want to continue in Step 2 / 3

Probably overall Storypoints are 5

Comment by Christopher Zimmermann [ 06/Apr/17 ]

Splitup plan sounds good to me.

Comment by Robert Kowalski [ 11/Apr/17 ]

depends on NPMCLI-131

Comment by Robert Kowalski [ 11/Apr/17 ]

Notes from discussion with creichenbach:

  • dev version for devs probably doesn't need public instance. maybe we can get rid of it. it would speed up each boot. what do light-module devs need?
  • the html file in the root is not neccessary for dev, but it might be confusing for existing users if it goes away
  • even if we just need the additional scripts and small mods for tomcat we still introduce another dependency which makes releasing harder
  • relying on git.magnolia-cms.com is not reliable to get those files, nexus is the single point of truth for people
Comment by Christopher Zimmermann [ 11/Apr/17 ]

"even if we just need the additional scripts and small mods for tomcat we still introduce another dependency which makes releasing harder"
Can you describe how much harder releasing would be? What exactly would need to be released in addition? Maybe you can ask in the morning standup who can provide an answer.

Could the zip build of these files be automated into one of the existing releases?

I think the additional work here is less then the work of providing additional bundles.

Comment by Robert Kowalski [ 12/Apr/17 ]

I think it has to be tested that everything still works for the CLI. Right now the bundles are self-contained and integration tested.

I will double check with someone who knows the release process better.

Comment by Cedric Reichenbach [ 12/Apr/17 ]

I'm not sure about the required effort for releasing those scripts and config files separately. If we did such a thing, it would probably be in the form of a blank, pre-configured, magnolia-ready tomcat. However, this would need some thorough architectural considerations first, about in which form to release, how to integrate with existing bundles etc.

On the other hand, if you have a look at what those files actually do, it's questionable if we need them at all. All they add on top of standard tomcat is:

  • Provide custom commands for startup/shutdown -> is hidden anyway in our case
  • Check maximum number of open files limit -> that's generally not relevant anymore since we moved away from jackrabbit (if I'm not mistaken)
  • Show the Magnolia banner at startup -> well...
  • Duplicate the author webapp if public isn't present -> not required if working with a single webapp while developing
  • Set some JVM args -> not sure about this one, but I guess we could live with defaults, or find another solution there
Comment by Christopher Zimmermann [ 08/Jun/17 ]

I like the point "it's questionable if we need them at all". Worth considering. An important consideration is simply the consistancy between what you get with the CLI vs the bundles. Ideally they would behave similiarly - for example so that existing docs and tutorials are correct.

It would be great to do the "thorough architectural considerations first, about in which form to release, how to integrate with existing bundles etc." and have a proposal or two about how to proceed.

Comment by Christopher Zimmermann [ 28/Nov/17 ]

I think a good next step would be to identify the tasks and estimate the effort that the CLI could get a webapp and get all other necessary files so that it behaves just like the bundles. IF this effort is not to high, then it's probably the best approach simply because Magnolia will behave in a consistent way - and so docs and trainers do not need to make exceptions.

Comment by Jaroslav Simak [ 11/Dec/17 ]

I can see couple of ways this can be done:

  1. create new maven module that would output tomcat with all the files in as a zip file
  2. get files from a tag that was used for release
  3. download existing bundle, remove webapps it contains, replace webapps it contains with the one user specified
  4. copy-paste files in the git to the npmcli module

Imho we should stay consistent and provide same functionality (=use same tomcat setup) for bundles and webapps when using mgnl jumpstart. I'd try to avoid confusion as much as possible, ideally user shouldn't notice if he's using bundle or webapp (except for webapp specific setup of course).

And for the solution, separate maven module for the tomcat seems most robust to me. The other ones are kinda "meh" .

Comment by Cedric Reichenbach [ 11/Dec/17 ]

I was involved the last time this was discussed (and then postponed), and the reason was basically that simple solutions (roughly 2. - 4. in your list) would be rather hacky, e.g. rely on content from our git repos. So it seems you came to the same conclusion as us regarding those.

What exactly would 1. mean? Would that be something similar to the bundle projects we already have, just a larger variety of supported webapps (and possibly without demo content)?

Comment by Jaroslav Simak [ 11/Dec/17 ]

Hey Cedric,

what i meant is to add new generated zip file that would be created upon release. That file would contain clean tomcat and stuff that is in our ce/ee bundle. Afaict we do create bundles for some of the modules we have (backup for example), this would be just yet another bundle containing mentioned stuff. Since we already know how to create such bundles, it shouldn't be a problem to add a maven plugin that would create it to the ce/ee bundles.

I hope that what i wrote makes sense.

Comment by Cedric Reichenbach [ 11/Dec/17 ]

Not sure if I understand correctly...

The backup bundle seems to be just a collection of related libraries/jars, but doesn't contain a tomcat or webapp if I'm not mistaken, right?

So what I understand is that:

  • Currently, we have just 3 tomcat bundles, community-demo, enterprise-pro-demo and enterprise-addons.
  • The CLI only supports the community-demo bundle.
  • We'd create and publish more tomcat-bundle projects, e.g. community-empty (containing empty-webapp), and let CLI users specify which one should be fetched.

This sounds like the cleanest solution without ripping apart our current setup, but has the disadvantage of having to create a bundle project for each webapp to be supported.

On a side note, from what I've seen and heard, the current state of bundle building is a bit messy, sprinkled with boilerplate config and could be refactored a bit before extending it even further. Then again, this sounds like a lot of headache. (feelsbadman)

Comment by Jaroslav Simak [ 11/Dec/17 ]

That's not what i meant, choosing backup module as an example was a poor idea, i will try to explain it better .

If you have a look at this pom.xml, theres maven-assembly-plugin that creates a bundle. It's descriptor can be found here. What i propose is either to add new configuration that would create "empty tomcat bundle" (no webapps inside), or add a new maven submodule to ce-packs/ee-packs, that would contain configuration for "empty tomcat bundle" and would contain the maven-assembly-plugin config in it's pom.xml. This way we'd have magnolia-ready tomcat tar.gz / zip file available for us on our nexus. We could download this file, unpack it, download webapp specified via cli, put it in and be ready to go.

Having bundles for all of the webapps seems too much overkill for me if we can generate tomcat with all the files we need without webapps.

I hope it's now clear what i meant .

Comment by Cedric Reichenbach [ 12/Dec/17 ]

Ah, now I see. So, like a barebone for dropping in any webapp. Sounds like a good plan to me.

Ideally, this could be used for the current bundles as well to mitigate the boilerplate config problem I mentioned above. If you look at the assembly config of community-demo and enterprise-pro-demo, they are largely the same. So e.g. the new barebone project would do most of that, and the bundle-specific ones would just drop in their corresponding webapps.

Comment by Jaroslav Simak [ 12/Dec/17 ]

Sounds good to me

Comment by Milan Divilek [ 23/Jan/18 ]

mgnl jumpstart --webapp magnolia‑enterprise‑standard‑webapp

magnolia‑enterprise‑standard‑webapp is not enough information to get the webapp. Also repository(e.g. magnolia.enterprise.releases) and groupId(e.g. info.magnolia.eebundle) are required.

So I decided not implement --webapp and --bundle options

Implemented solution
mgnl jumpstart prompts which webapp do you want - see
If required webapp is not community then it prompts also for credentials
Removed -e (enterprise) and -c (cloud) attributes.

Comment by Christopher Zimmermann [ 23/Jan/18 ]

Im fine with the list of all webapps right away - rather than the two part thing I proposed.

I think the default should be the magnolia-community-webapp.
(Rationale - if you start with the demo by accident, its hard to remove. If you start without the demo by accident, its easy to add.)

I still think we should support the --webapp parameter, it supports documentation and training exercises and automation. It could be limited to only work on the webapps that are "configured" in that list.

Im not sure how much use there would be to support getting ANY webapp by also being able to specify repository and groupid. (Seems nice for completeness, but I dont see a pressing usecase)

And, of course my original proposal to also be able to get bundles is totally redundant with being able to get webapps and was nonsense.

Comment by Milan Divilek [ 24/Jan/18 ]

I think the default should be the magnolia-community-webapp.
(Rationale - if you start with the demo by accident, its hard to remove. If you start without the demo by accident, its easy to add.)

I still think we should support the --webapp parameter, it supports documentation and training exercises and automation. It could be limited to only work on the webapps that are "configured" in that list.

that make sense, I'll implement it

Im not sure how much use there would be to support getting ANY webapp by also being able to specify repository and groupid. (Seems nice for completeness, but I dont see a pressing usecase)

Would be nice to have it, but i don't see use case if we limit download only from magnolia nexus..

Comment by Christopher Zimmermann [ 24/Jan/18 ]

Great.
OK, lets skip being able to get any webapp by specifying repo and groupid. We can see if a request or need comes in for it.

Comment by Christopher Zimmermann [ 25/Jan/18 ]

Error message:

Your mgnl-cli.json configuration ([path to mgnl-cli.json file]) does not contain the new tomcatDownloadUrl and availableWebapps nodes introduced in version 3.0.

Please either:
1. Remove or rename the mgnl-cli.json file and run the “mgnl customize-local-config” command in that directory to recreate the file with the new nodes - and then reapply your customizations, or
2. Edit the mgnl-cli.json file and add the following under the setupMagnolia node (Note that downloadUrl has changed):
[FULL CONFIG OF THOSE THREE NODES]

Generated at Mon Feb 12 04:46:43 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.