[NPMCLI-81] Identify best practice for files not used by magnolia Created: 26/Jan/17  Updated: 21/Feb/17  Resolved: 21/Feb/17

Status: Closed
Project: Magnolia CLI
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Neutral
Reporter: Christopher Zimmermann Assignee: Robert Kowalski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Date of First Response:
Epic Link: CLI: Resource handling
Sprint: Basel 84
Story Points: 1

 Description   

There are several instances of files that are related to a light module, that are not directly used by magnolia. Where should they be stored? How should these files be handled?

While we must preserve the freedom for developers to handle the files however they want - we should provide a convention so that light modules can be easily shared, understood, and used.

To keep in mind is both the case of companies developing a library of modules for in-house use, and in particular of developers sharing components in places like github and npm.

Some questions:

  • Should all the files be stored in the light module? Or should some of the files be stored in a separate location?
  • Where should tests be stored?
  • Where are examples stored?
  • Where are files to be processed - like less, sass, ES6, TS stored?
  • If all the files are in the light module - should something be done to remove them when using them in production - for security, tidiness, and to minimize resource consumption of the file watcher?


 Comments   
Comment by Robert Kowalski [ 26/Jan/17 ]

First brain dump:

Compiling JS

For compiled JS files, e.g. with babel, its common practice to store them in a folder called `src` and compile them to a folder called `lib` with a pre-publish hook. Another approach, especially used by frontend libs is using the folder dist, especially if you have mixed content that is compiled (e.g. JS, CSS), see https://github.com/JedWatson/react-select

Hooks

A prepublish hook is called when you run `npm publish`, before npm creates and sents the tarball (i.e. the package). You basically add your compile commands them under `scripts` as `prepublish`. Its advised to run that step before you publish and not as an installation hook. This way the downloaded "artifact" is always the same and can't differ depending on the configurations a local machine has.

Real-world example: https://github.com/apache/couchdb-nmo/blob/master/package.json#L7-L9

Reference: https://docs.npmjs.com/misc/scripts

Tests

In the Node world tests usually go to the folder tests, thats why test-runner like `mocha` default to this directory.

Ignoring tests - the npm world

With npm they aren't ignored by default with some kind of magic, but people can use .gitignore and/or .npmignore files to exclude them from publishing. Some people do, some not, and sometimes people receive pull requests adding it to ignore-list. For some cases there are two sides which will never agree: Isaac always includes unit tests in published packages, whcih can be handy when you inspect the package in case of an error. Others argue that it takes space / bandwith. We have to keep in mind that those packages are run in another context and are not exposed on a webserver.

For magnolia it definitely makes sense to ignore some files. If it is done by automatically we have to take care to communicate that in a prominent way, to avoid that users run into trouble, debug for hours and get mad. If not done automatically, the examples should always make use of the ignore-flag. In a publishing like process we could also print a warning if a test folder is found, but not added to a ignore-file

Special case frontend libs

For Frontend libs people sometimes even check in the generated code into git, so people can just drag and drop the files out of the cloned repo (example from a popular lib: https://github.com/JedWatson/react-select/tree/master/dist) – they also have SASS that they compile to css btw

Comment by Robert Kowalski [ 03/Feb/17 ]

https://github.com/robertkowalski/mgnl-bobby

and

https://github.com/robertkowalski/mgnl-custom

are leveraging the `.npmignore` and a prepublish hook to merge and compile the frontend JS.

Comment by Robert Kowalski [ 14/Feb/17 ]

Can i configure Magnolia not to serve certain files? Is the list of currently ignored files (I think `.git` directories are ignored) hardcoded somewhere? ping fgrilli

Comment by Federico Grilli [ 14/Feb/17 ]

rkowalski I think you're looking for this https://documentation.magnolia-cms.com/display/DOCS/MIME+type+mapping#MIMEtypemapping-RestrictingresponsestoconfiguredMIMEtypes

If you're looking for files observed by Magnolia then in magnolia.properties you can configure them with a regex

# Pattern to define which resources should be observed by ClasspathScanner
magnolia.resources.classpath.observation.pattern=.*\\.(ftl|yaml)$
Comment by Robert Kowalski [ 21/Feb/17 ]

Should all the files be stored in the light module? Or should some of the files be stored in a separate location?

I started to put my dev-related sourcefiles into a folder called `_dev` (I stole this from czimmermann). There are still files on the project level, like the .git folder and other project related files you don't want to share. The final light-module shouldn't contain them. One way to do that would be to set a folder called ``

For LM shared via npm they should be filtered out before a publish with a .npmignore. Some are by default, but _dev and other files would be a nice addition.

The both build-projects for Light-Mdoules "furbie" and "super-furbie" work whitelist based. Reason: the folder-structure for a Light-Module is in big parts always the same. With the many different frontent-tools and their config files its easier to maintain the whitelist containing the scaffolded folder-list by `mgnl create-light-module`.

Comment by Robert Kowalski [ 21/Feb/17 ]

czimmermann I think we can close this one as we worked out a good set of best practices in the last weeks. I put the ticket on tomorrows agenda

Comment by Christopher Zimmermann [ 21/Feb/17 ]

Conclusion: We recommend

  • Following established convention by putting all toolchain config files such as .babelrc etc at the root directory.
  • Putting other project related files in directory '_dev' - such as all src/resource files such as js, es6, ts, less, sass files and test scripts, and bootstraps for the tests, and sample content.
  • .npmignore file is used to specify files that are not included in npm package - and therefore will be present/cluttering packages/light modules retrieved from npm. (Users who want the source can get it from the linked source location if present ie github.
  • Note that the philosophy is to have package built before publishing on npm - so package is ready to use in Magnlla light-modules directory. (including any necessary declared npm package dependencies such as js project.
Comment by Christopher Zimmermann [ 21/Feb/17 ]

Will be documented here: https://wiki.magnolia-cms.com/display/PMTEAM/Conclusions+on+light+module+on+npm+topics

Generated at Mon Feb 12 04:46:20 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.