[NPMCLI-4] Improve yaml modifications Created: 05/Apr/16  Updated: 04/Nov/16  Resolved: 28/Apr/16

Status: Closed
Project: Magnolia CLI
Component/s: None
Affects Version/s: None
Fix Version/s: 0.0.1

Type: Bug Priority: Blocker
Reporter: Tomáš Gregovský Assignee: Aleksandr Pchelintcev
Resolution: Fixed Votes: 0
Labels: npm
Remaining Estimate: 0d
Time Spent: 3.75d
Original Estimate: Not Specified

Issue Links:
relation
is related to NPMCLI-53 Adding availability removes comments ... Closed
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Bug DoR:
[ ]* Steps to reproduce, expected, and actual results filled
[ ]* Affected version filled
Date of First Response:
Sprint: Basel 41
Story Points: 5

 Description   

scripts which are editing yaml configuration (e.g. add-availability) right now they take yaml code, transform it in to json, add/edit whats need and transform back to yaml. unfortunately during first transform from yaml to json, what doesn't looks like json is removed, so all comments '#...' or includes '!include' etc... ale lost during this edit.

potential fixes:
1. (idealy) do not use yaml > json > yaml conversion, but edit configuration directly in yaml format
or
2. improve yaml > json > yaml do not remove this things (propably !extends in future as well)
or
3. maybe use some wrapper around this special entries, before they are transformed to json, that transformation itself will not remove them, and then when transforming back to yaml, remove this wrapper as well
or ?



 Comments   
Comment by Christopher Zimmermann [ 11/Apr/16 ]

In the world of NPM and GULP there are lots modules that operate on JSON, which makes sense since these are javascript focussed tools.
I think we need to support transformation to JSON and back in our YAML files if possible, so that frontend developers can work with our YAML files.
For example, Tomas and I both naturally went for this solution (converting yaml to json), and for example If I search npm for "yaml merge", both results use the same technique (converting yaml to json, merging, and then back to yaml)
(https://www.npmjs.com/search?q=yaml+merge)

Some possibilities:

  1. Change our include syntax to be valid json. For example by using a keyword or keyword suffix.... "yamlinclude", "fileinclude", "magnoliainclude", "actions-yamlinclude".
  2. Create our own Json<>YAML transformers that preserve comments. Equivalent to https://www.npmjs.com/package/js-yaml (maybe by putting them in an extra comments JSON node.)

Big advantage of option 1 is that then all the existing npm libraries that use "js-yaml" will work out of the box. But of course it's bad that we have to change our syntax.

-

On the other hand, there's plenty of other advanced YAML syntax that is probably not going to be handled well in any transformation to JSON.
&, *, % etc. http://www.yaml.org/refcard.html

Comment by Aleksandr Pchelintcev [ 13/Apr/16 ]

I did some research on the topic today and here're some thoughts:

The current solution which does Yaml->JSON->YAML conversion is indeed flawed since:

  • it won't understand custom tags of ours
  • might produce distorted resulting Yaml (e.g. loss of anchors etc)

A possible solution is to handle this with decorators, but:

  • for simple cases (all files are in skeleton-ish form) decorators might be only confusing
  • scripts also optionally create necessary nodes like areas

One possible solution is to do the following:

  • Take a JS Yaml parser which follows the spec (so far only yaml-js seems to be applicable)
  • Do the following trick:
    • Parse/Compose the source Yaml file which produces a Yaml node graph
    • generate an availability snippet and parse it as well
    • inject the snippet's nodes into the source graph
    • serialise the resulting node graph, i.e. transform it back to Yaml

Such trick would require some customisation of Yaml parser/composer that would help us to track the node names (probably doable, but not trivial task). Provided we can resolve the injection point then - it should be trivial to do the rest.

Good thing here that we don't do actual Yaml conversion to anything, only parsing/serialising operations, which are lenient to the unknown tags.

Generated at Mon Feb 12 04:45:35 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.