[MAGNOLIA-1322] Differed storage of worklow expressions Created: 26/Jan/07  Updated: 23/Jan/13  Resolved: 23/Feb/07

Status: Closed
Project: Magnolia
Component/s: workflow
Affects Version/s: 3.0.1
Fix Version/s: 3.0.2, 3.1 M1

Type: Improvement Priority: Major
Reporter: Magnolia International Assignee: Magnolia International
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
relation
is related to MAGNOLIA-1352 HierarchyManager should be an interfa... Closed
is related to MAGNOLIA-1339 Keeping "backups" of proceeded workit... Closed
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Date of First Response:

 Description   

In an attempt to gain performance with the embedded workflow engine, we should try to defer the storage of expressions, since this hits jcr pretty wildly (every "step" of the workflow triggers at least one save on the hierarchyManager of the Expressions workspace)



 Comments   
Comment by Magnolia International [ 26/Jan/07 ]

commited in branches/magnolia-module-workflow-deferred-storage :
Instead of calling save on the HierarchyManager constantly, we just notify (ping() ) a thread. This thread will periodically check if it has been notified and it there are items to save, will call save after either 6 secs of inactivity (no notification) or will save anyway after 30 secs (of constant activity)

Comment by Nicolas Modrzyk [ 29/Jan/07 ]

Hey Gregory,

When a search is executed on the expression workspace and the hierarchy manager hasn't been save, I'm not sure all the proper nodes are returned in this new case.

There is a pretty poor chance of recovery either. Could this be turned on and off ?

Maybe some other options are:

  • to ask the workflow engine to not persist when the expressions hasn't change significantly ...
  • change the persistence in the jcr (only expand the nodes we need to search after, and blob the rest to the least number of nodes as possible)
  • could the node definitions be made simpler as well ?

I'm just afraid that customers will lose the state of their workflows (and that we need to give support on this after) if we only do a save every so often.

Comment by Magnolia International [ 29/Jan/07 ]

Nico, please let me first clarify that this is just a proposal/experiment - hence the work being done in a branch.

> When a search is executed on the expression workspace and the hierarchy manager hasn't been save,
> I'm not sure all the proper nodes are returned in this new case.

As far as I know, we don't do searches on this workspace.

> There is a pretty poor chance of recovery either. Could this be turned on and off ?

Of course

> to ask the workflow engine to not persist when the expressions hasn't change significantly ...

How do you define "significantly" ? Besides, as far as I understand, it stores a new expression everytime the workflow "progresses", i.e for every expression in the wf definition

> change the persistence in the jcr (only expand the nodes we need to search after, and blob the rest to the
> least number of nodes as possible)

Again, we don't do searches - as far as i can see, there's only one node per expression with a big blob of xml, and a couple of properties.

> could the node definitions be made simpler as well ?

We removed metadata from these nodes. We also disabled indexing, which also helped performace, but that currently has to be done manually after the workspaces were created...
More suggestions are welcome !

> I'm just afraid that customers will lose the state of their workflows (and that we need to give support on this after)
> if we only do a save every so often.

Well of course, we are too, and again, that's why we're just experimenting. OTOH, I don't think any "manual" data could be lost, since the "Store" (i.e where workitems are stored waiting for a user to proceed, our "worklist" equivalent I guess) is still saved synchronously.

I'm not really happy with the unsafe side of this solution either, but performance was totally unacceptable for a workflow-centered project, so we had to do some moves here. Tried many different ideas, and this was the only one that actually helped. We might have missed the obvious, of course; please suggest anything !

Comment by Magnolia International [ 07/Feb/07 ]

this is now optional.
You need a /modules/workflow/config/deferredExpressionStorage property set to true for the storage to be deferred.

Please review/comment !

Comment by Magnolia International [ 19/Feb/07 ]

todo : doc

Comment by Magnolia International [ 23/Feb/07 ]

commited to branches/magnolia-3.0 and trunk

Comment by Magnolia International [ 23/Feb/07 ]

documented - currently at
http://documentation.magnolia.info/magnoliaAuthor/draft/en/developer/modules/reference/workflow-configuration.html

Generated at Mon Feb 12 03:25:48 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.