[MAGNOLIA-7802] Content natural order not respected in search when searching by template Created: 15/May/20  Updated: 15/Mar/21  Resolved: 15/Mar/21

Status: Closed
Project: Magnolia
Component/s: None
Affects Version/s: 5.7.6, 6.1.5, 6.2.1
Fix Version/s: None

Type: Bug Priority: Neutral
Reporter: Jan Haderka Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: maintenance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2020-05-15 at 09.21.17.png     PNG File Screenshot 2020-05-15 at 09.21.35.png     PNG File Screenshot 2020-05-15 at 09.22.47.png    
Issue Links:
documentation
relation
Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)
Bug DoR:
[ ]* Steps to reproduce, expected, and actual results filled
[ ]* Affected version filled
Date of First Response:
Epic Link: Support
Sprint: Maintenance 9, Maintenance 10
Story Points: 3

 Description   

Timebox for investigation: 3 SP

Content natural order not respected in search when searching by template or by any other non indexed property.

To reproduce:

  1. startup dxcore with travel demo and login
  2. open jcr browser and go under /travel
  3. note all the pages using "travel standard" template ("travel-demo:pages/standard")
  4. reshuffle the pages
  5. go to jcr tools app and in queries tab run following query: select * from [mgnl:page] where [mgnl:template] = 'travel-demo:pages/standard'
  6. compare order of results

added screenshots to illustrate the discrepancy.



 Comments   
Comment by Jan Haderka [ 15/May/20 ]

issue can be made to go away by changing indexing configuration for the website workspace and including mgnl:template property to be indexed same as e.g. mgnl:tags ... and forcing reindexing of course.
The only reason above works is because it forces reindexing. But once nodes are moved around again, order is broken again

Comment by Michael Duerig [ 27/May/20 ]

I'd say this is a bug in jackrabbit. This has been reported before (apparently also in relation to Magnolia): https://issues.apache.org/jira/browse/JCR-3932

(If needed I could probably help to come up with and get a patch into Jackrabbit. )

Comment by Jan Haderka [ 27/May/20 ]

(If needed I could probably help to come up with and get a patch into Jackrabbit. )

I think that if you could it would be great. Seems to be there for quite while without any attention so unless we fix it, no one else probably would.

Comment by Yen Lac Tue [ 27/May/20 ]

Get these comments from mduerig
 

Fixing this ourself could be possible but would be somewhat lengthy process

Not sure what the situation is from the customer side.

But for now I'd say there is a work around (use xpath)

Fixing the bug would need to re-asses the effort (i.e. groom the bug ticket)

Actual steps for fixing would be:

- Develop a patch incl. test cases
- Get it into Jackrabbit trunk. possibly ask for review by the Jackrabbit team
- Backport to all affected Jackrabbit branches
- Release all branches that received a fix (there is a process involved here which is quite heave and takes about a week)
- Upgrade Magnolia to the fixed Jackrabbit version
Comment by Jan Haderka [ 01/Jun/20 ]

But for now I'd say there is a work around (use xpath)

mduerig fgrilli That is not really a workaround. The way how clients run into this is going over ${searchfn.searchPage()/searchContent() or by executing search query from template directly. But indeed as immediate remedy we should list it in known issues and mention it in documentation so that anyone searching for how to get order of templates knows not to use searchfn but instead resolve to executing jcr xpath query directly
Before deciding whether to commit to the effort of fixing it and going through the hurdle of releasing all branches, can we test if oak suffers from same issue or if it works fine there?

Comment by Michael Duerig [ 02/Jun/20 ]

can we test if oak suffers from same issue

I'll give it a shot.

Comment by Federico Grilli [ 02/Jun/20 ]

Thanks mduerig - I guess, for the time being, a mention under known issues in doc is in order. I'll create a proper docu ticket and link it here.

Comment by Michael Duerig [ 03/Jun/20 ]

I tried a couple of things but there seems to be no other workarounds here:

  • I checked whether there is some extra SQL2 syntax (e.g. DynamicOperand) to tweak queries but couldn't find anything helpful.
  • I checked whether there is some additional configuration options / feature flags to tweak SQL2 queries but apparently there are none that would help.
  • I traced the query implementation in Jackrabbit 2 to figure out why XPath honours the respectDocumentOrder setting and SQL2 doesn't. The reason is that there are two mostly separate query engines and the one responsible for SQL2 (and QOM) doesn't take that configuration setting into account. From looking at the implementation it is not clear to me how difficult it would be to implement this. There seems to be no "easy way out" though.
  • Finally I compared with Oak. Unfortunately Oak does not honour document order at all. Neither for XPath nor for SQL2. I didn't not check further though weather Oak has any feature flags, configuration options or syntax extensions that could help.
Comment by Jan Haderka [ 15/Mar/21 ]

The solution to this problem would require reimplementing query engine or changing repository altogether. Even newer version of JR (Oak) doesn't support natural order sorting.

Generated at Mon Feb 12 04:26:58 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.