[UMETRICS-20] Do not record an index line for every single module (server change) Created: 16/Sep/19  Updated: 26/Sep/19  Resolved: 25/Sep/19

Status: Closed
Project: Usage Metrics
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Neutral
Reporter: Maxime Michel Assignee: Maxime Michel
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
Acceptance criteria:
Empty
Task DoD:
[ ]* Doc/release notes changes? Comment present?
[ ]* Downstream builds green?
[ ]* Solution information and context easily available?
[ ]* Tests
[ ]* FixVersion filled and not yet released
[ ]  Architecture Decision Record (ADR)

 Description   

Kibana can't process nested properties very well, in our case the report object that Magnolia sends is made out of two arrays, one for modules, one for sites.

We only get totals from sites which is fine, but we transform every single module to a flattened property. This in turn has made us reach the max. field limit of 1000 once, and we are close to hitting the new limit (2000).



 Comments   
Comment by Maxime Michel [ 25/Sep/19 ]

The solution to this problem is to define new mapping so that every single string property doesn't result in two fields (ES weird default setting that wasn't helping us) and keep an eye on it for the time being. Index lifecycle policies would help but are not available on AWS ES.

Generated at Mon Feb 12 11:09:32 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.