[UMETRICS-30] Manage the lifecycle of the index Created: 02/Oct/20 Updated: 13/Nov/20 Resolved: 13/Nov/20 |
|
| Status: | Closed |
| Project: | Usage Metrics |
| Component/s: | server |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Neutral |
| Reporter: | Maxime Michel | Assignee: | Maxime Michel |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Template: |
|
| Acceptance criteria: |
Empty
|
| Task DoR: |
Empty
|
| Description |
|
Every day, usage metrics reports from Magnolia instances are sent to our AWS-managed ES server. This has been the case for months, and the problem is that at no point does the data come out. It is only a matter of time before this introduces larger problems (index health, Kibana responsiveness, disk space, billing). Kibana comes with a builtin tool to rotate the data. However, that feature is not available on the open-source version that AWS runs for us. This ticket is about researching and implementing:
Resources:
|
| Comments |
| Comment by Maxime Michel [ 19/Oct/20 ] |
|
A side note: all the work being done here is great, but what will truly make the index health turn green is that future indexes will have a number of shards set to 1, rather than the default of 5. Because we only have one node, shards couldn't be replicated to the target of 5, which led to the yellow health. Thanks jalonso for the tip! |
| Comment by Maxime Michel [ 21/Oct/20 ] |
|
Lambda now ready (PR merged), it only needs to be released, setup to run monthly, and QA'd. |
| Comment by Maxime Michel [ 12/Nov/20 ] |
|
Lambda is now configured on AWS, and there is an EventBridge rule to trigger it every first day of the month at midnight. |
| Comment by Maxime Michel [ 13/Nov/20 ] |
|
I will double-check on Dec. 1st whether the lambda kicked in successfully. |