[ABTEST-99] Research options to reduce AWS cost Created: 08/Nov/19  Updated: 26/Aug/22

Status: Open
Project: A/B Testing
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Ilgun Ilgun Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to ABTEST-411 Monitor cost of AWS resources whilst ... Open
dependency
is depended upon by ABTEST-132 Group up user timeline events and sen... Open
Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Date of First Response:
Epic Link: ABn Later
Story Points: 5
Team: AuthorX

 Description   

One idea is to group of events together and send them in batch to reduce API Gateway cost down. One API Gateway call per visitor might not scale well pricing-wise.

Another idea is to replace DynamoDB with AWS Timestream for the events and S3 for the aggregated data (timeline and conversions), which could help cutting infrastructure cost.

 

AC



 Comments   
Comment by Evzen Fochr [ 26/Nov/19 ]

Pricing according to latest data:
S3

  • storage 0.023$ /GB first 50G ....
  • PUT, COPY, POST, LIST requests (per 1,000 requests) - $0.005 (that mean 5$ per 1M requests)
  • GET, SELECT, and all other requests (per 1,000 requests) - $0.0004 (that mean 0.4 per 1M requests)

Dynamo

  • storage first 25g stored per month is free, $0.25 per GB-month thereafter
  • Write request units $1.25 per million write request units
  • Read request units $0.25 per million read request units

Reducing dynamo write requests
We can use TransactWriteItems which can contain up to 10 items and is counted as 2 write requests. That means with correct implementation we will get to $0.25 per 1M written rows

Reducing storage cost of dynamo

  • events per test are exported to file and stored in s3
    • after we do daily summary
    • or after we archive test in magnolia
  • whipped out completely and only test results are left in dynamo

Data transfer price - same for both services

For total size of dynamoDB item we can do computation https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/CapacityUnitCalculations.html !!! READ THE NOTE !!!

Generated at Sun Feb 11 22:52:51 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.