[ABTEST-444] Conversion ratios are incorrect Created: 16/Apr/21  Updated: 29/Apr/21  Resolved: 20/Apr/21

Status: Closed
Project: A/B Testing
Component/s: None
Affects Version/s: None
Fix Version/s: 1.0

Type: Bug Priority: Major
Reporter: Michael Duerig Assignee: Michael Duerig
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
Template:
Acceptance criteria:
Empty
Task DoD:
[X]* Doc/release notes changes? Comment present?
[X]* Downstream builds green?
[X]* Solution information and context easily available?
[X]* Tests
[X]* FixVersion filled and not yet released
[X]  Architecture Decision Record (ADR)
Bug DoR:
[X]* Steps to reproduce, expected, and actual results filled
[ ]* Affected version filled
Epic Link: ABn GA tasks
Sprint: Content Mngmt 02
Story Points: 2

 Description   

Steps to reproduce

  1.  Create an ABn test and expose it to some traffic. Wait until the data pipeline processed the data and results start showing up.
  2.  Manually calculate the conversion rates for each variant from the data in the ab-testing-data table: conversion_rate = converted / visited * 100
  3. Compare the manually calculated conversion rates with the ones in the result tab.

Expected results

The manually calculated conversion rates matches the ones in the result tab.

Actual results

The conversion rates in the result tab differ from the manually converted ones.

Workaround

None

Anlysis

The data pipeline calculates the conversion rate by the formula conversion_rate = converted / (visited + converted) * 100. This is caused by an incorrectly querying for all events in the formula's divisor instead of just the visited events.



 Comments   
Comment by Michael Duerig [ 26/Apr/21 ]

No differences any more when comparing the manually calculated conversions rates for the data in DynamoDB with the daily rates in the conversion rate chart on the result tab.

Generated at Sun Feb 11 22:56:17 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.