Uploaded image for project: 'A/B Testing'
  1. A/B Testing
  2. ABTEST-444

Conversion ratios are incorrect

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 1.0
    • None
    • None

    Description

      Steps to reproduce

      1.  Create an ABn test and expose it to some traffic. Wait until the data pipeline processed the data and results start showing up.
      2.  Manually calculate the conversion rates for each variant from the data in the ab-testing-data table: conversion_rate = converted / visited * 100
      3. Compare the manually calculated conversion rates with the ones in the result tab.

      Expected results

      The manually calculated conversion rates matches the ones in the result tab.

      Actual results

      The conversion rates in the result tab differ from the manually converted ones.

      Workaround

      None

      Anlysis

      The data pipeline calculates the conversion rate by the formula conversion_rate = converted / (visited + converted) * 100. This is caused by an incorrectly querying for all events in the formula's divisor instead of just the visited events.

      Checklists

        Acceptance criteria

        Attachments

          Activity

            People

              mduerig Michael Duerig
              mduerig Michael Duerig
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Checklists

                  Bug DoR
                  Task DoD