[ABTEST-451] Wording of Uplift description (Uplift vs. Minimum Detectable Effect) Created: 21/Apr/21  Updated: 19/May/21  Resolved: 17/May/21

Status: Closed
Project: A/B Testing
Component/s: None
Affects Version/s: None
Fix Version/s: 1.0

Type: Improvement Priority: Minor
Reporter: Laura Delnevo Assignee: Yen Lac Tue
Resolution: Fixed Votes: 0
Labels: VN-Implementation, VN-Testing
Remaining Estimate: Not Specified
Time Spent: 2.5h
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2021-04-21 at 15.35.13.png     PNG File Screenshot 2021-04-21 at 15.35.17.png    
Template:
Acceptance criteria:
Empty
Task DoD:
[X]* Doc/release notes changes? Comment present?
[X]* Downstream builds green?
[X]* Solution information and context easily available?
[X]* Tests
[X]* FixVersion filled and not yet released
[X]  Architecture Decision Record (ADR)
Documentation update required:
Yes
Date of First Response:
Epic Link: ABn GA tasks
Sprint: Content Mngmt 04
Story Points: 1

 Description   
  • Label change from Uplift to MDE
  • Description replaces "Uplift is the improvement (in %) you would like to achieve relative to the base conversion." with "Minimum Detectable Effect is the improvement (in %) you would like to detect relative to the base conversion."
       

Context

Feedback from Sandra:

  • Our UI assumes an uplift, while MDE measures the relative change. If a variation performs worse than the original, there would be no uplift
  • In addition, I believe that the description is not accurate. This metric does not relate to the improvement a user would like to achieve. Using the word “achieve” implies wishful thinking. Instead, it relates to the relative change in conversion rate a user would like to be able to detect. It is known as “Minimum Detectable Effect”

This is how Optimizely and AB Testy describe the Minimum Detectable Effect:

 Optimizely

AB Tasty

See that they call it “Minimum Detectable Effect” and use the word “detect” in their explanation.



 Comments   
Comment by Michael Duerig [ 11/May/21 ]

We use uplift synonymously with MDE. See also https://docs.google.com/presentation/d/1sQTrHcseg56KgvUmhvvJypd9qj_CjImfjvmU6Novv_g/edit?usp=sharing

Generated at Sun Feb 11 22:56:22 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.