[ABTEST-377] Create a running ABn Test on Travel Created: 15/Dec/20 Updated: 07/Apr/21 Resolved: 04/Feb/21 |
|
| Status: | Closed |
| Project: | A/B Testing |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Neutral |
| Reporter: | Laura Delnevo | Assignee: | Chuong Doan Huy |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | 3d 2.5h | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Template: |
|
||||||||
| Acceptance criteria: |
Empty
|
||||||||
| Task DoR: |
Empty
|
||||||||
| Epic Link: | ABn Internal Beta | ||||||||
| Sprint: | Add-Ons 54, Add-Ons 55, Add-Ons 56 | ||||||||
| Story Points: | 1 | ||||||||
| Description |
|
Create running ABn Tests on Travel in order to accommodate the following scenarios: * scenario 1 (default): with the following settings: baseConversion: 10%, Uplift: 20%, confidenceRate: 85%, allocation: 100%, create a running test with 3 variants (A, B and C). Original page is travel/tour-type select one variant to be the winner and redirect traffic so that it has more than 11% of conversions select one variant to be the loser, and redirect less traffic than on the original variant (variant-A) * scenario 2: keep the same settings/traffic as in the scenario A, but decrease the allocation to 50. Original page is travel/destination this test should reach the significant results in double the time * scenario 3: keep the same settings/traffic as in the scenario A, but decrease the confidenceRate to 80%. Original page is travel/tour this test should reach the significant results faster than the test in scenario A, observe the estimated running time * scenario 4: keep the same settings/traffic as in the scenario A, but restrict the test to only one segment. Original page is travel/about in the traffic generator, find a way to signal that some traffic is coming from the segment group (e.g.: cookie, header_param), and see that the test is reaching significant results slower than in scenario-A
|