Minimum Sample Size for A/B Test (Power analysis)

The purpose of A/B testing is to find out if the treatment group performs significantly better than the control group in a certain success metric (e.g. conversion rate). How can we tell if it is “significantly” better? What can we do to tell if it is significantly better? The null hypothesis of our statistical test is that there is no difference between treatment and control group and we would like to prove that they are statistically different from each other (reject null hypothesis). In order for the tests to be statistically robust, it is crucial to do a priori power…