A/B Testing and Statistical Significance

Random assignment plus a big enough sample size is the closest thing to a controlled experiment the real world offers.

Definition

A/B testing is a controlled experiment that randomly assigns users to two conditions and measures whether an observed difference in outcomes is larger than what chance alone would produce.

Step 1

Predict first

In one or two sentences, what do you think this concept means? Wrong is fine; the small struggle is what makes the answer stick.