A/B testing is standard method for trialling different options. However, how do you know how long to run the trial for, and how much certainly can you infer from the results? In this paper we show you how we can run better trials, save money and get more confidence in the outcome by applying probability theories during the trials.