In this blog post, I delve into the development of tea-tasting, a Python package designed for the statistical analysis of A/B tests. This package offers a range of statistical methods and approaches right out of the box, such as Student’s t-test, Bootstrap, and variance reduction with CUPED, as well as support for various data backends. What sets tea-tasting apart is its extensible API, which allows users to define custom metrics and use their preferred statistical tests. The package also includes detailed documentation to guide users through the advantages of utilizing tea-tasting for experiment analysis. (Word count: 100)
https://e10v.me/tea-tasting-analysis-of-experiments/