In this paper, the authors delve into the complex world of tree ensembles, shedding light on their effectiveness and the factors that contribute to their success. They propose a new perspective that views tree ensembles as adaptive and self-regulating smoothers, offering a deeper understanding of how they make predictions. Surprisingly, they challenge the common belief that the superior performance of tree ensembles is solely due to variance reduction, arguing that there are three distinct mechanisms at play. By quantifying the smoothing effect of ensembling, the authors aim to provide a more objective way of evaluating and comparing tree ensembles.
https://arxiv.org/abs/2402.01502