The author discusses the limitations of Pearson’s correlation and similar measures when comparing two sequences of data that may not have a linear relationship. They introduce Hoeffding’s D as a more robust measure of dependency that can handle non-linear relationships and outliers better. Hoeffding’s D calculates a statistic based on the difference between the observed joint distribution of ranks and what would be expected if the two sequences were independent. This unique approach helps quantify the dependency between the sequences more accurately. The author presents Hoeffding’s D as a powerful and general measure of association or dependence, making it a valuable tool for analyzing complex relationships in data.
https://github.com/Dicklesworthstone/hoeffdings_d_explainer