Sampling is a powerful tool for extracting meaning from large datasets. By reducing data into representative samples using SQL, even FAANG-sized datasets become manageable. Built-in support for sampling, such as TABLESAMPLE clauses in SQL dialects, is available in some database systems. For more challenging scenarios, algorithms like A-ES allow for weighted, deterministic sampling without replacement. Clever tweaks, like using logarithms for numerical stability, ensure accurate results. By exploiting column-oriented storage formats, filtering pushdown, and deterministic pseudorandom functions, more efficient and controlled sampling is achievable. The A-ES algorithm leverages the theory of Poisson processes to provide fair and independent samples, making it a valuable tool for data analysis.
https://blog.moertel.com/posts/2024-08-23-sampling-with-sql.html