The author delves into the intricacies of Log-Structured Merge trees (LSM), focusing on the use of lock-free concurrent skiplists in LSM implementation, despite the complexities involved. Skiplists are lauded for their ability to support multiple concurrent writers, with insights into insertion processes using compare-and-swap operations. Surprising findings include the optimal value of p in skiplists, derived from simulations suggesting p=1/e as the ideal choice. The author’s experimentation with skiplists and exploration of alternative data structures, like copy-on-write Btrees, highlight a deep dive into LSM simplicity and efficiency challenges. Follow the author on Bluesky for further updates and insights.
https://buttondown.com/jaffray/archive/fine-ill-play-with-skiplists/