The author embarked on a challenging project to create MonKafka, a Kafka Broker implementation, delving into the complexities of compression algorithms supported by Kafka, including GZIP, Snappy, LZ4, and ZSTD. The post then delves into a detailed explanation of compression techniques such as Run-Length Encoding (RLE), Lempel-Ziv (LZ), and Huffman Coding, focusing on lossless compression. The author humorously recounts their learning journey, highlighting Professor Bill Bird’s entertaining lecture on GZIP. The post concludes with a peek at the Golang implementation of the DEFLATE algorithm, showcasing token representations and hash functions used for back references.
https://cefboud.github.io/posts/compression/