For the past 3 years, I’ve been dealing with memory issues in Mess With DNS resulting in OOM killing. This only happened once a day and was not a big concern until last week when it impacted the backup script. To address the problem, I explored different solutions such as using SQLite, a trie structure, and optimizing memory usage by deduplicating data. Ultimately, switching to netip.Addr instead of net.IP saved 70MB of memory. Despite some minor drawbacks like slower lookups, the overall memory usage decreased, showing that simple adjustments can make a significant difference. Through this process, I discovered new tools and approaches, making it a rewarding learning experience.
https://jvns.ca/blog/2024/10/27/asn-ip-address-memory/