In recent years, web technologies have become more hostile towards web scraping. The shift towards dynamic apps built with frameworks like React and CSS-in-JS has made it difficult for web scrapers to extract data. Large companies like Facebook have also implemented measures to protect their public data, making it harder for machines to access. However, a new tool called puppeteer-heap-snapshot has emerged, allowing users to extract data from heap snapshots taken from browsers’ JavaScript runtimes. This method has proven successful in scraping data from closed platforms like YouTube, Facebook, Instagram, and Twitter. Although it has its limitations, it offers a more efficient and future-proof approach to web scraping.
https://www.adriancooney.ie/blog/web-scraping-via-javascript-heap-snapshots