Let us serve you, but don’t bring us down

Archive.org experienced an hour-long outage due to tens of thousands of requests per second for their public domain OCR files launched from 64 virtual hosts on Amazon’s AWS services. Despite the efforts of engineers who worked on the holiday weekend, joy was short-lived as another 64 addresses started the same activity and caused another hour outage. As a result, those who were planning to use Archive.org’s materials in bulk were advised to start slowly, and ramp up, and in the event of being blocked, they were asked to reach out. The Internet Archive can sustain usage, but users should use the service without bringing the website down in the process.

https://blog.archive.org/2023/05/29/let-us-serve-you-but-dont-bring-us-down/

To top