The search engine is preparing to move to a new server, thanks to a grant they received. Currently, the website may use either the old or new server until the migration is complete. The author apologizes for the chaotic nature of the post, as the past few weeks have been busy with setting up the migration and unexpected presentation invitations. The new server is powerful, with 512 GB of RAM and dual Epyc 7543 CPUs. The author discusses the decision to use Debian Bookworm as the operating system and explains the design choices for deploying the search engine on the server. They also address potential concerns with page faults and virtualization, concluding that it’s better to optimize the software rather than resort to virtualization. Additionally, the author highlights the importance of making the best use of available hardware and discusses the system’s resilience and recovery capabilities in the event of disk failures. They explain the process for regenerating data from backups or crawl data. The author concludes by mentioning the ongoing modifications to the software and the benefits of keeping data sets small. They note that the configuration is still in progress, but having both the old and new servers provides flexibility in case of downtime.
https://www.marginalia.nu/log/90-new-server-design/