Google has open-sourced Magika, its AI-powered file-type identification system, to assist others in accurately detecting binary and textual file types. Magika utilizes a custom, highly optimized deep-learning model that enables precise file identification within milliseconds, even on a CPU. Accurate file-type detection has historically been challenging due to variations in file formats and the need for handcrafted rules. Magika outperforms other tools by 20% on a benchmark of 1 million files, particularly excelling in identifying textual files. It is currently used by Google to improve the identification accuracy of file types and will be integrated with VirusTotal for enhanced cybersecurity.
https://opensource.googleblog.com/2024/02/magika-ai-powered-fast-and-efficient-file-type-identification.html