In the digital world, identifying the type of files we encounter is crucial for various reasons, such as ensuring user safety and maintaining security. The challenge lies in accurately and swiftly detecting the content of files, especially when dealing with a vast array of file formats. Current methods may not always be efficient or precise, leading to potential risks or misclassifications.
Meet Magika: An innovative file-type detection tool powered by artificial intelligence (AI) and deep learning. Magika uses a custom and highly optimized Keras model, weighing only about 1MB. What sets Magika apart is its ability to deliver precise file identification within milliseconds, even when running on a single CPU. This efficiency is a significant improvement over existing solutions.
Magika’s impressive capabilities are demonstrated by its evaluation on a dataset of over 1 million files across more than 100 content types, covering binary and textual file formats. The tool achieves a remarkable 99% or higher precision and recall, outperforming other approaches in the field. This level of accuracy is crucial for applications like Gmail, Drive, and Safe Browsing, where files need to be routed to the appropriate security and content policy scanners.
Metrics further highlight Magika’s efficiency, with an inference time of about five milliseconds per file after the model is loaded. Additionally, Magika supports batching, enabling users to process multiple files simultaneously and speeding up the overall detection process. Importantly, the inference time remains nearly constant, regardless of the file size, as Magika intelligently uses a limited subset of the file’s bytes.
Magika employs a per-content-type threshold system, ensuring that predictions are trustworthy. If needed, the tool can return a generic label like “Generic text document” or “Unknown binary data” when the confidence level is lower. Magika offers three prediction modes with varying error tolerance: high confidence, medium confidence, and best guess.
In conclusion, Magika stands out as a powerful and open-source solution for file type detection. Its versatility makes it an essential tool for enhancing user safety and security. While it already surpasses existing methods, the Magika team acknowledges room for improvement and encourages community feedback for further enhancements and support for additional content types.
Niharika is a Technical consulting intern at Marktechpost. She is a third year undergraduate, currently pursuing her B.Tech from Indian Institute of Technology(IIT), Kharagpur. She is a highly enthusiastic individual with a keen interest in Machine learning, Data science and AI and an avid reader of the latest developments in these fields.
