Researchers from the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) and Google Research have developed a diffusion model called Alchemist, which can alter the material properties of objects in images. This system allows users to adjust attributes such as roughness, metallicity, albedo, and transparency in both real and AI-generated images. By using a slider-based interface, users can make continuous changes to these properties, offering new possibilities for photo editing in areas such as video games, AI visual effects, and robotic training data.
Alchemist builds on the capabilities of the Stable Diffusion 1.5 model to focus on low-level attributes of objects, providing a more straightforward way to modify material properties in images. The system outperforms existing methods by offering precise control over these properties, enabling users to create visually striking images with ease.
One of the key advantages of Alchemist is its ability to provide precise control over material properties in images, which can be beneficial for tasks such as video game design, graphic design, and robotic training. By refining textures and material appearances, creators can enhance the realism of their projects and achieve desired visual effects with accuracy.
The researchers trained Alchemist on a synthetic dataset of 3D objects, allowing the model to learn how to manipulate material attributes in images. This approach offers a practical solution for controlling generative AI models, making it easier for users to achieve their desired visual effects.
While Alchemist has shown promising results, there are still some limitations to address, such as issues with inferring illumination and generating physically implausible transparencies. The researchers are working to improve the model’s performance and explore how it can enhance 3D assets for graphics at a scene level.
The team’s work will be presented at CVPR in June and was supported by funding from the National Science Foundation, as well as gifts from Google and Amazon.
