In the background of image recognition software that can identify our friends on social media and wildflowers in our yard are neural networks, a type of artificial intelligence inspired by how our own brains process data. While neural networks quickly process data, it is difficult to trace the origin of errors that are obvious to humans – like mistaking a Converse high-top for an ankle boot. This limitation restricts their use in critical areas such as healthcare image analysis or research. However, a new tool developed at Purdue University simplifies the process of finding these errors, making it as easy as spotting mountaintops from an airplane.
“In a sense, if a neural network were able to speak, we’re showing you what it would be trying to say,” said David Gleich, a Purdue professor of computer science in the College of Science who developed the tool. The tool, featured in a paper published in Nature Machine Intelligence, helps identify areas where the network requires more information to fulfill a given task. Gleich advises using this tool for high-stakes neural network decision scenarios or image prediction tasks.
The code for the tool is available on GitHub, along with demonstrations of its use cases. Gleich collaborated on the research with Tamal K. Dey, also a Purdue professor of computer science, and Meng Liu, a former Purdue graduate student who earned a doctorate in computer science.
During testing, Gleich’s team discovered neural networks mislabeling images in databases of various subjects, including chest X-rays, gene sequences, and apparel. For instance, the neural network repeatedly mistook images of cars from the Imagenette database for cassette players. This occurred because the images were sourced from online sales listings, including tags for the cars’ stereo equipment.
Neural network image recognition systems are algorithms that mimic the weighted firing pattern of neurons to analyze and identify images. These systems are trained with a “training set” of images that provide data on each pixel, tags, and other information, along with the image’s classification. Using this training set, the network learns and extracts the necessary information to match input values with the appropriate category. This information, represented as an embedded vector, calculates the probability of an image belonging to each category. Typically, the correct identity of the image lies within the category with the highest probability.
However, these embedded vectors and probabilities do not correlate with the decision-making process humans recognize. The network produces an embedded vector of 128 numbers that do not correspond to physical features but enable image classification. In other words, the inner workings of a trained system are like a “black box” filled with unrecognizable numbers across multiple layers.
Gleich’s approach visualizes the relationship that the computer perceives among all the images in a database, rather than tracing the decision-making path of a single image through the network. This approach provides a bird’s-eye view of how the neural network organizes the images. The relationship among the images is based on the embedded vectors and probabilities generated by the network. To enhance the view and identify areas of confusion, Gleich’s team developed a method of splitting and overlapping the classifications. They then map these relationships onto a Reeb graph, a tool from the field of topological data analysis. The graph represents related image groups as dots, color-coded by classification. Overlapping dots of different colors indicate areas of high probability for multiple classifications. By zooming in on these overlapping dots, areas of confusion, such as the car labeled as both a car and a cassette player, become apparent.
“What we’re doing is taking the complex information generated by the network and providing people with an understanding of how the network sees the data at a macroscopic level,” Gleich explained. “The Reeb map highlights the important aspects, the major groups, and their relationships, allowing us to identify errors.”
“Topological Structure of Complex Predictions” was supported by the National Science Foundation and the U.S. Department of Energy.
