A growing number of tools enable users to create online data representations, such as charts, that are accessible for individuals who are blind or have low vision. However, most tools require an initial visual chart that can then be converted into an accessible format.
This presents barriers that hinder blind and low-vision users from constructing their own customized data representations, limiting their ability to explore and analyze important information.
A team of researchers from MIT and University College London (UCL) aims to revolutionize accessible data representations.
They developed a software system called Umwelt (meaning “environment” in German) that allows blind and low-vision users to create personalized, multimodal data representations without the need for an initial visual chart.
Umwelt is an authoring environment designed for screen-reader users, featuring an editor that enables users to upload a dataset and create customized representations, such as scatterplots, incorporating three modalities: visualization, textual description, and sonification (converting data into nonspeech audio).
The system, capable of representing various data types, includes a viewer that allows blind or low-vision users to interactively explore data representations, seamlessly switching between modalities to interact with data in different ways.
In a study with five expert screen-reader users, Umwelt was found to be useful and easy to learn. Users praised the interface for empowering them to create data representations, filling a gap they had previously experienced. They also noted that Umwelt could facilitate communication between individuals relying on different senses.
“We must remember that blind and low-vision individuals are not isolated. They exist within contexts where they want to discuss data with others,” says Jonathan Zong, an electrical engineering and computer science (EECS) graduate student and lead author of the paper introducing Umwelt. “I am optimistic that Umwelt will change how researchers approach accessible data analysis. Enabling full participation of blind and low-vision individuals in data analysis involves viewing visualization as just one component of a larger, multisensory puzzle.”
Joining Zong on the paper are fellow EECS graduate students Isabella Pedraza Pineros and Mengzhu “Katie” Chen; Daniel Hajas, a UCL researcher working with the Global Disability Innovation Hub; and senior author Arvind Satyanarayan, associate professor of computer science at MIT leading the Visualization Group in the Computer Science and Artificial Intelligence Laboratory. The paper will be presented at the ACM Conference on Human Factors in Computing.
Shifting the Focus Away from Visualization
The researchers have previously developed interactive interfaces that enhance the experience for screen reader users exploring accessible data representations. Through this work, they realized that most tools for creating such representations involve converting existing visual charts.
To decenter visual representations in data analysis, Zong and Hajas, who lost his sight at age 16, began co-designing Umwelt over a year ago.
At the outset, they recognized the need to rethink how the same data could be represented using visual, auditory, and textual forms.
“We needed to find a common ground among the three modalities. By creating this new language for representations and making the output and input accessible, the whole becomes greater than the sum of its parts,” says Hajas.
In developing Umwelt, they considered what makes each sense unique in how people use them.
For example, a sighted user can observe the overall pattern of a scatterplot while also focusing on different data points. In contrast, a person listening to a sonification experiences data linearly, as data are converted into tones played back one at a time.
“If you only focus on translating visual features into nonvisual ones, you overlook the distinct strengths and weaknesses of each modality,” adds Zong.
Umwelt was designed to provide flexibility, allowing users to easily switch between modalities when one better suits their task at any given time.
To use the editor, users upload a dataset to Umwelt, which uses heuristics to automatically generate default representations in each modality.
For instance, if the dataset contains stock prices for companies, Umwelt might create a multiseries line chart, a textual structure grouping data by ticker symbol and date, and a sonification using tone length to represent prices for each date, organized by ticker symbol.
The default heuristics are meant to assist users in getting started.
“In any creative tool, starting from scratch can be daunting. This is compounded in a multimodal tool where you have to specify things in three different representations,” Zong explains.
The editor links interactions across modalities, so changes made to the textual description, for example, are reflected in the corresponding sonification. Users can build a multimodal representation using the editor, switch to the viewer for initial exploration, and return to the editor to make adjustments.
Enhancing Communication About Data
Testing Umwelt involved creating diverse multimodal representations, from scatterplots to multiview charts, to ensure the system effectively represented different data types. The tool was then tested by five expert screen reader users.
Study participants generally found Umwelt useful for creating, exploring, and discussing data representations. One user described Umwelt as an “enabler” that reduced the time needed to analyze data. Users agreed that Umwelt could facilitate easier communication about data with sighted colleagues.
Future plans for the researchers include developing an open-source version of Umwelt for further development by others. They also aim to integrate tactile sensing into the software system as an additional modality, allowing for tools like refreshable tactile graphics displays to be used.
“Aside from the impact on end users, I hope that Umwelt can serve as a platform for exploring scientific questions about how people use and perceive multimodal representations and how we can enhance the design beyond this initial stage,” Zong concludes.
This research received support from the National Science Foundation and the MIT Morningside Academy for Design Fellowship.
