ColorBless: Augmenting Color Information with Binocular Luster Effect

Overview

Most digital colorblind techniques change the underlying colors to make them less confusing, like the Daltonize technique shown below.

Screenshot 2016-10-30 11.50.18.png

In the Daltonized Colorblind Image, even though the orange and banana are distinguishable now, the colors of the orange and tomato seen by normal color vision people (shown in Daltonize Original Image) become a bit odd and off.

ColorBless is the world’s first technique that enables normal and colorblind people to view the same screen at the same time without changing the underlying color nor adding any color labels. What’s more, it’s much faster than existing techniques to tell confusing colors apart.

colorbless image.png

Colorblind user, wearing 3D glasses, can use ColorBless to differentiate confusing colors without affecting the colors seen by normal color vision people. Moreover, our experiments suggest that it is more effective than existing colorblind techniques.

Problem Statement

Colorblind people have difficulties distinguishing certain colors apart, like certain reds and greens, or blues and yellows. Problems arise when these colors are used to represent information.

Screenshot 2016-10-30 12.31.53.png

An illustration of the colors different colorblind people sees.

To solve these problems with digital images, most colorblind users either pay more attention to the differences (through interviews, we learned that they can see minute differences if they pay close attention), or they can use algorithm programs that recolor a particular image to create a bigger color contrast (e.g. Daltonize).

However, because recoloring programs rely on color-based coding, it still takes the colorblind users some time to process the information since they are less confident with deciphering colors in general. On top of that, since the algorithm recolors all colors in the image, and not just the confusing ones, it affects the color perception of normal people who looks at the same image. These scenarios are quite commonplace, like in a business meeting. This is problematic if the colors contain some semantic meanings that are important to the interpretation of the information.

The problems above motivated us to explore new technologies that have the affordances to tackle them. We focused on stereoscopic 3D to develop ColorBless, which solves both problems at the same time. ColorBless leverages the fact that in a 3D watching scenario, people who wear 3D glasses and those who don’t would see different visuals on the image, and ColorBless uses the 3D glasses to apply a binocular effect called the luster effect (illustration video here) only to the confusing colors in the image to distinguish them apart.

Users

ColorBless supports colorblind users and scenarios where normal and colorblind users are working together on visual-based information. There are approximately 200 million colorblind people all around the globe (8% of all male and 0.5% of female) who can benefit from our technique. ColorBless solves two very important problems for both colorblind and normal people. First, it allows colorblind people to differentiate confusing colors in digital images faster than any existing techniques, putting them in the same efficiency bracket as normal people. Second, applying ColorBless onto digital images doesn’t change the underlying color, therefore the color meaning is not altered. With ColorBless, normal and colorblind people can finally work together.

Process

I lead the ColorBless R&D team that consists of 6 members, including two software engineers, three research advisors, and a project lead (me). In the beginning, I first explored the design space of the luster effect in the colorblind domain by conducting numerous pilot studies and interviews with actual colorblind users. I worked with the engineers to develop the prototype for the pilot studies. I also conducted literature review at the same time to better understand the colorblind techniques available. After each pilot, I collected the feedback and used those insights to improve our technique as well as to develop the next experimental study. Through this iterative approach, I was able to understand their problem in tandem with how technologies can potentially help them, and subsequently, narrowed down the design space and maneuver around the trade-offs to come up with a solution that works. I also designed the necessary psychophysics studies to ensure the user experience is good for our target users, such as determining the intensities of the luster effect that are comfortable, as well as the luster ranges that are perceivable.

Constraints & Main Findings

We had a very ambitious idea in the beginning of this project — to provide different levels of luster effect to different severity of colorblind conditions. To achieve this, we need to perform quantization function on the luster intensity based on the brightness differences seen with the left and right eye (with the 3D glasses). However, we faced time and technical constraints since we only had very short time to come up with a usable prototypical system. This forced us to think outside the box to generate simpler, but more creative solutions to use luster effect to distinguish confusing colors. More importantly, this constraint drove us to understand the core of the problem and motivated us to be laser-focused on solving them. When the project was completed, we ended up with a solution that is much simpler yet more effective than the one we devised in the first place. It outperforms recoloring technique (Daltonize) and augmentation technique that uses different line gradient to represent different colors in terms of color differentiation time, color retention on the original image, and subjective preferences in different scenarios. For more details, please refer to our report.

Design Process Stories

In the beginning, we learned that apart from seeing 3D effects, a stereoscopic display can also produce interesting binocular visual effects for the glass wearers without changing the underlying images. Based on this property, we came up with the idea of exploring these effects as an artifact to differentiate two different colors. Immediately, this reminded us of colorblind. However, we did not understand the affordances and possible advantages of the luster effect yet. Therefore, we invited two colorblind participants to view this effect on an image with confusing colors, and we immediately noticed that they can differentiate colors rapidly and unequivocally. We used these insights to review the colorblind literature, and we found very specific issues of the existing colorblind solutions that our technique could solve.

With this newfound focus, we improved the technique in a targeted approach by designing a series of experiments to provide us with the parameters we needed. Throughout the process, we had to abandon a lot of cool and ambitious ideas that might be useful to the users but did not solve the problems that existing techniques couldn’t solve.

Lessons Learned 

This is the first project I worked on during my graduate study. Before that, I had no experience in human-computer interaction and user experience. I learned three main lessons from this project that significantly guide my future work.

First, working on this project taught me the value of imposing meaningful constraints in a project without diminishing the usefulness of the artifact (see constraints).

Second, the design process taught me to be comfortable with killing my own ideas even though a lot of time had been spent on them, and the importance of orienting the problem as the principle while developing a product.

Third, I learned that it is important to have a conviction on the design process and don’t be afraid to defend the problem you have identified. At the same time, as a beginning researcher in the field, it is also crucial to keep an open mind on the different perspectives. Treading between having conviction and being open-mindedness is a balance that I have learned to be more aware of in my future work.

Advertisements