An Important Visual Cue for Empathetic Robots

It’s been four decades since the invention of the digital camera, and through relentless innovation the image sensors at their core have become incredibly small and sensitive.

Much of the innovation has focused on trichromatic color vision; how we create a world of kaleidoscopic hues from just three inputs – red, green, and blue (RGB). By experimenting with every possible configuration of red, green, and blue photodetectors, vision scientists have found they can capture the greatest spatial resolution and color sensitivity by using specific geometries of evenly spaced R, G, and B sensors.

So why have humans (and other primates) evolved a very different eye structure in which the green and red photoreceptors are packed closely and unevenly together? Until now, the prevailing theory has centered on foraging – that primate color vision has been tuned by evolution to help detect between subtle shades of green and red, which is useful for avoiding poisonous berries or for selecting ripening fruit against the green leaves in a tree.

But new research challenges that view, arguing that human color vision has evolved to provide an even more important capability – detecting “social signaling,” such as blushing or other facial color changes. The paper, Experimental evidence that primate trichromacy is well suited for detecting primate social colour signals, appears in the 14 June 2017 issue of Proceedings of the Royal Society Biological Sciences.

The authors start from the fact that humans and primates are social beings, living in groups and communicating in a variety of ways. For instance, some species of monkeys communicate by displaying redder coloration on their faces or genitals during mating, or in other social interactions. Similarly, our own facial color can change in social situations due to blushing, an important non-verbal signal.

The researchers had 60 human subjects view a series of digital photographs of female rhesus macaque monkeys, a species whose facial color changes with their reproductive status; for instance, a female’s face become redder when she is ready to mate. The scientists developed software that approximates the way colors look under different types of color vision, including the type of trichromatic vision seen in most artificial systems (with even spacing of the green and red photoreceptors). Some of the study’s subjects viewed photos of the monkeys’ faces that had been altered to appear as a human or primate would see them, while others saw pictures as a digital camera would. Other subjects saw photos as they would appear to a color-blind person.

The experiment found that overall, when asked to discriminate between the different colors of the monkeys in the photos, the subjects viewing the images using the human / primate visual system identified changes in the monkeys’ face coloring more accurately and more quickly than those using the camera system. According to co-author Amanda Melin of the University of Calgary, “these results support a rarely tested idea that social signaling itself, such as the need to detect blushing and facial color changes, might have had a role in the evolution or maintenance of the unusual type of color vision shown in primates, especially those with conspicuous patches of bare skin, including humans, macaques, and many others.”

This work has important practical implications, in particular for the design of robots, which are showing up in more and more homesstoresfactoriesoffices and hospitals. Incorporating these new insights on color vision will enable next-generation robots to get better at recognizing human emotions, such as when a person is pale from fatigue or red-faced with anger.

Longer term, I suspect that robot perception will integrate insights from other systems, especially the Facial Action Coding System (FACS) pioneered by psychologist Paul Ekman, which categorizes the physical expression of emotions by precisely tracking the movement of facial muscles.

The goal is to create robots that respond with appropriate, empathetic behaviors, so we come to view them as trusted companions, not just servants.