CAMBRIDGE, Mass. -- Countering the notion that color is unimportant for human face recognition, MIT researchers report in an upcoming issue of the journal Perception that when images are blurry, the brain relies on color cues to pinpoint identity.
For a long time, color was not considered crucial in recognizing faces because in experiments, people tended to identify accurately faces that were artificially colored, even in hot pink and electric blue.
In this latest study, Pawan Sinha, assistant professor of computation neuroscience at MIT, and Andrew W. Yip, a recent MIT graduate, found that when subjects look at images that are degraded--blurry or otherwise unclear--they actually do rely heavily on color. In addition to checking hair or skin hue, the brain probably uses color cues to determine where the hairline starts and ends, for example.
"Color cues do play an important role in face recognition," Sinha said. "Their contribution becomes evident when shape cues are degraded. [With blurry or faraway images], recognition performance with color images is significantly better than with gray-scale images."
In addition to helping understand the basic workings of the recognition processes in the brain, this research could lead to improved machine vision systems. Because the experimental stimuli used by Yip and Sinha mimics the appearance of human faces at a distance or of images taken with poor-quality cameras, the results also have implications for systems that attempt to recognize people videotaped by surveillance and security cameras. In previous work, Sinha developed software that, given blurry images, comes up with clearer pictures of what the original faces probably look like.
Other potential uses include better criminal identification kits that allow police artists to create sketches of crime suspects based on eyewitness descriptions.
IDENTIFYING FRIENDS AND FOES
Face recognition is extremely important in human beings' roles as social animals. A longstanding problem in the field of visual perception is understanding exactly how the human brain achieves its impressive face recognition abilities. What role do different attributes play in face recognition? Luminance, texture and the configuration of features may all play a role in the brain's judgment of identity.
Sinha is exploring several different avenues to find out exactly what the brain looks for in a face.
He tests how subjects identify images of faces that have been manipulated in a variety of ways; collects and analyzes caricatures; and even looks at how police artists sketch criminal suspects.
In his Hirshfeld Project, named for famed caricaturist Al Hirshfeld, Sinha is compiling and analyzing 5,000 digitized caricatures of celebrity and noncelebrity faces. "We are looking for the top 10 or 20 attributes on a face to identify the landmark points. We can come up with hundreds of attributes--the key is to narrow them down to the most significant," he said.
SEEING THE LIGHT
Sinha has recently initiated an ambitious project with the goal of observing children, blind since birth, who suddenly have their sight restored.
A fundamental unanswered question in neuroscience is: Does the brain have to learn to use visual cues to recognize faces, or is this a preprogrammed skill we are born with? If it is learned, what is the timeline over which the strategies develop, and how do internal representations of faces change during this time?
To answer these questions, Sinha has creating Project Prakash ("light" in Sanskrit) in which he is working with hospitals in India that treat blind children. India has a high incidence of congenital blindness and many children go untreated for years. "It's often ignorance or a lack of a small amount of money that prevents parents from seeking treatment," he said.
He hopes that by observing children who are old enough to indicate what they are seeing when they see for the first time, he will get an idea of the nature of the learning curve: Will the children have to learn how to see? Is there a critical period for the development of some abilities that, once it has passed, will not allow the person ever to have them?
"Through a combination of our theoretical and experimental work, we hope to eventually have a very good handle on how the human vision system encodes objects and faces," Sinha said.
This work is funded by the Defense Advance Research Projects Agency, the Alfred P. Sloan Foundation Fellowship in Neuroscience and the MediaLab Asia initiative.
