By combining a clever physical interface with computer-vision algorithms, researchers in MIT’s Department of Brain and Cognitive Sciences have created a simple, portable imaging system that can achieve resolutions previously possible only with large and expensive lab equipment. The device could provide manufacturers with a way to inspect products too large to fit under a microscope and could also have applications in medicine, forensics and biometrics.
The heart of the system, dubbed GelSight, is a slab of transparent, synthetic rubber, one of whose sides is coated with a paint containing tiny flecks of metal. When pressed against the surface of an object, the paint-coated side of the slab deforms. Cameras mounted on the other side of the slab photograph the results, and computer-vision algorithms analyze the images.
Not only do the images produced by GelSight, a new, portable imaging system from researchers in the Department of Brain and Cognitive Sciences, have a resolution that previously required expensive laboratory equipment, but they're 3-D, too. Here, GelSight images particles of ink spelling the word 'ink' on a piece of paper. (Image: Micah Kimo Johnson)
In a 2009 paper, Edward Adelson, the John and Dorothy Wilson Professor of Vision Science and a member of the Computer Science and Artificial Intelligence Laboratory, and Micah Kimo Johnson, who was a postdoc in Adelson’s lab at the time, reported on an earlier version of GelSight, which was sensitive enough to detect the raised ink patterns on a $20 bill. At this year’s Siggraph — the premier conference on computer graphics — Adelson and Johnson, along with graduate student Alvin Raj and postdoc Forrester Cole, are presenting a new, higher-resolution version of GelSight that can register physical features less than a micrometer in depth and about two micrometers across.
Moreover, because GelSight makes multiple measurements of the rubber's deformation, with light coming in at several different angles, it can produce 3-D models of an object, which can be manipulated on a computer screen.
Traditionally, generating micrometer-scale images has required a large, expensive piece of equipment such as a confocal microscope or a white-light interferometer, which might take minutes or even hours to produce a 3-D image. Often, such a device has to be mounted on a vibration isolation table, which might consist of a granite slab held steady by shock absorbers. But Adelson and Johnson have built a prototype sensor, about the size of a soda can, which an operator can move over the surface of an object with one hand, and which produces 3-D images almost instantly.
Adelson and Johnson are already in discussion with one major aerospace company and several manufacturers of industrial equipment, all of whom are interested in using GelSight to check the integrity of their products. The technology has also drawn the interest of experts in criminal forensics, who think that it could provide a cheap, efficient way to identify the impressions that particular guns leave on the casings of spent shells. There could also be applications in dermatology — distinguishing moles from cancerous growths — and even biometrics. The resolution provided by GelSight is much higher than is required to distinguish fingerprints, but “the fingerprinting people keep wanting to talk to us,” Adelson says, laughing.
Although GelSight’s design is simple, it addresses a fundamental difficulty in 3-D sensing. Johnson illustrates the problem with a magnified photograph of an emery board, whose surface, in close-up, looks a lot like marmalade — a seemingly gelatinous combination of reds and oranges.
“The optical property of the material is making it very complicated to see the surface structure,” Johnson says. “The light is interacting with the material. It’s going through it, because the crystals are transparent, but it’s also reflecting off of it.”
When a surface is pressed into the GelSight gel, however, the metallic paint conforms to its shape. All of a sudden, the optical properties of the surface become perfectly uniform. “Now, the surface structure is more readily visible, but it’s also measurable using some fairly standard computer-vision techniques,” Johnson explains.
GelSight grew out of a project to create tactile sensors for robots, giving them a sense of touch. But Adelson and Johnson quickly realized that their system provided much higher resolution than tactile sensing required.
Once they recognized how promising GelSight was, they decided to see how far they could push the resolution. The first order of business was to shrink the flecks of metal in the paint. “We need the pigments to be smaller than the features we want to measure,” Johnson explains. But the different reflective properties of the new pigments required the use of a different lighting scheme, and that in turn required a redesign of the computer-vision algorithm that measures surface features.
“I think it’s just a dandy thing,” says Paul Debevec, an associate professor of graphics research at the University of Southern California. “It’s absolutely amazing what they get out of it.” Debevec’s lab has been investigating the use of polarized light to compensate for the irregular reflective properties of some surfaces, but, he says, “they’re getting detail at the level that’s, for little patches, well more than an order of magnitude better than I’ve ever seen measured for these kinds of surfaces.”
As a graphics researcher, Debevec — whose PhD thesis work was the basis for the effects in the movie The Matrix — is particularly interested in what GelSight will reveal about the surface characteristics of human skin. “This kind of data is absolutely necessary to simulate that accurately,” Debevec says. “It’s pure gold.”
