You can check a person’s vital signs — pulse, respiration and blood pressure — manually or by attaching sensors to the body. But a student in the Harvard-MIT Health Sciences and Technology program is working on a system that could measure these health indicators just by putting a person in front of a low-cost camera such as a laptop computer’s built-in webcam.
So far, graduate student Ming-Zher Poh has demonstrated that the system can indeed extract accurate pulse measurements from ordinary low-resolution webcam imagery. Now he’s working on extending the capabilities so it can measure respiration and blood-oxygen levels. He hopes eventually to be able to monitor blood pressure as well. Initial results of his work, carried out with the help of Media Lab student Daniel McDuff and Professor of Media Arts and Sciences Rosalind Picard, were published earlier this year in the journal Optics Express.
Poh suggests that such noninvasive monitoring could prove useful for situations where attaching sensors to the body would be difficult or uncomfortable, such as for monitoring burn victims or newborns. It could also be used for initial telemedicine screening tests over the Internet using a patient’s own webcam or even cell-phone camera.
Such a system could also be built into a bathroom mirror so that patients who need ongoing monitoring, or just people who want to keep track of their own health, could get pulse, respiration, oxygen saturation and blood-pressure readings routinely while they brush their teeth or wash up, displayed in a corner of the mirror.
Measuring brightness
The system measures slight variations in brightness produced by the flow of blood through blood vessels in the face. Public-domain software is used to identify the position of the face in the image, and then the digital information from this area is broken down into the separate red, green and blue portions of the video image. In tests, the pulse data derived from this setup were compared with the pulse determined by a commercially available FDA-approved blood-volume pulse sensor.
The big challenge was dealing with movements of the subject and variations in the ambient lighting. But Poh was able to adapt signal-processing techniques originally developed to extract a single voice from a roomful of conversations, a method called Independent Component Analysis, in order to extract the pulse signal from the “noise” of these other variations.
The system produced pulse rates that agreed to within about three beats per minute with the rates obtained from the approved monitoring device, and was able to obtain valid results even when the subject was moving a bit in front of the camera. In addition, the system was able to get accurate pulse signals from three people in the camera’s view at the same time.
Graduate student Ming-Zher Poh discusses a system he designed that could measure vital signs just by having a person sit in front of a low-cost camera such as a laptop computer's built-in webcam.
Video: Melanie Gonick
The concept of using a camera to detect such health information is not entirely new, but the innovations that allow the use of such low-cost camera equipment is. Fokko Wieringa, senior scientist at TNO Science & Industry in the Netherlands, published a paper describing a photographic pulse-detection system in 2005, but he says “the exciting thing about this new method is that they identify a fixed region on the face and track it (thus improving motion artifact tolerance), plus the clever processing method. The achieved gain in signal quality allows them to use a simple and cheap camera, even on moderately moving persons.” The ability to monitor multiple people at once is also new, he says. “These combined features are very original.”
The project won third place and a prize of $50,000 in June in the second annual Primary Healthcare competition run by CIMIT (Center for Integration of Medicine and Innovative Technology), an organization created by a group of physicians at Boston-area hospitals in collaboration with mechanical engineering faculty at MIT in order to develop new devices to meet clinical needs. The competition is open to teams of graduate or undergraduate students in engineering from anywhere in the U.S.
Poh continues to work on developing the capability to get blood pressure and blood-oxygen measurements from the same video images. Extracting such data from optical imagery should work, he says, since conventional blood oxygen sensors already work by using optical detection, although they use a dedicated light source rather than ambient lighting.
“It’s not going to be easy,” he says of the next steps. “But it theoretically should be possible.”
Wieringa, who was not involved in this work, says, “There are many pitfalls in the road from idea to approved medical device, and even more once a device is used in practice. However, the results achieved now look quite encouraging. To be honest, my hands are itching to exchange ideas and cooperate with these young researchers; it's exciting stuff.”
So far, graduate student Ming-Zher Poh has demonstrated that the system can indeed extract accurate pulse measurements from ordinary low-resolution webcam imagery. Now he’s working on extending the capabilities so it can measure respiration and blood-oxygen levels. He hopes eventually to be able to monitor blood pressure as well. Initial results of his work, carried out with the help of Media Lab student Daniel McDuff and Professor of Media Arts and Sciences Rosalind Picard, were published earlier this year in the journal Optics Express.
Poh suggests that such noninvasive monitoring could prove useful for situations where attaching sensors to the body would be difficult or uncomfortable, such as for monitoring burn victims or newborns. It could also be used for initial telemedicine screening tests over the Internet using a patient’s own webcam or even cell-phone camera.
Such a system could also be built into a bathroom mirror so that patients who need ongoing monitoring, or just people who want to keep track of their own health, could get pulse, respiration, oxygen saturation and blood-pressure readings routinely while they brush their teeth or wash up, displayed in a corner of the mirror.
Measuring brightness
The system measures slight variations in brightness produced by the flow of blood through blood vessels in the face. Public-domain software is used to identify the position of the face in the image, and then the digital information from this area is broken down into the separate red, green and blue portions of the video image. In tests, the pulse data derived from this setup were compared with the pulse determined by a commercially available FDA-approved blood-volume pulse sensor.
The big challenge was dealing with movements of the subject and variations in the ambient lighting. But Poh was able to adapt signal-processing techniques originally developed to extract a single voice from a roomful of conversations, a method called Independent Component Analysis, in order to extract the pulse signal from the “noise” of these other variations.
The system produced pulse rates that agreed to within about three beats per minute with the rates obtained from the approved monitoring device, and was able to obtain valid results even when the subject was moving a bit in front of the camera. In addition, the system was able to get accurate pulse signals from three people in the camera’s view at the same time.
Graduate student Ming-Zher Poh discusses a system he designed that could measure vital signs just by having a person sit in front of a low-cost camera such as a laptop computer's built-in webcam.
Video: Melanie Gonick
The concept of using a camera to detect such health information is not entirely new, but the innovations that allow the use of such low-cost camera equipment is. Fokko Wieringa, senior scientist at TNO Science & Industry in the Netherlands, published a paper describing a photographic pulse-detection system in 2005, but he says “the exciting thing about this new method is that they identify a fixed region on the face and track it (thus improving motion artifact tolerance), plus the clever processing method. The achieved gain in signal quality allows them to use a simple and cheap camera, even on moderately moving persons.” The ability to monitor multiple people at once is also new, he says. “These combined features are very original.”
The project won third place and a prize of $50,000 in June in the second annual Primary Healthcare competition run by CIMIT (Center for Integration of Medicine and Innovative Technology), an organization created by a group of physicians at Boston-area hospitals in collaboration with mechanical engineering faculty at MIT in order to develop new devices to meet clinical needs. The competition is open to teams of graduate or undergraduate students in engineering from anywhere in the U.S.
Poh continues to work on developing the capability to get blood pressure and blood-oxygen measurements from the same video images. Extracting such data from optical imagery should work, he says, since conventional blood oxygen sensors already work by using optical detection, although they use a dedicated light source rather than ambient lighting.
“It’s not going to be easy,” he says of the next steps. “But it theoretically should be possible.”
Wieringa, who was not involved in this work, says, “There are many pitfalls in the road from idea to approved medical device, and even more once a device is used in practice. However, the results achieved now look quite encouraging. To be honest, my hands are itching to exchange ideas and cooperate with these young researchers; it's exciting stuff.”
