Skip to content ↓

Detecting emotions with wireless signals

Measuring your heartbeat and breath, CSAIL device can tell if you’re excited, happy, angry, or sad.
Watch Video
Press Inquiries

Press Contact:

Adam Conner-Simons
Phone: 617-324-9135
MIT Computer Science & Artificial Intelligence Lab
Close
Left to right: PhD student Fadel Adib, PhD student Mingmin Zhao, and Professor Dina Katabi pose with their EQ-Radio device, which can detect emotion using wireless signals.
Caption:
Left to right: PhD student Fadel Adib, PhD student Mingmin Zhao, and Professor Dina Katabi pose with their EQ-Radio device, which can detect emotion using wireless signals.
Credits:
Photo: Jason Dorfman/MIT CSAIL
Professor Dina Katabi (center) explains how PhD student Fadel Adib's (right) face is neutral but that EQ-Radio's analysis of his heartbeat and breathing show that he is sad.
Caption:
Professor Dina Katabi (center) explains how PhD student Fadel Adib's (right) face is neutral but that EQ-Radio's analysis of his heartbeat and breathing show that he is sad.
Credits:
Photo: Jason Dorfman/MIT CSAIL

As many a relationship book can tell you, understanding someone else’s emotions can be a difficult task. Facial expressions aren’t always reliable: A smile can conceal frustration, while a poker face might mask a winning hand.

But what if technology could tell us how someone is really feeling?

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed “EQ-Radio,” a device that can detect a person’s emotions using wireless signals.

By measuring subtle changes in breathing and heart rhythms, EQ-Radio is 87 percent accurate at detecting if a person is excited, happy, angry or sad — and can do so without on-body sensors.

MIT professor and project lead Dina Katabi envisions the system being used in entertainment, consumer behavior, and health care. Film studios and ad agencies could test viewers’ reactions in real-time, while smart homes could use information about your mood to adjust the heating or suggest that you get some fresh air.

“Our work shows that wireless signals can capture information about human behavior that is not always visible to the naked eye,” says Katabi, who co-wrote a paper on the topic with PhD students Mingmin Zhao and Fadel Adib. “We believe that our results could pave the way for future technologies that could help monitor and diagnose conditions like depression and anxiety.”

EQ-Radio builds on Katabi’s continued efforts to use wireless technology for measuring human behaviors such as breathing and falling. She says that she will incorporate emotion-detection into her spinoff company Emerald, which makes a device that is aimed at detecting and predicting falls among the elderly.

Using wireless signals reflected off people’s bodies, the device measures heartbeats as accurately as an ECG monitor, with a margin of error of approximately 0.3 percent. It then studies the waveforms within each heartbeat to match a person’s behavior to how they previously acted in one of the four emotion-states.

The team will present the work next month at the Association of Computing Machinery’s International Conference on Mobile Computing and Networking (MobiCom).

How it works

Existing emotion-detection methods rely on audiovisual cues or on-body sensors, but there are downsides to both techniques. Facial expressions are famously unreliable, while on-body sensors such as chest bands and ECG monitors are inconvenient to wear and become inaccurate if they change position over time.

EQ-Radio instead sends wireless signals that reflect off of a person’s body and back to the device. Its beat-extraction algorithms break the reflections into individual heartbeats and analyze the small variations in heartbeat intervals to determine their levels of arousal and positive affect.

These measurements are what allow EQ-Radio to detect emotion. For example, a person whose signals correlate to low arousal and negative affect is more likely to tagged as sad, while someone whose signals correlate to high arousal and positive affect would likely be tagged as excited.

The exact correlations vary from person to person, but are consistent enough that EQ-Radio could detect emotions with 70 percent accuracy even when it hadn’t previously measured the target person’s heartbeat.

“Just by knowing how people breathe and how their hearts beat in different emotional states, we can look at a random person’s heartbeat and reliably detect their emotions,” says Zhao.

For the experiments, subjects used videos or music to recall a series of memories that each evoked one the four emotions, as well as a no-emotion baseline. Trained just on those five sets of two-minute videos, EQ-Radio could then accurately classify the person’s behavior among the four emotions 87 percent of the time.

Compared with Microsoft’s vision-based “Emotion API”, which focuses on facial expressions,  EQ-Radio was found to be significantly more accurate in detecting joy, sadness, and anger. The two systems performed similarly with neutral emotions, since a face’s absence of emotion is generally easier to detect than its presence.

One of the CSAIL team’s toughest challenges was to tune out irrelevant data. In order to get individual heartbeats, for example, the team had to dampen the breathing, since the distance that a person’s chest moves from breathing is much greater than the distance that their heart moves to beat. 

To do so, the team focused on wireless signals that are based on acceleration rather than distance traveled, since the rise and fall of the chest with each breath tends to be much more consistent —  and, therefore, have a lower acceleration — than the motion of the heartbeat.

Although the focus on emotion-detection meant analyzing the time between heartbeats, the team says that the algorithm’s ability to captured the heartbeat’s entire waveform means that in the future it could be used for non-invasive health monitoring and diagnostic settings.

“By recovering measurements of the heart valves actually opening and closing at a millisecond time-scale, this system can literally detect if someone’s heart skips a beat,” says Adib. “This opens up the possibility of learning more about conditions like arrhythmia, and potentially exploring other medical applications that we haven’t even thought of yet.”

Press Mentions

Scientific American

In an article for Scientific American about the future of robotics, Prof. Emeritus Rodney Brooks highlights Prof. Dina Katabi’s work developing devices that use wireless signals to detect a person’s emotions. 

Scientific American

CSAIL researchers have a developed a system that can predict human emotions by using wireless signals to monitor breathing and heartbeats, writes Edd Gent in a Scientific American article. "The idea is that you can enable machines to recognize our emotions so they can interact with us at much deeper levels," says graduate student Fadel Adib.

Associated Press

A device created by CSAIL researchers can detect emotions by wirelessly measuring heartbeats, according to the Associated Press. The device is “87 percent accurate in using heartrate and what it’s already learned about a person to recognize joy, pleasure, sadness or anger.”

Boston Herald

CSAIL researchers have developed a device that can determine a person’s mood using wireless signals, write Jordan Graham and Donna Goodison for The Boston Herald. “We view this work as the next step in helping develop computers that can better understand us at an emotional level,” explains Mingmin Zhao.

Forbes

CSAIL researcher have created a device that uses changes in heart beat and breathing to detect emotions, writes Forbes’ Kevin Murnane. The heart of the system,” writes Murnane, “is the algorithm that extracts the heartbeat from the RF signal. It’s an impressive achievement that solves a difficult problem.”

Popular Science

CSAIL researchers have developed a device that can determine emotion by analyzing reflections from wireless signals bounced off the human body, writes Mary Beth Griggs for Popular Science. “Because it can measure heart rate, it might also be a less invasive way for doctors to monitor patient's heartbeats, potentially watching for conditions like arrhythmias,” writes Griggs. 

CNN

Matt McFarland writes for CNN that CSAIL researchers have developed a non-invasive device that uses radio waves to detect human emotion. "Imagine if the machines around you can understand when you are stressed or your emotional state is negative,” explains Prof. Dina Katabi."[It could] try to detect depression."  

Related Links

Related Topics

Related Articles

More MIT News

Video game rendition of MIT's McDermott Court with a building covered in icing and rainbow sprinkles

Scene at MIT: Mystery MIT

Held virtually for the first time due to the pandemic, this year’s MIT Mystery Hunt took the form of a puzzle-filled multiplayer game on an alternate-reality MIT campus.

Read full story