Augmented and virtual reality

Scientific American reporter Riis Williams explores how MIT researchers created “smart gloves” that have tactile sensors woven into the fabric to help teach piano and make other hands-on activities easier. “Hand-based movements like piano playing are normally really subjective and difficult to record and transfer,” explains graduate student Yiyue Luo. “But with these gloves we are actually able to track one person’s touch experience and share it with another person to improve their tactile learning process.”

AP reporter Ronald Blum spotlights the premiere of Prof. Jay Scheib’s augmented reality-infused production of Wagner’s “Parsifal” at the Bayreuth Festival in Germany. “We sort of focus on a future society in which myth has become possible again," says Scheib. "But at the same time, we’re not that far in the future and the third act is set around a broken lithium-ion field. We’re set in a world that is somehow post-planet and post-collapse of energy production.”

Popular Science reporter Andrew Paul spotlights “Ways of Seeing” a documentary project that aims to create “extended reality” (XR) experiences of significant architectural locales in Afghanistan as part of an effort to preserve the country’s historical sites. Paul notes that the project combines “cutting edge 3D imaging, drone photography, and virtual reality combined with painstakingly detailed hand drawings.”

Research from MIT and elsewhere have developed a mobile app that uses computer-vision techniques and AI to detect post-surgery signs of infection as part of an effort to help community workers in Kirehe, a district in Rwanda’s Eastern province, reports Shefali Malhotra for Science. “The researchers are now improving the app so it can be used across more diverse populations such as in Ghana and parts of South America,” writes Malhotra.

Kirin Sinha ‘14 founded Illumix, a technology company that combines 2D models to interactive 3D models to provide immersive AR experiences, reports Rebecca Suhrawardi for Forbes. “Illumix is enabling high-quality and real-time virtual try-on which has implications for the fashion industry ranging from higher conversion rates, fewer returns, and more environmentally-friendly,” says Sinha.

MIT researchers have created a new headset, called X-AR, that can help users find hidden or lost items by sending a wireless signal to any item that has a designated tag on it, reports WHDH. The augmented reality headset “allows them to see things that are otherwise not visible to the human eye,” explains Prof. Fadel Adib. “It visualizes items for people and then it guides them towards items.” 

Boston.com reporter Ross Cristantiello writes that MIT researchers have developed a new augmented reality headset that combines computer vision and wireless perception to allow users to track and find objects hidden from view. “The system relies on radio frequency signals that can pass through everyday materials like cardboard, plastic, and wood,” Cristantiello explains.

MIT engineers have developed an augmented reality headset that uses RFID technology to allow wearers to find objects, reports Tony Ho Tran for The Daily Beast. “The device is intended to assist workers in places like e-commerce warehouses and retail stores to quickly find and identify objects,” writes Tran. “It can also help technicians find tools and items they need to assemble products.” 

An augmented reality headset developed by MIT engineers, called X-AR, uses RFID technology to help users find hidden objects, reports Andrew Paul for Popular Science. “X-AR’s creators were able to guide users with nearly 99 percent accuracy to items scattered throughout a warehouse testing environment,” writes Paul. “When those products were hidden within boxes, the X-AR still even boasted an almost 92 percent accuracy rate.” 

MIT researchers devised a new way to arrange LED pixels to create screens with a much higher resolution than is currently possible, reports The Economist. The new technique, which involves stacking micro LEDS, could also be used to make “VR images that appear far more lifelike than today’s.”

Prof. Cynthia Breazeal discusses her work exploring how artificial intelligence can help students impacted by Covid, including refugees or children with disabilities, reports Ryan Heath for Politico. “We want to be super clear on what the role is of the robot versus the community, of which this robot is a part of. That's part of the ethical design thinking,” says Breazeal. “We don't want to have the robot overstep its responsibilities. All of our data that we collect is protected and encrypted.”

Mashable spotlights how MIT’s baseball pitching coach is using motion capture technology to help analyze and teach pitching techniques. Using the technology, Coach Todd Carroll can “suggest real-time adjustments as a player is pitching so that just one session using the technology improves their game.”

MIT researchers have created 3D models of spiderwebs to help transform the web’s vibrations into sounds that humans can hear, writes Angela Moore for Reuters. “Spiders utilize vibrations as a way to communicate with the environment, with other spiders,” says Prof. Markus Buehler. “We have recorded these vibrations from spiders and used artificial intelligence to learn these vibrational patterns and associate them with certain actions, basically learning the spider’s language.” 

In a new data sonification project, a team of MIT researchers have translated the vibrations of a spider’s web into music, writes Maddie Bender for Motherboard. The team "used the physics of spiderwebs to assign audible tones to a given string’s unique tension and vibration," writes Bender. "Summing up every string’s tone created an interactive model of a web that could produce sound through manipulation or VR navigation."

A team of MIT researchers have translated the vibrations of a spider’s web into music, reports Isaac Schultz for Gizmodo. “Spiders live in this vibrational universe,” says Prof. Markus Buehler. “They live in this world of vibrations and frequencies, which we can now access. One of the things we can do with this instrument with this approach is we can, for the first time, begin to feel a little bit like a spider or experience the world like the spider does.”