D. Fox Harrell, associate professor of digital media with appointments in the MIT Comparative Media Studies/Writing program and in the MIT Computer Science and Artificial intelligence Laboratory (CSAIL), has recently been awarded several grants to advance his research at the intersection of the social sciences and digital technology. These grants, funded by the National Science Foundation (NSF), the MIT CSAIL Qatar Computing Research Institute (QCRI) Alliance, and the MIT Center for Art, Science, and Technology (CAST), together amount to $1.35 million in support for Harrell’s groundbreaking interdisciplinary research.
Harrell’s new set of complementary initiatives builds upon his NSF CAREER Grant research project, “Computing for Advanced Identity Representation,” to delve more deeply into the dynamic relationship between virtual avatars and personal identities. He was able to push this work in innovative new directions while spending the 2014-15 year as a fellow at Stanford University in the Center for Advanced Study in the Behavioral Sciences (with the support of the Lenore Annenberg and Wallis Annenberg Fellowship in Communication).
“For five years I have been researching this intersection between human experiences and our identities as implemented across digital technologies such as video games and social media,” Harrell says. “These new projects focus specifically on how people’s personal identities and values are formed and changed by that technology.”
Virtual identities, real-world values
Whether choosing a type of character in a video game or selecting which photos and information to share on social media, Harrell says that human values are already embedded in the design of those technologies, while cultural norms influence the way people use them. Computational systems, he explains — whether in video games, social media, e-commerce accounts, or otherwise — “map elements of ourselves from the real world and our imaginations onto highly structured data.” In his work, Harrell explores what is gained, lost, and transformed in that process.
Online video games, for instance, let players create their own avatars, offering character options based on profession and race (i.e. thief, warrior, healer). Each type of player character comes with its own set of strengths and weaknesses. Some are stronger, others are faster or smarter, some have special powers, and so on.
Meanwhile, characteristics (including, but not limited to gender and race) are quite often employed to structure each player character's balance of strength, agility, speed, and other qualities, as well as their appearance.
“We can use AI machine-learning techniques to reveal the biases implemented in games by developers,” Harrell explains. “For example, my CSAIL PhD student Chong-U Lim and I have shown how player characters that are female and/or of certain races are statistically disadvantaged in one best-selling video game.”
“Our AI analyses not only reveal the assumptions that game developers build into systems,” says Harrell, “we can also discover biases of players. For example, we have revealed a phenomenon where players creating avatars of the opposite gender often exaggerate gender stereotypes.” He argues that while this is due, in part, to conventions in the games, it is also due to broader assumptions about identity that both developers and players share.
While much of Harrell’s research reveals these biases, he is also doing something to address them. Harrell has invented a platform called Chimeria, which allows games to support more diverse avatars and thus reflect the broad diversity of users with greater sensitivity. Using Chimeria, developers are able to implement characters whose identities impact the way other characters interact with them.
“These identities are dynamic,” says Harrell. “We can we track the way their behaviors change how they are perceived and how the player sees them. We are using virtual identities to impact how people imagine themselves and others.”
Research with a global scope
Harrell’s recent series of grants address the needs of both local and international communities. Locally, the NSF grant will support a series of computer science learning courses in public middle and high schools in Boston and Cambridge, Massachusetts. These courses use an educational video game developed in Harrell’s MIT Imagination, Computation, and Expression Laboratory (ICE Lab) that customizes student engagement. This cutting-edge curriculum sees programming as only one component of computer science. In addition, students must understand how computing impacts their lives and society at large. The game also doubles as a research environment, allowing Harrell and his team to look at the ways these virtual identities impact student learning.
One goal of the project is to explore avatars as a tool to support students from groups that are typically underrepresented in computer science, helping them learn key concepts, encouraging their excitment about the field, and allowing them to see themselves as learners and makers in the field of computer science. Harrell sees the game — and its customizable avatars — as an opportunity for students to see their identities as beneficial to STEM learning.
Meanwhile, as principle investigator on the Qatar Computing Research Institute-CSAIL grant, Harrell and his collaborator Sarah Vieweg at QCRI will examine the complex relationships between real versus virtual and online versus offline identities, focusing again on virtual representations. Despite the widespread use of social media and other digital technologies in the Middle East, everyday relationships to such technologies have not been extensively explored. Harrell’s unique approach uses AI in addition to qualitative social science methods to analyze these issues.
“These social media platforms weren’t designed with other cultures in mind,” Harrell says. “What we want to find out is how a versatile identity system can be used to spur value-driven design, and to design systems informed by awareness of tensions with a given culture’s values. These techniques can then be used to support values not only in areas such as Qatar, but also diverse communities in the U.S. and elsewhere.”
Also at the global scale, with support from the MIT Center for Art, Science and Technology, Harrell is collaborating with photojournalist Karim Ben Khelifa, director of the virtual reality project "The Enemy." By designing dynamic avatars for "The Enemy," Harrell will work with Ben Khelifa to gather user data to enable customized experiences with avatars that can foster empathy in the midst of global conflicts.
“I’m interested in the ways virtual identities can be used to encourage social change,” Harrell says. “Artful uses of technologies to imagine ourselves can be a powerful tool for social intervention.”
