Inside the minds of voters

Any analysis of exit polling reveals a welter of numbers whose meaning remains slightly elusive, with issues or candidate characteristics described as “very important,” “somewhat important,” or “not important at all” by voters. But it is not always clear how these findings fit together.

Now, a new paper co-written by an MIT political scientist suggests a way to assess the relative impact of several factors at once, using a method known as “conjoint analysis” that is not currently employed in political polling.

The method behind conjoint analysis is fairly simple: Respondents in public opinion surveys are given hypothetical matchups between two candidates whose characteristics — say, religion, wealth, ethnic background — are randomly altered in the survey. Given a representative sample of voters making choices based on these hypothetical matchups, it is possible to determine the relative weight the electorate gives to any of these candidate characteristics.

Does religion matter more than candidate wealth? Conjoint analysis can provide a direct comparison. Moreover, because most voters weigh several factors at once when making choices at the ballot box, conjoint analysis can reveal the relative weight of many factors at once.

“Researchers are good at examining the effects of single attributes of a candidate on the voting public,” says Teppei Yamamoto, an assistant professor of political science at MIT. “But people actually usually use different dimensions of a candidate when they decide how to vote. We thought this would be an ideal approach for political science, to find out which aspects of political candidates are important to people.”

The paper, “Causal Inference in Conjoint Analysis,” is published in the latest issue of Political Analysis, a journal that focuses on political science methodology. In addition to Yamamoto, the authors are Jens Hainmueller of Stanford University and Daniel Hopkins of Georgetown University.

Choosing a candidate, not a car

The concept of conjoint analysis is not novel, but until now, the method has been applied primarily to marketing, not politics. Automakers, for instance, will use focus groups to determine the relative significance of a vehicle’s many features, which can help direct the design process. However, there is at least one major difference between this kind of market research and political polling: Consumers are less likely than voters to link together various characteristics when making choices.

“In designing a car, it’s less of a concern that different aspects of the car will interact with each other in consumer decisions,” Yamamoto explains. “People’s preferences about car color are almost independent of whether or not the car has a manual transmission. But in political decisions, or social decisions in general, those aspects can interact with each other.” For example, he suggests, voters might tend to reflexively link candidate characteristics, such as competence, with a candidate’s political party.

Yamamoto’s paper lays out a precise way in which conjoint analysis could be translated to politics. For one thing, the successful use of conjoint analysis in politics, he thinks, depends on the randomization of the characteristics presented to survey respondents, as a way of decoupling the connections voters tend to make between certain characteristics. If that is done right, Yamamoto believes, the method would prove useful not only to pollsters, but to campaigns and consultants.

“I can imagine a political consultant wanting to do a sample of constituents and then advising their politician clients that [voters] seem to value certain things a lot, so why not design a campaign emphasizing that,” Yamamoto says.

Are voters being honest?

Others think the paper contains significant methodological insight. Kenneth Scheve, a political scientist at Stanford University, calls the work “a terrific paper” that makes the methodology “useful for modern public opinion scholarship,” in part by describing how conjoint analysis can be adapted to political polling. 

“I expect their approach to be widely applied to any problems of multidimensional choice, certainly those prevalent in politics but really for any choices that individuals make when the object of choice has a number of different potentially relevant features,” Scheve says.

In the paper, the authors address potential limitations of the method. Pollsters and researchers may be interested in voter preferences that are not easily expressed in terms of rankings, for instance. A more common concern about conjoint analysis is that it relies too heavily on the stated preferences of respondents, which would be problematic if voters were unwilling to provide candid answers about subjects such as ethnicity.

To examine this concern, Yamamoto and Hainmueller are conducting studies of how closely such expressed preferences match the actions of voters. Currently, they are studying the stated views of Swiss citizens to see if they match election results. Such studies could help reveal the relationship between what voters tell pollsters and what voters really think.