Skip to content ↓

MIT grad directs Spielberg in the science of moviemaking

John Underkoffler (center) and fellow Media Lab graduate Alan Lasky (left) make some behind-the-scenes adjustments on an earlier film project. Underkoffler is now the science and technology advisor on Ang Lee's movie adaptation of "The Incredible Hulk."
John Underkoffler (center) and fellow Media Lab graduate Alan Lasky (left) make some behind-the-scenes adjustments on an earlier film project. Underkoffler is now the science and technology advisor on Ang Lee's movie adaptation of "The Incredible Hulk."
Photo courtesy / Rob Poor

Making predictions has always been a tricky job.

Creators of early science fiction could take refuge in the gap between what was conceivable and what was feasible. The futurists of today don't have that luxury: daily technological advances have sent the shelf life of science fiction into decline.

That's why smart sci-fi makers look for inspiration in the very places responsible for those advances. And that's why MIT played a formative role in the movie "Minority Report," now in theaters.

Long before the film went into production, director Steven Spielberg convened a three-day conference about what life will be like in the year 2054. He invited writers, urban planners, technological innovators and various other prognosticators, including several from MIT. Later, members of his crew came to the MIT Media Laboratory in search of strange new technologies that might be used in the movie.

But the most direct input came from John Underkoffler (S.B. 1988, S.M., Ph.D.), who was the science and technology advisor for the film. As an acknowledged "film freak" with more than a decade of experience at the Media Lab working on everything from holograms to computer interfaces, it was an opportunity to apply his knowledge in a new way.

"The science and technology advisor role is a new, new thing in the film industry," said Underkoffler, who joined the production after he met crew members at the Media Lab. "Most people, myself included, are surprised to find out how broad the job is."

Underkoffler, who received his Ph.D. in media arts and sciences in 1999, was involved in mapping out "future history" during script revisions--trying to determine how political and social forces will shape technological innovation. He also acted as a resource to the film's visual artists as they created props, sets and costumes. And best of all, he was on the set during filming to provide on-the-spot technical advice and coach actors in the finer points of futuristic living.


Set in Washington, D.C., in 2054, "Minority Report" depicts a time when murder can be forseen and prevented with the help of a small group of computer-assisted psychics known as "pre-cogs." Tom Cruise plays Jon Anderton, chief of the Pre-Crime Unit and absolute believer in the system, until he is fingered as the next killer. The film resembles a futuristic version of "The Fugitive," but with philosophical undercurrents on the relative value of public safety versus personal freedom.

To guide his suggestions for the movie, Underkoffler referred to both the Philip K. Dick story on which it is based and a transcript of the brainstorming conference.

The Dick story served primarily as a foundation. Most of the smaller technological details of the story, such as computers that rely on paper punch cards, had to be reimagined.

"The story is really a sketch; the characters aren't fleshed out," Underkoffler said. Noting that Dick was paid by the word, Underkoffler said, "he had alimony to pay and it shows. That said, I think he had more great ideas than any other science fiction writer."

The 15 attendees at the brainstorming conference built on these ideas in an attempt to craft a future world that was a natural extension of the present. Douglas Coupland, author of "Generation X," was there, as was virtual-reality pioneer Jaron Lanier and urban planner Peter Calthorpe.

The MIT contingent included Neil Gershenfeld, director of the Center for Bits and Atoms at the Media Lab; Stewart Brand, a former visiting researcher at the Media Lab; and William Mitchell, dean of the School of Architecture and Planning and professor of media arts and sciences.

The group understandably didn't reach consensus, but the discussion did yield many insights that made their way onto the screen. Underkoffler characterized the meeting as a "collision of futurist ideas," and he detected an underlying utopian tone in much of what was said. Held during the high point of the Internet economy in 1999, the discussion reflected its time, but some ideas were later amended to reflect changing circumstances.

"When I compare the transcript with the finished movie, I think Steven came out with something much grayer and more ambiguous than what was originally there," Underkoffler said.


One of the most dazzling technologies in "Minority Report" is the computer system that Cruise's character uses to analyze "pre-crime" scenes. The challenge was to invent an interface that allowed the user to shift rapidly through hundreds of images from the minds of the pre-cogs, occasionally pausing to zoom in on potential clues.

"From the beginning, Steven said, 'Forget the keyboard and mouse and forget voice interaction, because we've been seeing that in science fiction for at least 30 years and everyone is used to it,'" Underkoffler said.

Underkoffler set out to create a 'gestural' method of input, whereby specific hand movements would represent different commands. In doing so, he built on research that has been taking place at the Media Lab for years, including some of his own.

As a member of the lab's Tangible Media Group, Underkoffler worked on a project dubbed "The Luminous Room." An exercise in immersive computing, the idea was to incorporate displays into the architecture of a room and to allow those displays not to only convey information to the user, but to collect it from them as well. This concept is similar, in spirit if not in detail, to how computing works in the movie.

When it came time to shoot, Spielberg would first describe to Underkoffler how he wanted Anderton to interact with the computer. This involved a great deal of imagination on everyone's part, because the images that appear on the giant glass display in the final film were not added until post-production. Underkoffler would next translate the commands into his gestural language and quickly teach the movements to Cruise. Then the cameras rolled.


Underkoffler hasn't had to forsake his roots as a researcher to get to Hollywood. In fact, he sees his movie work as a natural extension of what he did while at MIT.

"In the '30s and '40s, most science fiction just featured caricatures of scientists and there was very little real scientific information there. Now a lot of science from the movies flows back into real research. The 'cyber' research of the last 10 years is very much influenced by the cyberpunk literature of the 10 years before that," he said.

The gestural interface he created for "Minority Report" may look like a cross between orchestral conducting and tae kwon do, but it also looks like an efficient and powerful way to control a computer that may induce researchers to seek a real-world version.

So many of the projects underway at the Media Lab seem like science fiction anyway that the biggest difference between the lab and soundstage may be how the business is conducted. "MIT has this very strong, very entrenched culture and Hollywood has an equally strong, equally entrenched culture. So it makes for an interesting cultural interface," Underkoffler said.

Underkoffler seems to have taken the transition in stride: he's now working as the science and technology advisor on Ang Lee's film "The Incredible Hulk," due in summer 2003.

After 16 years at MIT, Underkoffler calls the change of pace "a necessary nutrient. Since it is--or ought to be--the job of everyone at MIT not only to develop new ideas but also to send them out into the world, it's a worthy exercise to consider what alternate venues there may be for 'publication,'" he said. "Cinema's certainly an effective one, under the right conditions."

A version of this article appeared in MIT Tech Talk on July 17, 2002.

Related Topics

More MIT News