Skip to content ↓

Judging a book through its cover

New computational imaging method identifies letters printed on first nine pages of a stack of paper.
Watch Video
Press Inquiries

Press Contact:

Abby Abazorius
Phone: 617-253-2709
MIT News Office

Media Download

A new imaging system can read closed books. “The Metropolitan Museum in New York showed a lot of interest in this, because they want to, for example, look into some antique books that they don’t even want to touch,” says Barmak Heshmat, a research scientist at the MIT Media Lab.
Download Image
Caption: A new imaging system can read closed books. “The Metropolitan Museum in New York showed a lot of interest in this, because they want to, for example, look into some antique books that they don’t even want to touch,” says Barmak Heshmat, a research scientist at the MIT Media Lab.
Credits: Courtesy of Barmak Heshmat
At the moment, the algorithm can correctly deduce the distance from the camera to the top 20 pages in a stack, but past a depth of nine pages, the energy of the reflected signal is so low that the differences between frequency signatures are swamped by noise.
Download Image
Caption: At the moment, the algorithm can correctly deduce the distance from the camera to the top 20 pages in a stack, but past a depth of nine pages, the energy of the reflected signal is so low that the differences between frequency signatures are swamped by noise.
Credits: Courtesy of Barmak Heshmat

*Terms of Use:

Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license. You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT."

Close
A new imaging system can read closed books. “The Metropolitan Museum in New York showed a lot of interest in this, because they want to, for example, look into some antique books that they don’t even want to touch,” says Barmak Heshmat, a research scientist at the MIT Media Lab.
Caption:
A new imaging system can read closed books. “The Metropolitan Museum in New York showed a lot of interest in this, because they want to, for example, look into some antique books that they don’t even want to touch,” says Barmak Heshmat, a research scientist at the MIT Media Lab.
Credits:
Photo courtesy of Barmak Heshmat.
At the moment, the algorithm can correctly deduce the distance from the camera to the top 20 pages in a stack, but past a depth of nine pages, the energy of the reflected signal is so low that the differences between frequency signatures are swamped by noise.
Caption:
At the moment, the algorithm can correctly deduce the distance from the camera to the top 20 pages in a stack, but past a depth of nine pages, the energy of the reflected signal is so low that the differences between frequency signatures are swamped by noise.
Credits:
Image courtesy of Barmak Heshmat.

MIT researchers and their colleagues are designing an imaging system that can read closed books.

In the latest issue of Nature Communications, the researchers describe a prototype of the system, which they tested on a stack of papers, each with one letter printed on it. The system was able to correctly identify the letters on the top nine sheets.

“The Metropolitan Museum in New York showed a lot of interest in this, because they want to, for example, look into some antique books that they don’t even want to touch,” says Barmak Heshmat, a research scientist at the MIT Media Lab and corresponding author on the new paper. He adds that the system could be used to analyze any materials organized in thin layers, such as coatings on machine parts or pharmaceuticals.

Video thumbnail Play video
Spatial resolution, spectral contrast, and occlusion are three major bottlenecks in current imaging technologies for non-invasive inspection of complex samples such as closed books. A new study reports successful unsupervised content extraction through a densely layered structure similar to that of a closed book.

Heshmat is joined on the paper by Ramesh Raskar, an associate professor of media arts and sciences; Albert Redo Sanchez, a research specialist in the Camera Culture group at the Media Lab; two of the group’s other members; and by Justin Romberg and Alireza Aghasi of Georgia Tech.

The MIT researchers developed the algorithms that acquire images from individual sheets in stacks of paper, and the Georgia Tech researchers developed the algorithm that interprets the often distorted or incomplete images as individual letters. “It’s actually kind of scary,” Heshmat says of the letter-interpretation algorithm. “A lot of websites have these letter certifications [captchas] to make sure you’re not a robot, and this algorithm can get through a lot of them.”

Timing terahertz

The system uses terahertz radiation, the band of electromagnetic radiation between microwaves and infrared light, which has several advantages over other types of waves that can penetrate surfaces, such as X-rays or sound waves. Terahertz radiation has been widely researched for use in security screening, because different chemicals absorb different frequencies of terahertz radiation to different degrees, yielding a distinctive frequency signature for each. By the same token, terahertz frequency profiles can distinguish between ink and blank paper, in a way that X-rays can’t.

Terahertz radiation can also be emitted in such short bursts that the distance it has traveled can be gauged from the difference between its emission time and the time at which reflected radiation returns to a sensor. That gives it much better depth resolution than ultrasound.

The system exploits the fact that trapped between the pages of a book are tiny air pockets only about 20 micrometers deep. The difference in refractive index — the degree to which they bend light — between the air and the paper means that the boundary between the two will reflect terahertz radiation back to a detector.

In the researchers’ setup, a standard terahertz camera emits ultrashort bursts of radiation, and the camera’s built-in sensor detects their reflections. From the reflections’ time of arrival, the MIT researchers’ algorithm can gauge the distance to the individual pages of the book.

True signals

While most of the radiation is either absorbed or reflected by the book, some of it bounces around between pages before returning to the sensor, producing a spurious signal. The sensor’s electronics also produce a background hum. One of the tasks of the MIT researchers’ algorithm is to filter out all this “noise.”

The information about the pages’ distance helps: It allows the algorithm to hone in on just the terahertz signals whose arrival times suggest that they are true reflections. Then, it relies on two different measures of the reflections’ energy and assumptions about both the energy profiles of true reflections and the statistics of noise to extract information about the chemical properties of the reflecting surfaces.

At the moment, the algorithm can correctly deduce the distance from the camera to the top 20 pages in a stack, but past a depth of nine pages, the energy of the reflected signal is so low that the differences between frequency signatures are swamped by noise. Terahertz imaging is still a relatively young technology, however, and researchers are constantly working to improve both the accuracy of detectors and the power of the radiation sources, so deeper penetration should be possible.

"So much work has gone into terahertz technology to get the sources and detectors working, with big promises for imaging new and exciting things,” says Laura Waller, an associate professor of electrical engineering and computer science at the University of California at Berkeley. “This work is one of the first to use these new tools along with advances in computational imaging to get at pictures of things we could never see with optical technologies. Now we can judge a book through its cover!"

Press Mentions

The Wall Street Journal

Daniel Akst of The Wall Street Journal writes that by bouncing electromagnetic waves off of pages, MIT researchers have developed a way to read closed books. The system could potentially be used also be used to count stacks of money and detect counterfeit currency, Akst explains. 

PBS NewsHour

A new device developed by MIT researchers can read the pages of a book without opening the cover, reports Nsikan Akpan for the PBS NewsHour. The tool may “unlock the secrets of old books or ancient texts too fragile to be disturbed by human touch.”

Associated Press

MIT researchers have developed a system that uses terahertz waves to read the pages of a closed book, reports Michael Casey for the AP. Research scientist Barmak Heshmat explains that the system works better than X-rays, as “it can contrast between the blank paper and the part that has ink.”

Popular Science

MIT researchers have developed a new computational imaging technique that can read closed books, reports G. Clay Whittaker for Popular Science. The technique could be useful for “rare book research, where opening a book may be impossible due to damage, or not worth the risk of damage.”

CBS News

Writing for CBS News, Charles Choi explores a system developed by MIT researchers that can identify letters in a closed book. The system could be used to examine ancient books or to “scan through large amounts of documents without having to mechanically separate the pages, which could be useful for libraries, banks and others,” says research scientist Barmak Heshmat. 

Salon

Salon reporter Scott Eric Kaufman writes that MIT researchers have created a system that can read the pages of a closed book and could be used to examine manuscripts too fragile to handle. “The system works by shooting pulses of radiation from a terahertz camera and measuring how long it takes for them to bounce back." 

Related Links

Related Topics

Related Articles

More MIT News