It’s another case of a class project that turned into a bit more than the typical assignment.
The story began last fall in the MIT course 6.810 (Engineering Interactive Technologies) taught by Stefanie Mueller, an associate professor in MIT’s Department of Electrical Engineering and Computer Science. The students, who were mostly undergraduates, were asked to do a final group project, and they were assisted in this effort by graduate students who were helping Mueller teach the course.
Now, the product of this effort, along with an accompanying paper, will be presented in August in Vancouver, Canada, at SIGGRAPH 2022, “the world’s largest, most influential annual conference in computer graphics and interactive techniques.”
Ticha Sethapakdi, a third-year PhD student based at MIT’s Computer Science and Artificial Intelligence Laboratory, thought hard about a task that could be accomplished in a couple of months and would also enable students to utilize the skills they’d learned in class. Perhaps they could develop an instant camera, she thought, but instead of following the Polaroid example, it might be fun to make a camera that produced moving images. A cheap and portable device like this, Sethapakdi mused, could be a great icebreaker in social situations, and might, more importantly, open the door to other applications that were not immediately obvious to members of their research team.
To keep within the modest budgets allotted to them, Sethapakdi recognized that they’d have to adopt an easy, low-tech approach that would not require costly and sophisticated equipment. She hit upon a technique called kinegrams — which were invented in the late-1800s, just before the introduction of the first animated movies — and she decided this might be a good way to go.
Here, in short, is how a kinegram works. First, you start with several images or frames — let’s say three — of the same object, such as a butterfly, in different poses. You then cut each image into horizontal strips, perhaps 10 in all, of a uniform width. Next, you take these three images and combine them onto a page to create a single “interlaced image” consisting of 30 horizontal strips altogether — starting with the top strips from images one, two, and three and continuing to alternate in that fashion until the last three strips are sequentially laid out. Next you take a “striped overlay,” a separate sheet that has transparent strips — again, 10 in all — of the same width as the strips on the interlaced images. Each transparent strip, in turn, is interspersed with 10 opaque strips that are twice as wide as the transparent ones.
If you then lined up the two sheets, placed the striped overlay on top of the interlaced image, and moved the overlay down quickly, you would see the butterfly moving between its different poses. It’s a rather crude form of animation, though somewhat more complex than the flipbook cartoons that grade-schoolers can make.
The challenge, however, was to find a way of producing kinegrams quickly and easily with a lightweight, portable device. And that’s how the KineCAM came into being. All its components, save for the batteries, fit into a box of dimensions 3.5” x 5.6” x 1.7”. The pieces include a small computer called a Raspberry Pi, an attached video camera (comparable to those found in cellphones), a thermal printer (which is similar to that used to print receipts), a camera shutter button, and an LED indicator light — all of which can be bought, “off the shelf,” for less than $100 altogether.
As Sethapakdi and her coauthors — Mueller, postdoc Mackenzie Leake, and two undergraduates, Catalina Monsalve Rodriguez and Miranda J. Cai — explain in their paper: “to create the kinegram, our system records the video for a fixed unit of time and selects n frames from it. It then decomposes the frames into strips of width w pixels, interlaces them, and composites them into a single image.”
“We have to program the system ourselves,” Sethapakdi notes. “The software takes a few frames of videos, slices up the images, and sends the sliced-up version to the printer.” The turnaround time is remarkably brief, taking only about 16 seconds to go from shooting the pictures to printing. Everything is done with the KineCAM itself, except for creating the striped overlays, which are printed in advance on transparent film using a conventional inkjet printer.
One advantage of the approach, says Sethapakdi, “is that you can go out into the world and, on the spot, make these animated photographs.” One can, of course, take pictures with a cellphone, she adds. “But there’s something very appealing and intimate about having an actual physical receipt — a one-of-a-kind copy of some experience.” And a moving picture might be able to capture aspects of that experience that a still photo cannot.
The MIT team believes the KineCAM has applications in computer graphics and “rotoscoping,” a traditional animation technique. Sethapakdi considers it likely that artists and researchers will come up with some interesting ideas of their own. “We are open-sourcing this project so that others can modify the camera and adapt the code and design it to do anything they want it to do. We’re happy to let others explore the possibilities.”
She’ll soon be speaking to a target audience at SIGGRAPH 2022. Someone attending that conference may well have plans for KineCAM that had not yet occurred to its inventors — and that’s a prospect Sethapakdi is actively looking forward to.
