Monster Technology — The ‘Frankencamera’

It’s big and it ain’t pretty, but it’s an experimentation in computation photography that went so right. It’s a new programmable camera platform called “Frankencamera.”

A Stanford University photography research group demonstrated sample applications of the open-source digital photography software platform on a F2 camera and the Nokia N900 smartphone at the SIGGRAPH 2010 conference in Los Angeles this week.

The programmable Frankencamera took this photo by synchronizing the light from a strobe and a flash. (Image: Marc Levoy)

This new platform allows users to create novel camera capabilities and is available as a free download for “mobile computers.”

"We're going public with Frankencamera," said Stanford computer science and electrical engineering professor Marc Levoy before the show. "We are releasing code so that people can create new imaging applications on their Nokia N900s."

In addition, the researchers have been awarded a $1 million grant from the National Science Foundation, shared with colleagues at MIT, to begin making professional-style, single-lens reflex (SLR) cameras, equipped with the software platform, for free distribution to computational photography professors around the country. Non-academics could buy the camera at cost. Levoy said he expects those cameras will be available within a year.

Program and shoot

Computational photography refers to the ways computers can extend the capabilities of digital imaging by combining multiple photographs taken with different camera settings to create an image that could not be taken in a single shot, or with an ordinary camera.

Some of these new ways of combining images can be done in Photoshop or another such program, but until now they could not be done inside the camera, Levoy said. That's because commercial cameras are closed to development by all but their manufacturers. Frankencamera, on the other hand, brings computational photography directly to the camera, by making the camera a programmable platform.

Frankencamera began in 2006 when Levoy and Kari Pulli, a Nokia Fellow who heads a research team at Nokia Research Center (NRC) Palo Alto, and a former research associate in Levoy's lab, reasoned that computational photography shouldn't be relegated to clunky research equipment in academic labs, as it has been for years. Instead it should be developed for use in the field on portable, consumer-friendly cameras.

"We thought it was time to make the research more nimble and get the results into smaller form-factors," said Pulli. "At NRC, we believe in an open innovation model that enables consumers and university researchers to use our research algorithms, and add their own, to create even more interesting capabilities."

Marc Levoy, professor of computer science and of electrical engineering, holds a prototype of the open source, programmable Frankencamera. (Image: L.A. Cicero)

At SIGGRAPH the researchers described how the Frankencamera platform exposes all the photographic and computational hardware on the camera – light sensors, flashes, focus, shutters and image processors – to a programmer's control through a software interface. The interface is part of a software "stack" that brings together familiar programming elements: a Linux operating system and the ubiquitous C++ programming language.

"The N900 is a camera phone, but it runs a version of Linux almost as complete as that installed on personal computers," Pulli said.

To help demonstrate and inspire development with Frankencamera, the team discussed six apps they’ve created on the platform. Using a prototype of their SLR version, for example, Levoy’s group hooked two flashes to the camera and programmed one to blink like a strobe light and the other to fire at the end of the strobe sequence. Then they flung some playing cards in the air. The result was a shot that captured the cards’ trajectories and also caught them still in mid-air with remarkable clarity.

Fine control of multiple flashes may yield captivating art, but sometimes photographers are caught with terrible lighting conditions that a flash can't solve. Confronted with inadequate light, a photographer with a conventional camera must choose either a quick exposure that will look sharp, but dark and noisy, or a long exposure that will have enough light, but will likely be blurry.

>