Search
Menu
Cognex Corp. - Smart Sensor 3-24 GIF LB

Monster Technology — The ‘Frankencamera’

Facebook X LinkedIn Email
It’s big and it ain’t pretty, but it’s an experimentation in computation photography that went so right. It’s a new programmable camera platform called “Frankencamera.”

A Stanford University photography research group demonstrated sample applications of the open-source digital photography software platform on a F2 camera and the Nokia N900 smartphone at the SIGGRAPH 2010 conference in Los Angeles this week.


The programmable Frankencamera took this photo by synchronizing the light from a strobe and a flash. (Image: Marc Levoy)

This new platform allows users to create novel camera capabilities and is available as a free download for “mobile computers.”

"We're going public with Frankencamera," said Stanford computer science and electrical engineering professor Marc Levoy before the show. "We are releasing code so that people can create new imaging applications on their Nokia N900s."

In addition, the researchers have been awarded a $1 million grant from the National Science Foundation, shared with colleagues at MIT, to begin making professional-style, single-lens reflex (SLR) cameras, equipped with the software platform, for free distribution to computational photography professors around the country. Non-academics could buy the camera at cost. Levoy said he expects those cameras will be available within a year.

Program and shoot

Computational photography refers to the ways computers can extend the capabilities of digital imaging by combining multiple photographs taken with different camera settings to create an image that could not be taken in a single shot, or with an ordinary camera.

Some of these new ways of combining images can be done in Photoshop or another such program, but until now they could not be done inside the camera, Levoy said. That's because commercial cameras are closed to development by all but their manufacturers. Frankencamera, on the other hand, brings computational photography directly to the camera, by making the camera a programmable platform.

Frankencamera began in 2006 when Levoy and Kari Pulli, a Nokia Fellow who heads a research team at Nokia Research Center (NRC) Palo Alto, and a former research associate in Levoy's lab, reasoned that computational photography shouldn't be relegated to clunky research equipment in academic labs, as it has been for years. Instead it should be developed for use in the field on portable, consumer-friendly cameras.

"We thought it was time to make the research more nimble and get the results into smaller form-factors," said Pulli. "At NRC, we believe in an open innovation model that enables consumers and university researchers to use our research algorithms, and add their own, to create even more interesting capabilities."


Marc Levoy, professor of computer science and of electrical engineering, holds a prototype of the open source, programmable Frankencamera. (Image: L.A. Cicero)

At SIGGRAPH the researchers described how the Frankencamera platform exposes all the photographic and computational hardware on the camera – light sensors, flashes, focus, shutters and image processors – to a programmer's control through a software interface. The interface is part of a software "stack" that brings together familiar programming elements: a Linux operating system and the ubiquitous C++ programming language.

Meadowlark Optics - Building system MR 7/23

"The N900 is a camera phone, but it runs a version of Linux almost as complete as that installed on personal computers," Pulli said. 

To help demonstrate and inspire development with Frankencamera, the team discussed six apps they’ve created on the platform. Using a prototype of their SLR version, for example, Levoy’s group hooked two flashes to the camera and programmed one to blink like a strobe light and the other to fire at the end of the strobe sequence. Then they flung some playing cards in the air. The result was a shot that captured the cards’ trajectories and also caught them still in mid-air with remarkable clarity.

Fine control of multiple flashes may yield captivating art, but sometimes photographers are caught with terrible lighting conditions that a flash can't solve. Confronted with inadequate light, a photographer with a conventional camera must choose either a quick exposure that will look sharp, but dark and noisy, or a long exposure that will have enough light, but will likely be blurry.

 
Eddy Talvala holds the Frankencamera, programmed to individual needs of the photographer. It is part of an attempt by a team of Stanford researchers to build an open-source camera architecture. The group has just released a version of their software for Nokia smartphones.

An app called "low-light imaging" shoots both exposure speeds in rapid succession and then automatically combines them, resulting in a photo that is both bright and sharp.

Another app offers a different take on the low-light problem. Dubbed "lucky imaging," the app takes advantage of a motion sensor attached to the Nokia N900 to prevent shaky hands from adding blur to a long exposure. In this mode, the camera shoots constantly but only stores images taken when the camera is at its most still. The result is the sharpest possible image.

Levoy said he hopes the entire computational photography community will step forward with other ideas and that they will share them with each other online. The apps will be found through the Frankencamera project website.

For Levoy, a key question is whether commercial manufacturers – both of dedicated cameras and camera smart phones – will also open up their platforms. Since the Frankencamera effort was first publicized last fall, Levoy said, he has had a steady stream of meetings, especially with smart phone makers.

"What we're hoping is that if making your camera programmable adds value to the camera that this could shift the entire camera industry," Levoy said. "We'd like to democratize cameras the way that the Maker Faire is democratizing crafts so that anyone can play the game."

In addition to Nokia Research Center, funding comes from Adobe Systems, Kodak, Hewlett-Packard and the Walt Disney Company. In addition to authors at Stanford and NRC, other authors on the SIGGRAPH paper are affiliated with Ulm University in Germany, the University of California-Santa Barbara and Disney Research in Zurich, Switzerland.

David Orenstein is the associate communications director for the School of Engineering.

For more information, visit:  www.stanford.edu 




Published: July 2010
Glossary
focus
1. The focal point. 2. To adjust the eyepiece or objective of a telescope so that the image is clearly seen by the observer. 3. To adjust the camera lens, plate, or film holder so that the image is rendered distinct. 4. To move an entire microscope body tube relative to a specimen to obtain the sharpest possible image.
Adobe SystemsAmericasC++ programming languageCaliforniacamerasCommunicationscomputational photographyConsumerDisney ResearchEuropeF2 cameraflashesfocusFrankencameraGermanyHewlett-Packardimage processorsImagingimaging applicationsKari PulliKodaklight sensorsLight SourcesLinuxlucky imagingMarc LevoyMassachusettsMITmobile computersNational Science FoundationNokia N900open-source digital photography software platformPhotoshopprogrammable camera platformResearch & TechnologySensors & DetectorsshuttersSIGGRAPH 2010SLR camerassmartphoneStanford UniversitySwitzerlandterrible lighting conditionsUlm UniversityUniversity of California-Santa BarbaraWalt Disney Company

We use cookies to improve user experience and analyze our website traffic as stated in our Privacy Policy. By using this website, you agree to the use of cookies unless you have disabled them.