This year at PhotoCamp, I gave a short overview of the concept of computational photography, how it stands to impact digital photography in the years to come. Along with my talk, Andrew Ferguson discussed the ins and outs of blogging about photography, and Duane Storey gave one of the best non-technical overviews of HDR imaging I've heard. Kris Krug moderated, and I think a good time was had by all.
The story goes something like this: Due to the complexity of darkroom techniques, and the limitations in what kind of image manipulations we can perform optically while exposing the print from the negative, we have come to view the light that falls on the piece of film in the camera (or the sensor) as the final image. In the old days, it was either impractical or impossible to perform much alterations to the image, so it wasn't attempted.
Digital photography requires computers. No matter how hard you rub the CF card on your monitor or printer, you'll never get an image from your camera to appear. For all intents and purposes, your computer is a giant brain capable of applying a vast number of image manipulations photographs.
There is all this computation available, and the most that people can think to do to their photographs after they are taken is to adjust the white balance.
It's a very interesting area of work, and a large component of my PhD research. I can't wait to see and share more about it in the future.
Posted on August 18, 2007
Tags:
barcamp,
compphoto,
photocamp,
photography,
vancouver
Here are my notes from my PhotoCamp presentation I gave at BarCamp Vancouver over the weekend.
Idea
While we have used digital cameras much the same way that we have film cameras, digital photography has fundamental differences from conventional film-based devices. The ability to interact with sensors has opened up a number of opportunities for the capture of novel image types. The combination of different methods of capture with new processing techniques allows for new image forms. The umbrella term for this family of techniques is known in the research community as computational photography.
There is ongoing research in changing all the major clusters of the photographic process. People have investigated changes in lighting, camera optics, digital sensors, and image procesing. The point of the talk was to provide an overview of what interesting features might be available on your camera in the future.
Fourier slice photography
This paper presents a camera that samples the 4D light field on its sensor in a single photographic exposure. This is achieved by inserting a microlens array between the sensor and main lens, creating a plenoptic camera. Each microlens measures not just the total amount of light deposited at that location, but how much light arrives along each ray. By re-sorting the measured rays of light to where they would have terminated in slightly different, synthetic cameras, we can compute sharp photographs focused at different depths. This property allows us to extend the depth of field of the camera without reducing the aperture, enabling shorter exposures and lower image noise. Especially in the macrophotography regime, we demonstrate that we can also compute synthetic photographs from a range of different viewpoints.
Flash-no flash photography
This technique enhances the appearance of photographs shot in dark environments by combining a picture taken with the available light and one taken with the flash. It preserves the ambiance of the original lighting and insert the sharpness and more reliable color information from the flash image. It uses the bilateral filter to decompose the two images into detail and large-scale layers. It reconstructs the image using the large scale of the available lighting and the detail of the flash. We detect and correct artifacts due to the flash shadow. The output images provide the combined advantages of available illumination and flash photography.
Graph processing
This framework uses graph-cut optimization to choose good seams within the constituent images so that they can be combined as seamlessly as possible; and gradient-domain fusion to further reduce any remaining visible artifacts in the composite. The power of this framework lies in its generality; we show how it can be used for a wide variety of applications, including "selective composites" (for instance, group photos in which everyone looks their best), relighting, extended depth of field, panoramic stitching, clean-plate production, stroboscopic visualization of movement, and time-lapse mosaics.
Structured photography
Photo tourism is a system for browsing large collections of photographs in 3D. Our approach takes as input large collections of images from either personal photo collections or Internet photo sharing sites, and automatically computes each photo's viewpoint and a sparse 3D model of the scene. Our photo explorer interface enables the viewer to interactively move about the 3D space by seamlessly transitioning between photographs, based on user control.
High dynamic range imaging
One of the most interesting advances in photography and imaging is what is known as high dynamic range imaging (HDR or HDRI). In the context of photography, the purpose is to extend the dynamic range (or ratio of brightest to darkest areas) beyond it's current limitations. The goal is to capture all of the luminance data for later, and not have images that have areas that are overexposed or underexposed. For more information on HDRI and how it applies to photography, check out this article.
The process of HDR imaging follows the same basic flow as conventional photography. You capture the scene via some method, store and process it, then display it by some means, but all of the methods for HDR images differ from their conventional counterparts. On the acquisition end, there are several means of creating HDR images. For static scenes, an exposure sequence can be combined into a single HDR image, while new HDR cameras are also under development. On the display end, there are two categories, tonemapping and native display. Tonemapping is method of compressing the image contrast to the dynamic range of a conventional display. There is much more to tonemapping than the Flickr group and some operators work better than others, and you need to know what you want to do in order to pick the right one. The other option is to use a high dynamic range display, such as the ones by BrightSide Technologies (warning: shameless plug).
Posted on August 27, 2006
Tags:
barcamp,
compphoto,
hdr,
photocamp,
photography,
vancouver
I'm now sitting in my friends' place in Jersey City, somehow oddly awake after an exhausting series of days. I've been in Boston, attending an amazing symposium on "Computational Photography and Video":http://photo.csail.mit.edu/ hosted by MIT. It brought together the top 200 researchers in computer graphics, computer vision, image processing, and some cutting edge photographers to discuss where this intersection of computers and cameras is going.
Some of the more established names in this new field made presentations on work they had done to date, and people spent a good deal of time just talking with one another and brainstorming. It was a 3 day whirlwind which left my brain hurting every evening from the amount of knowledge it'd tried to absorb.
Beyond reports on state of the art techniques, a lot of time was dedicated to thinking about what exactly taking a photograph is doing. Both in theoretical terms of capturing very complex scenes, and thinking what of the traditional concepts of camera and image will continue to apply in the future. One thing that struck me was that there were almost as many ideas about what conputational photography means as there were people in attendance. Just some of the interpretations and variants present were:
-
Digital photography - image processing applied to make "better" images
-
Computational photography - processing captured images to make "new" images
-
Computational imaging - creative types of lenses, optics and uses to produce "new" images
-
Computational sensors - better CCD and CMOS that produce "smarter" pixels, meaning less work afterwards to produce the desired image
-
Computational illumination - Sharing parts of the previous 4, with an emphasis on creating special lighting to tell more about the scene
I'm still digesting all of my notes from it (I'm already out of practice of sitting in an auditorium and scribbling for 8 hours a day). I'll wait for the slides to make it online before I really go in depth with some of the more interesting stuff to make sure that my understanding was complete. But, in the mean time, here are some bullet points on things that caught me:
-
The scene is this huge and complex thing with many dimensions of data beyond the 3 that you normally think of. A conventional camera lens is very restrictive in how you can interact with the scene. New types of optics that completely do away with the lens as you know it are being investigated.
-
Computation can happen before the image is recorded to memory, making some current problems potentially much less troublesome.
-
Mice are cheaper than rats. (Yes, this does deal with computational photography, but I'll leave you guessing for now)
-
Sensors have come a long way, but the huge explosion the last 6 years has seen in image quality won't last much longer. Sensors will continue to increase in quality and size, but at a much slower pace that one is used to seeing from electronics in general. The laws of physics weight much heavier on their shoulders.
-
Sometimes it's better to solve hard problems poorly. In certain circumstances, doing so can allow you much more freedom to get better results later on.
I think that wraps up what I can get my head around at the moment. I'll be in New York City until Saturday evening before heading on to Washington DC until June 2nd.
Posted on May 26, 2005
Tags:
compphoto,
photography,
scpv