Photometric Image Processing for HDR Displays

Teaser 1 Teaser 2 Teaser 3
The left image represents the original HDR image tonemapped for print. The center image represents the low-frequency luminance image of the backlight, and the right image represents the LCD image compensated for the backlight.
Matthew Trentacoste , Wolfgang Heidrich

Abstract

Many real-world scenes contain a dynamic range that exceeds conventional display technology by several orders of magnitude. Through the combination of several existing technologies, new high dynamic range displays, capable of reproducing a range of intensities much closer to that of real environments, have been constructed. These benefits come at the cost of more optically complex devices; involving two image modulators, controlled in unison, to display images. We present several methods of rendering images to this new class of devices for reproducing photometrically accurate images. We discuss the process of calibrating a display, matching the response of the device with our ideal model. We then derive series of methods for efficiently displaying images, optimized for different criteria and evaluate them in a perceptual framework.

Background

No single known material is capable of reproducing the luminances and bit depths in the resolutions and form factors required for displaying HDR images, and a fundamental change in how output device display images is required. A conventional display uses a single high resolution LCD panel as an optical filter in front of a uniform light source, like a fluorescent lamp. The limited contrast of a LCD panel requires an additional optical modulator to be added, and the design of an HDR display accomplishes this by replacing the uniform light with a second low resolution, high contrast display. There are many ways to create the display, but in practice either a projector or a grid of ultra-bright LEDs is used. By simultaneously controlling the LCD panel and the second display, the two work in tandem to produce the final image.

This new configuration offers many benefits over conventional displays, but presents several additional challenges. If one desires to alter the luminance of a pixel by using the low resolution backlight, the surrounding pixels are altered as well. Fundamentally, this limitation implies that HDR displays cannot exactly reproduce the luminances of a real scene. However, since the display is intended to be viewed by human subjects, exact reproduction is not necessary. As long as the display introduces less distortion than the human visual system, the original image and the displayed image will appear the same.

Unlike conventional displays, the pixels in the HDR display are no longer completely independent of one another. It is therefore necessary to employ image-processing algorithms to factor an HDR into values to send to the LCD panel, and to the low resolution back plane, respectively. This thesis addresses the challenge of: given an image as input, compute a matching set of front and back images such that the optics of the display combine to produce the same observed image as the original.

Overview

Given an image within the displayable color space, we must determine the LED driving values and LCD panel image, that when combined by the optics of a given HDR display, minimize the perceived error between the original and the reconstruction. Not only must the pair of images accomplish that goal, but those images must be displayable by the monitor hardware. The hardware constraints force us to search for two LDR images that can be combined to approximate an HDR image. The same general approach applies to both form factors of HDR displays discussed in HDR Display Systems , but we focus on LED displays. All implementations were done for the BrightSide DR-37P in particular.

The backlight image in the center is a low-frequency approximation of the original image on the left. The approximation is unable to reproduce the high-contrast boundary; producing less light than desired on the brighter side and more light than desired on the darker side. The LCD image on the right of the figure below compensates for this blur by letting more light through on the brighter side and less light through on the darker side, so that the end result is a high-contrast boundary between two uniform regions of luminance.

Blur correction
Blur correction steps. From the input image (left), the low frequency backlight (center) is computed and simulated, which is used to compensate the original to produce the LCD panel image (right).

Based upon the previously detailed arguments our approach is decomposed into several stages, with the corresponding flowchart and images:

  • Given the desired image I , determine target backlight B
  • Determine the LED driving levels d that most closely approximate B .
  • Given d , simulate the resulting backlight B' .
  • Determine the LCD panel p that corrects for the low resolution of the backlight B' , and when combined with B' by the display optics, approximates I .
Image processing flowchart
Flowchart of stages of the implementation. The scene-referred input image is used to determine the desired backlight, which in turn is used to determine the LED driving values. The backlight is simulated from these driving values, and this simulation is used to compensate the original image.

We address the details of the algorithm on the two hardware platforms currently used in production, a graphics processing unit (GPU) and a field-programmable gate array (FGPA) located in the HDR display. We also describe the methods used in the software testbed to provide high-quality comparisons by which we can judge the chosen optimizations.

Results

Here are several exapmles results of our techniques and evaluation of the quality of the images produced by the hardware compared with the desired image. We make use of Mantiuk et al's High dynamic range visible differences predictor (HDR VDP) to perform the comparison and show that, while the hardware limitations prevent reproducing the exact luminances of the original, a human observer cannot readily detect the majority of the differences.

The following sets of images depict the following: 1) original HDR image tone-mapped for display, 2) simulated HDR displayed image, 3) error map depicting VDP output.

Original image Simulated image HDR VDP output
HDR VDP comparison of original and display image of an indoor/outdoor apartment scene.
Original image Simulated image HDR VDP output
HDR VDP comparison of original and display image of an outdoor scene.

Publications

Presentations