Recurrent Dynamic Range Extension

Natural scenes contain a wide range of intensities spanning multiple orders of magnitude. This high dynamic range (HDR) can be perceived by our visual system without problem. However, the low dynamic range (LDR) of common cameras fails to capture the rich contrast, and most casual photos have lost colors and details in saturated pixels. Dynamic range extension aims to revert the clipping. Restoring the lost information in the highlights not only results in more realistic images, but also enables the physical modeling of light and opens up many downstream applications, such as image-based lighting and illumination-aware image editing. In this thesis, we approach this task using three novel formulations. Leveraging the physical properties of the scene, our models tame the long-tailed distribution of HDR radiance and relax the dependency on large HDR datasets by separating the problem into simpler sub-tasks. Looking into the image formation process, we observe important differences between the intrinsic components. For one, separating illumination from reflectance enables us to extend the dynamic range of the former without compressing the latter. Analogously, we can adapt to different material properties by separating moderate diffuse reflectance from the bright directed light of active light sources. Or, we take inspiration from the exposure mechanisms within the camera and progressively extend the dynamic range of an input image by one exposure value at a time. Our novel formulations allow for dedicated networks with a more precise supervision. We can drastically extend the limited diversity of HDR data by learning diffuse reflectance or a single-EV extension from widely available RAW image collections. This provides our models with a strong prior over natural images, combined with a realistic highlight distribution learned from full HDR data. Our novel perspectives improve performance and enable generalization to everyday photographs. We evaluate our approaches extensively on various benchmarks, including a novel multi-illuminant dataset. Our models show state-of-the-art results against a wide set of baselines, generate vivid colors, and reproduce bright highlights with a wide dynamic range. We outperform heavy diffusion-based networks with our streamlined architectures and create 4K results in less than a second.

The low dynamic range (LDR) of common cameras fails to capture the rich contrast in natural scenes, resulting in loss of color and details in saturated pixels. Reconstructing the high dynamic range (HDR) of luminance present in the scene from single LDR photographs is an important task with many applications in computational photography and realistic display of images. The HDR reconstruction task aims to infer the lost details using the context present in the scene, requiring neural networks to understand high-level geometric and illumination cues. This makes it challenging for data-driven algorithms to generate accurate and high-resolution results. In this work, we introduce a physically-inspired remodeling of the HDR reconstruction problem in the intrinsic domain. The intrinsic model allows us to train separate networks to extend the dynamic range in the shading domain and to recover lost color details in the albedo domain. We show that dividing the problem into two simpler sub-tasks improves performance in a wide variety of photographs.