Flash and Ambient Illuminations Dataset

The Dataset

The Flash and Ambient Illuminations Dataset (FAID) consists of aligned flash-only and ambient-only illumination pairs captured with mobile devices by many crowd-workers participated in our collection effort. This dataset accompanies our ECCV 2018 paper.

Capture Setup

The photographs were captured in 12-bit raw and jpeg formats using iPhone 6s and iPhone 7 devices by the crowd-workers we reached through the Amazon's Mechanical Turk platform. The raw images were used to generate and represent the flash and ambient illuminations. The jpeg images proccesed by iPhones are provided as well in the dataset. In our setup, the flash and no-flash photograph pairs were captured in burst, typically 0.5-1.0 seconds apart. The two photographs were then aligned via homography estimations, cropped and resized to 1080x1440 resolution. The aligned raw flash and no-flash photographs are then used to compute the flash-only illumination. The photographs and their alignment went through a review process before getting included in the dataset.

Scenes and Illumination Conditions

The scenes are typically well-illuminated indoor spaces, without any visible direct light sources. The objects and materials otherwise has a wide variety. We defined the scenes to be indoors for a reliable flash illumination estimation, and well-illuminated in order to get a useful ambient illumination as well. As two illuminations are provided separately, the ambient illumination can be made dimmer and added to the flash illumination using the raw values to create a typical flash photograph taken in a dark envrionment.

Categories

The dataset is divided into 6 categories: People, Shelves, Plants, Toys, Rooms, and Objects. The first 5 categories were defined in order to give the crowd-workers a starting point when they are photographing, and the Objects category was then defined to include images that did not fit well in any of the defined categories.

Color Mapping

The illuminations are provided as raw sensor readings, and the corresponding exif information are available in the dataset as separate files. The illuminations can be mapped to first XYZ and then RGB color spaces via the color matrix. The color matrix and the calibration illuminant we used are available in the exif data of the PNG files as well as the .mat files with the complete exif information. A sample code in MATLAB for the color mapping is available here. Note that even after the color mapping, the images will not match the colors of the jpeg photographs we provide, as the jpeg photographs went through the pre-processing pipeline of the cameras.

File Description

We provide the illumination pairs in 16-bit PNG files. These files already have the color matrix and calibration illuminant information necessary for the color mapping. The full exif information originally attached to the DNG files are provided as .mat files separately. We also provide the flash and no-flash photographs processed by the iPhones. Note that the flash photograph in this case includes both flash and ambient illuminations.

License and Credit

We provide this dataset for research purposes only, and it is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. We hereby give the rights to use the images in figures and supplementary material of scientific publications. If you use this dataset, please cite our paper:

Included in our extended dataset

–

Extended Flash and Ambient Illuminations Dataset Brief Description We combine and extend existing flash/no-flash datasets to create a diverse set of real-world flash/ambient pairs suitable for training deep networks. Our dataset is constructed from three existing datasets: The Multi-Illumination Dataset (MID), The Flash and Ambient Illuminations Dataset (FAID), and the Deep Flash Portrait Dataset (DPD). We propose a pipeline for compositing portraits from DPD onto backgrounds from FAID, and discuss multiple considerations for normalizing and augmenting flash/no-flash data. This dataset accompanies our CVPR 2023 paper. BibTeX @INPROCEEDINGS{Maralan2023Flash, author={Sepideh Sarajian Maralan and Chris Careaga and Ya\u{g}{\i}z Aksoy}, title={Computational Flash Photography through Intrinsics}, journal={Proc. CVPR}, year={2023}, }

Files

ECCV Publication

A Dataset of Flash and Ambient Illumination Pairs from the Crowd Yağız Aksoy, Changil Kim, Petr Kellnhofer, Sylvain Paris, Mohamed Elgharib, Marc Pollefeys and Wojciech Matusik ECCV, 2018 Abstract Illumination is a critical element of photography and is essential for many computer vision tasks. Flash light is unique in the sense that it is a widely available tool for easily manipulating the scene illumination. We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations. Different than the typical use of crowdsourcing in generating computer vision datasets, we make use of the crowd to directly take the photographs that make up our dataset. As a result, our dataset covers a wide variety of scenes captured by many casual photographers. We detail the advantages and challenges of our approach to crowdsourcing as well as the computational effort to generate completely separate flash illuminations from the ambient light in an uncontrolled setup. We present a brief examination of illumination decomposition, a challenging and underconstrained problem in flash photography, to demonstrate the use of our dataset in a data-driven approach. Manuscript BibTeX @INPROCEEDINGS{flashambient, author={Ya\u{g}{\i}z Aksoy and Changil Kim and Petr Kellnhofer and Sylvain Paris and Mohamed Elgharib and Marc Pollefeys and Wojciech Matusik}, booktitle={Proc. ECCV}, title={A Dataset of Flash and Ambient Illumination Pairs from the Crowd}, year={2018}, }