Colour - HDRI - Examples: Merge from Raw Files

Through this example, some Canon EOS 5D Mark II CR2 files will be merged together in order to create a single radiance image.

The following steps will be taken:

  • Conversion of the CR2 files to DNG files using Adobe DNG Converter.
  • Conversion of the DNG files to intermediate demosaiced linear Tiff files using Dave Coffin's dcraw.
  • Creation of an image stack using DNG and intermediate Tiff files:
    • Reading of the DNG files Exif metadata using Phil Harvey's ExifTool.
    • Reading of the intermediate Tiff files pixel data using OpenImageIO.
    • White balancing of the intermediate Tiff files.
    • Conversion of the intermediate Tiff files to RGB display colourspace.
  • Merging of the image stack into a radiance image.
  • Display of the final resulting radiance image.

Note: Some steps can be performed using alternative methods or simplified, for instance the DNG conversion can be entirely avoided. Our interest here is to retrieve the camera levels and the Adobe DNG camera colour profiling data.

CR2 Files Conversion to DNG and Intermediate Files


In [1]:
import logging
import matplotlib.pyplot as plt
import numpy as np
import os

import colour
from colour.plotting import *

from colour_hdri import (
    EXAMPLES_RESOURCES_DIRECTORY,
    Image,
    ImageStack,
    camera_space_to_sRGB,
    convert_dng_files_to_intermediate_files,
    convert_raw_files_to_dng_files,
    filter_files,
    read_exif_tag,
    image_stack_to_radiance_image,
    update_exif_tags,
    weighting_function_Debevec1997)
from colour_hdri.plotting import plot_radiance_image_strip

logging.basicConfig(level=logging.INFO)


RESOURCES_DIRECTORY = os.path.join(EXAMPLES_RESOURCES_DIRECTORY,
                                   'frobisher_001')

colour.utilities.filter_warnings()

colour.utilities.describe_environment();


===============================================================================
*                                                                             *
*   Interpreter :                                                             *
*       python : 3.7.4 (default, Sep  7 2019, 18:27:02)                       *
*                [Clang 10.0.1 (clang-1001.0.46.4)]                           *
*                                                                             *
*   colour-science.org :                                                      *
*       colour : 0.3.14                                                       *
*       colour-hdri : v0.1.5-60-g9648d86                                      *
*                                                                             *
*   Runtime :                                                                 *
*       imageio : 2.6.1                                                       *
*       matplotlib : 3.0.3                                                    *
*       numpy : 1.17.3                                                        *
*       scipy : 1.3.1                                                         *
*       six : 1.12.0                                                          *
*       recordclass : 0.12.0.1                                                *
*                                                                             *
===============================================================================

In [2]:
colour_style();

In [3]:
RAW_FILES = filter_files(RESOURCES_DIRECTORY, ('CR2',))

DNG_FILES = convert_raw_files_to_dng_files(RAW_FILES, RESOURCES_DIRECTORY)
    
INTERMEDIATE_FILES = convert_dng_files_to_intermediate_files(
    DNG_FILES, RESOURCES_DIRECTORY, demosaicing=True)

update_exif_tags(zip(DNG_FILES, INTERMEDIATE_FILES))

XYZ_TO_CAMERA_SPACE_MATRIX = colour.utilities.as_float_array(
    [float(M_c) for M_c in read_exif_tag(
        DNG_FILES[-2], 'ColorMatrix2').split()]).reshape((3, 3))


INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.tiff" file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.tiff' file.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng' image 'ColorMatrix2' exif tag value: '0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1496 0.1937 0.6651'

In [4]:
plot_image(colour.cctf_encoding(colour.read_image(
            str(INTERMEDIATE_FILES[-2]))[1250:2250, 3000:4000, ...]),
           {'text': os.path.basename(INTERMEDIATE_FILES[-2])});


Radiance Image Merge


In [5]:
def merge_from_raw_files(
        dng_files,
        output_directory,
        batch_size=5,
        white_balance_multipliers=None,
        weighting_function=weighting_function_Debevec1997):
    paths = []
    for dng_files in colour.utilities.batch(dng_files, batch_size):
        image_stack = ImageStack()
        for dng_file in dng_files:
            image = Image(dng_file)
            image.read_metadata()
            image.path = str(dng_file.replace('dng', 'tiff'))
            image.read_data()
            white_balance_multipliers_e = np.power(
                image.metadata.white_balance_multipliers, -1)
            logging.info('\tWhite Balance Multipliers (Exif): {0}'.format(
                white_balance_multipliers_e))
            white_balance_multipliers = (
                white_balance_multipliers
                if white_balance_multipliers is not None
                else white_balance_multipliers_e)
            logging.info('\tWhite Balance Multipliers (Used): {0}'.format(
                white_balance_multipliers))
            # Avoiding white balance normalisation thus avoiding highlights
            # recovery as weighting will exclude saturated luminance values.             
            # white_balance_multipliers /= np.max(white_balance_multipliers)
            image_data = image.data * white_balance_multipliers
            # image_data = highlights_recovery_blend(
            #    image.data * white_balance_multipliers,
            #    white_balance_multipliers)
            image.data = camera_space_to_sRGB(image_data,
                                              XYZ_TO_CAMERA_SPACE_MATRIX)
            image_stack.append(image)

        path = os.path.join(
            output_directory,
            '{0}_{1}_MRF.{2}'.format(
                os.path.splitext(os.path.basename(image_stack.path[0]))[0],
                batch_size,
                'exr'))
        paths.append(path)

        logging.info('Merging "{0}"...'.format(path))
        logging.info('\tImage stack "F Number" (Exif): {0}'.format(
            image_stack.f_number))
        logging.info('\tImage stack "Exposure Time" (Exif): {0}'.format(
            image_stack.exposure_time))
        logging.info('\tImage stack "ISO" (Exif): {0}'.format(
            image_stack.iso))
        image = image_stack_to_radiance_image(image_stack,
                                              weighting_function,
                                              weighting_average=True)

        logging.info('Writing "{0}"...'.format(path))
        colour.write_image(image, path)

    return paths


PATHS = merge_from_raw_files(DNG_FILES, RESOURCES_DIRECTORY)


INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.tiff" image.
INFO:root:	White Balance Multipliers (Exif): [ 2.42089718  1.          1.54687415]
INFO:root:	White Balance Multipliers (Used): [ 2.42089718  1.          1.54687415]
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.tiff" image.
INFO:root:	White Balance Multipliers (Exif): [ 2.42089718  1.          1.54687415]
INFO:root:	White Balance Multipliers (Used): [ 2.42089718  1.          1.54687415]
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.tiff" image.
INFO:root:	White Balance Multipliers (Exif): [ 2.42089718  1.          1.54687415]
INFO:root:	White Balance Multipliers (Used): [ 2.42089718  1.          1.54687415]
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.tiff" image.
INFO:root:	White Balance Multipliers (Exif): [ 2.42089718  1.          1.54687415]
INFO:root:	White Balance Multipliers (Used): [ 2.42089718  1.          1.54687415]
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.tiff" image.
INFO:root:	White Balance Multipliers (Exif): [ 2.42089718  1.          1.54687415]
INFO:root:	White Balance Multipliers (Used): [ 2.42089718  1.          1.54687415]
INFO:root:Merging "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598_5_MRF.exr"...
INFO:root:	Image stack "F Number" (Exif): [ 8.  8.  8.  8.  8.]
INFO:root:	Image stack "Exposure Time" (Exif): [  2.00000000e-03   1.66666667e-02   1.25000000e-01   1.00000000e+00
   8.00000000e+00]
INFO:root:	Image stack "ISO" (Exif): [ 100.  100.  100.  100.  100.]
/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/generation/radiance.py:130: RuntimeWarning: invalid value encountered in true_divide
  image_c /= weight_c
INFO:root:Writing "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598_5_MRF.exr"...

Radiance Image Display


In [6]:
plot_radiance_image_strip(colour.read_image(PATHS[0]));


/Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/colour-hdri-bkoxrqd3-py3.7/lib/python3.7/site-packages/colour/models/rgb/transfer_functions/srgb.py:94: RuntimeWarning: invalid value encountered in less_equal
  V = np.where(L <= 0.0031308, L * 12.92, 1.055 * spow(L, 1 / 2.4) - 0.055)