Colour - HDRI - Examples: Merge from Raw Files with Post Demosaicing

Through this example, some Canon EOS 5D Mark II CR2 files will be merged together prior to demosaicing in order to create a single radiance image.

Note: The final result exhibits a magenta cast, we haven't investigated the exact origin yet.

The following steps will be taken:

  • Conversion of the CR2 files to DNG files using Adobe DNG Converter.
  • Conversion of the DNG files to intermediate non demosaiced linear Tiff files using Dave Coffin's dcraw.
  • Creation of an image stack using DNG and intermediate Tiff files:
    • Reading of the DNG files Exif metadata using Phil Harvey's ExifTool.
    • Reading of the intermediate Tiff files pixel data using OpenImageIO.
  • Merging of the image stack into a radiance image.
  • White balancing of the radiance image.
  • Demosaicing of the radiance image.
  • Conversion of the radiance image to RGB display colourspace.
  • Display of the final resulting radiance image.

Note: Some steps can be performed using alternative methods or simplified, for instance the DNG conversion can be entirely avoided. Our interest here is to retrieve the camera levels and the Adobe DNG camera colour profiling data.

CR2 Files Conversion to DNG and Intermediate Files


In [1]:
import logging
import numpy as np
import os

import colour
from colour.plotting import *
from colour_demosaicing import (
    demosaicing_CFA_Bayer_DDFAPD,
    masks_CFA_Bayer)

from colour_hdri import (
    EXAMPLES_RESOURCES_DIRECTORY,
    Image,
    ImageStack,
    camera_space_to_sRGB,
    convert_dng_files_to_intermediate_files,
    convert_raw_files_to_dng_files,
    filter_files,
    read_exif_tag,
    image_stack_to_radiance_image,
    update_exif_tags,
    weighting_function_Debevec1997)
from colour_hdri.plotting import plot_radiance_image_strip

logging.basicConfig(level=logging.INFO)

RESOURCES_DIRECTORY = os.path.join(EXAMPLES_RESOURCES_DIRECTORY,
                                   'frobisher_001')

colour.utilities.filter_warnings()

colour.utilities.describe_environment();


===============================================================================
*                                                                             *
*   Interpreter :                                                             *
*       python : 3.7.4 (default, Sep  7 2019, 18:27:02)                       *
*                [Clang 10.0.1 (clang-1001.0.46.4)]                           *
*                                                                             *
*   colour-science.org :                                                      *
*       colour : 0.3.14                                                       *
*       colour-demosaicing : 0.1.5                                            *
*       colour-hdri : v0.1.5-60-g9648d86                                      *
*                                                                             *
*   Runtime :                                                                 *
*       imageio : 2.6.1                                                       *
*       matplotlib : 3.0.3                                                    *
*       numpy : 1.17.3                                                        *
*       scipy : 1.3.1                                                         *
*       six : 1.12.0                                                          *
*       recordclass : 0.12.0.1                                                *
*                                                                             *
===============================================================================

In [2]:
colour_style();

In [3]:
# In order to get proper black and white levels from .dng files
# we need to change the dng conversion arguments.
import colour_hdri.process.dng

colour_hdri.process.dng.DNG_CONVERSION_ARGUMENTS = '-e -d "{0}" "{1}"'

RAW_FILES = filter_files(RESOURCES_DIRECTORY, ('CR2',))

DNG_FILES = convert_raw_files_to_dng_files(RAW_FILES, RESOURCES_DIRECTORY)
    
INTERMEDIATE_FILES = convert_dng_files_to_intermediate_files(
    DNG_FILES, RESOURCES_DIRECTORY, demosaicing=False)

update_exif_tags(zip(DNG_FILES, INTERMEDIATE_FILES))

XYZ_TO_CAMERA_SPACE_MATRIX = colour.utilities.as_float_array(
    [float(M_c) for M_c in read_exif_tag(
        DNG_FILES[-2], 'ColorMatrix2').split()]).reshape((3, 3))


INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.CR2" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.tiff" file.
INFO:root:Converting "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng" file to "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.tiff" file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.tiff' file.
INFO:root:Copying '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng' file exif data to '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.tiff' file.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng' image 'ColorMatrix2' exif tag value: '0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1496 0.1937 0.6651'

In [4]:
plot_image(colour.cctf_encoding(colour.read_image(
            str(INTERMEDIATE_FILES[-2]))[1250:2250, 3000:4000, ...]),
           {'text': os.path.basename(INTERMEDIATE_FILES[-2])});


Radiance Image Merge and Demosaicing


In [5]:
def merge_from_raw_files_with_post_demosaicing(
        dng_files,
        output_directory,
        batch_size=5,
        black_level=None,
        white_level=None,
        white_balance_multipliers=None,
        weighting_function=weighting_function_Debevec1997,
        CFA_pattern='RGGB'):
    paths = []
    for dng_files in colour.utilities.batch(dng_files, batch_size):
        image_stack = ImageStack()
        for dng_file in dng_files:
            image = Image(dng_file)
            image.read_metadata()
            image.path = str(dng_file.replace('dng', 'tiff'))
            image.read_data()
            image_stack.append(image)

        path = os.path.join(
            output_directory,
            '{0}_{1}_MRFPD.{2}'.format(
                os.path.splitext(os.path.basename(image_stack.path[0]))[0],
                batch_size,
                'exr'))
        paths.append(path)

        logging.info('Scaling "{0}"...'.format(', '.join(
            image_stack.path)))
        black_level_e = (0 if
                         image_stack.black_level[0] is None else
                         np.max(image_stack.black_level[0]))
        white_level_e = (1 if
                         image_stack.white_level[0] is None else
                         np.min(image_stack.white_level[0]))
        logging.info('\tBlack Level (Exif): {0}'.format(
            image_stack.black_level))
        logging.info('\tWhite Level (Exif): {0}'.format(
            image_stack.white_level))
        black_level = black_level if black_level is not None else black_level_e
        white_level = white_level if white_level is not None else white_level_e
        logging.info('\tBlack Level (Used): {0}'.format(
            black_level))
        logging.info('\tWhite Level (Used): {0}'.format(
            white_level))
        # Scaling should be performed on individual planes, for convenience
        # and simplicity the maximum of the black level and the minimum of
        # the white level are used for all planes.
        image_stack.data = (image_stack.data - black_level) * (
            1 / (white_level_e - black_level_e))

        logging.info('Merging "{0}"...'.format(path))
        logging.info('\tImage stack "F Number" (Exif): {0}'.format(
            image_stack.f_number))
        logging.info('\tImage stack "Exposure Time" (Exif): {0}'.format(
            image_stack.exposure_time))
        logging.info('\tImage stack "ISO" (Exif): {0}'.format(
            image_stack.iso))
        image = image_stack_to_radiance_image(image_stack, weighting_function)
        image[np.isnan(image)] = 0

        logging.info('White Balancing "{0}"...'.format(path))
        white_balance_multipliers_e = np.power(
            image_stack.white_balance_multipliers[0], -1)
        logging.info('\tWhite Balance Multipliers (Exif): {0}'.format(
            white_balance_multipliers_e))
        white_balance_multipliers = (white_balance_multipliers
                                     if white_balance_multipliers is not None
                                     else white_balance_multipliers_e)
        logging.info('\tWhite Balance Multipliers (Used): {0}'.format(
            white_balance_multipliers))
        # For consistency and comparison ease with 
        # *Colour - HDRI - Example: Merge from Raw Files* example, the 
        # white balance multipliers are not normalised here too.
        # white_balance_multipliers /= np.max(white_balance_multipliers)
        R_m, G_m, B_m = masks_CFA_Bayer(image.shape, CFA_pattern)
        image[R_m] *= white_balance_multipliers[0]
        image[G_m] *= white_balance_multipliers[1]
        image[B_m] *= white_balance_multipliers[2]

        logging.info('Demosaicing "{0}"...'.format(path))
        image = demosaicing_CFA_Bayer_DDFAPD(image, CFA_pattern)

        logging.info('Writing "{0}"...'.format(path))
        colour.write_image(image, path)

    return paths


PATHS = merge_from_raw_files_with_post_demosaicing(
    DNG_FILES, RESOURCES_DIRECTORY)


INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.tiff" image.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.tiff" image.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.tiff" image.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.tiff" image.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng" image metadata.
INFO:root:Reading '/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.dng' image exif data.
INFO:root:Reading "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.tiff" image.
INFO:root:Scaling "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598.tiff, /Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2599.tiff, /Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2600.tiff, /Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2601.tiff, /Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2602.tiff"...
INFO:root:	Black Level (Exif): [[ 0.01560998  0.01560998  0.01560998  0.01560998]
 [ 0.01560998  0.01560998  0.01560998  0.01560998]
 [ 0.01562524  0.01562524  0.01562524  0.01562524]
 [ 0.01560998  0.01560998  0.01560998  0.01560998]
 [ 0.01560998  0.01560998  0.01560998  0.01560998]]
INFO:root:	White Level (Exif): [[ 0.23804074]
 [ 0.23804074]
 [ 0.23804074]
 [ 0.23804074]
 [ 0.23804074]]
INFO:root:	Black Level (Used): 0.01560997940032044
INFO:root:	White Level (Used): 0.23804074158846417
INFO:root:Merging "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598_5_MRFPD.exr"...
INFO:root:	Image stack "F Number" (Exif): [ 8.  8.  8.  8.  8.]
INFO:root:	Image stack "Exposure Time" (Exif): [  2.00000000e-03   1.66666667e-02   1.25000000e-01   1.00000000e+00
   8.00000000e+00]
INFO:root:	Image stack "ISO" (Exif): [ 100.  100.  100.  100.  100.]
INFO:root:White Balancing "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598_5_MRFPD.exr"...
INFO:root:	White Balance Multipliers (Exif): [ 2.42089718  1.          1.54687415]
INFO:root:	White Balance Multipliers (Used): [ 2.42089718  1.          1.54687415]
INFO:root:Demosaicing "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598_5_MRFPD.exr"...
INFO:root:Writing "/Users/kelsolaar/Documents/Development/colour-science/colour-hdri/colour_hdri/resources/colour-hdri-examples-datasets/frobisher_001/IMG_2598_5_MRFPD.exr"...

Camera Space to sRGB Colourspace Conversion and Radiance Image Display


In [6]:
IMAGE = colour.read_image(PATHS[0])

IMAGE_sRGB = camera_space_to_sRGB(IMAGE, XYZ_TO_CAMERA_SPACE_MATRIX)

colour.write_image(IMAGE_sRGB, PATHS[0].replace('.exr', '_sRGB.exr'))

plot_radiance_image_strip(IMAGE_sRGB);