Computer Vision Homework1 (due: ~2014/09/24 09:00)

20115009 Kyunghoon Kim( kyunghoon@unist.ac.kr )


In [1]:
from PIL import Image # Load PIL(Python Image Library)
import matplotlib.pyplot as plt # Load pyplot library
import numpy as np # Load Numerical Library

I prepared the 2x2 size image with red, blue.

We can see the process for image processing in python.


In [170]:
im = Image.open('2x2.jpg') # Load the image file
imarray = np.array(im) # Construct the array

In [180]:
plt.imshow(im); # Show the image;
np.array(im) # each pixel have 3 values.


Out[180]:
array([[[254,   0,   0],
        [  1,  66, 255]],

       [[  1,  66, 255],
        [254,   0,   0]]], dtype=uint8)

In [173]:
imgray = im.convert('L') # Convert the image to gray scale

In [182]:
plt.imshow(imgray); # It looks like color, but each pixels are just one values.;
np.array(imgray)


Out[182]:
array([[75, 68],
       [68, 75]], dtype=uint8)

In [188]:
plt.imshow(imgray, cmap=plt.cm.gray) # To show the gray scale, we use the cmap(color map);



In [190]:
plt.imshow(imgray, cmap=plt.get_cmap('gray')) # or use it;



In [192]:
imgray_array = np.array(imgray) # Let imgray_array be the array of imgray.
print imgray_array # As we know, each pixels have a value.


[[75 68]
 [68 75]]

In [193]:
imgray_array.flatten() # To make the histogram, we use the flatten()


Out[193]:
array([75, 68, 68, 75], dtype=uint8)

In [197]:
plt.hist(imgray_array.flatten(), 256, [0,256]);
# Using the hist(), we can draw histogram. 256 means the count of equispaced pieces, [0, 256] is range.


Out[197]:
(array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  2.,  0.,  0.,  0.,  0.,  0.,  0.,  2.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]),
 array([   0.,    1.,    2.,    3.,    4.,    5.,    6.,    7.,    8.,
           9.,   10.,   11.,   12.,   13.,   14.,   15.,   16.,   17.,
          18.,   19.,   20.,   21.,   22.,   23.,   24.,   25.,   26.,
          27.,   28.,   29.,   30.,   31.,   32.,   33.,   34.,   35.,
          36.,   37.,   38.,   39.,   40.,   41.,   42.,   43.,   44.,
          45.,   46.,   47.,   48.,   49.,   50.,   51.,   52.,   53.,
          54.,   55.,   56.,   57.,   58.,   59.,   60.,   61.,   62.,
          63.,   64.,   65.,   66.,   67.,   68.,   69.,   70.,   71.,
          72.,   73.,   74.,   75.,   76.,   77.,   78.,   79.,   80.,
          81.,   82.,   83.,   84.,   85.,   86.,   87.,   88.,   89.,
          90.,   91.,   92.,   93.,   94.,   95.,   96.,   97.,   98.,
          99.,  100.,  101.,  102.,  103.,  104.,  105.,  106.,  107.,
         108.,  109.,  110.,  111.,  112.,  113.,  114.,  115.,  116.,
         117.,  118.,  119.,  120.,  121.,  122.,  123.,  124.,  125.,
         126.,  127.,  128.,  129.,  130.,  131.,  132.,  133.,  134.,
         135.,  136.,  137.,  138.,  139.,  140.,  141.,  142.,  143.,
         144.,  145.,  146.,  147.,  148.,  149.,  150.,  151.,  152.,
         153.,  154.,  155.,  156.,  157.,  158.,  159.,  160.,  161.,
         162.,  163.,  164.,  165.,  166.,  167.,  168.,  169.,  170.,
         171.,  172.,  173.,  174.,  175.,  176.,  177.,  178.,  179.,
         180.,  181.,  182.,  183.,  184.,  185.,  186.,  187.,  188.,
         189.,  190.,  191.,  192.,  193.,  194.,  195.,  196.,  197.,
         198.,  199.,  200.,  201.,  202.,  203.,  204.,  205.,  206.,
         207.,  208.,  209.,  210.,  211.,  212.,  213.,  214.,  215.,
         216.,  217.,  218.,  219.,  220.,  221.,  222.,  223.,  224.,
         225.,  226.,  227.,  228.,  229.,  230.,  231.,  232.,  233.,
         234.,  235.,  236.,  237.,  238.,  239.,  240.,  241.,  242.,
         243.,  244.,  245.,  246.,  247.,  248.,  249.,  250.,  251.,
         252.,  253.,  254.,  255.,  256.]),
 <a list of 256 Patch objects>)

In [194]:
np.histogram(imgray_array.flatten(),256,[0,256]) # for the output, first array is count, second array is range of value.


Out[194]:
(array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2,
        0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0]),
 array([   0.,    1.,    2.,    3.,    4.,    5.,    6.,    7.,    8.,
           9.,   10.,   11.,   12.,   13.,   14.,   15.,   16.,   17.,
          18.,   19.,   20.,   21.,   22.,   23.,   24.,   25.,   26.,
          27.,   28.,   29.,   30.,   31.,   32.,   33.,   34.,   35.,
          36.,   37.,   38.,   39.,   40.,   41.,   42.,   43.,   44.,
          45.,   46.,   47.,   48.,   49.,   50.,   51.,   52.,   53.,
          54.,   55.,   56.,   57.,   58.,   59.,   60.,   61.,   62.,
          63.,   64.,   65.,   66.,   67.,   68.,   69.,   70.,   71.,
          72.,   73.,   74.,   75.,   76.,   77.,   78.,   79.,   80.,
          81.,   82.,   83.,   84.,   85.,   86.,   87.,   88.,   89.,
          90.,   91.,   92.,   93.,   94.,   95.,   96.,   97.,   98.,
          99.,  100.,  101.,  102.,  103.,  104.,  105.,  106.,  107.,
         108.,  109.,  110.,  111.,  112.,  113.,  114.,  115.,  116.,
         117.,  118.,  119.,  120.,  121.,  122.,  123.,  124.,  125.,
         126.,  127.,  128.,  129.,  130.,  131.,  132.,  133.,  134.,
         135.,  136.,  137.,  138.,  139.,  140.,  141.,  142.,  143.,
         144.,  145.,  146.,  147.,  148.,  149.,  150.,  151.,  152.,
         153.,  154.,  155.,  156.,  157.,  158.,  159.,  160.,  161.,
         162.,  163.,  164.,  165.,  166.,  167.,  168.,  169.,  170.,
         171.,  172.,  173.,  174.,  175.,  176.,  177.,  178.,  179.,
         180.,  181.,  182.,  183.,  184.,  185.,  186.,  187.,  188.,
         189.,  190.,  191.,  192.,  193.,  194.,  195.,  196.,  197.,
         198.,  199.,  200.,  201.,  202.,  203.,  204.,  205.,  206.,
         207.,  208.,  209.,  210.,  211.,  212.,  213.,  214.,  215.,
         216.,  217.,  218.,  219.,  220.,  221.,  222.,  223.,  224.,
         225.,  226.,  227.,  228.,  229.,  230.,  231.,  232.,  233.,
         234.,  235.,  236.,  237.,  238.,  239.,  240.,  241.,  242.,
         243.,  244.,  245.,  246.,  247.,  248.,  249.,  250.,  251.,
         252.,  253.,  254.,  255.,  256.]))

In [202]:
hist,bins = np.histogram(imgray_array.flatten(),256,[0,256])
# So I assign the variable 'hist' and 'bins' for count and range of pieces

In [199]:
cdf = hist.cumsum() # Cumulative Distribution Function
print 'CDF :\n', cdf
cdf_normalized = cdf * hist.max()/ cdf.max() # Normalized Cumulative Distribution Function
print 'cdf_normalized :\n', cdf_normalized


CDF :
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 2 2 2 2
 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4]
cdf_normalized :
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1
 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2]

In [204]:
plt.plot(cdf_normalized, color = 'b'); # Plot the normalized CDF;



In [209]:
plt.hist(imgray_array.flatten(), 256, [0,256], color = 'r'); # Plot of histogram;
plt.plot(cdf_normalized, color = 'b'); # Plot of normalized CDF;
plt.xlim([0,256]); # range of X-axis;
plt.legend(('cdf','histogram'), loc = 'upper right'); # Attach the legend;


Now we try it for the big image.

I take a photo in Taehwa river, but the below of image is dark.


In [2]:
im = Image.open('flower.jpg')
imarray = np.array(im)
plt.imshow(imarray); # Image Show;



In [14]:
imgray = im.convert('L') # Convert to gray scale image
imgray_array = np.array(imgray) # Assign as array
plt.imshow(imgray, cmap=plt.get_cmap('gray')); # Image show as gray scale;



In [15]:
print np.array(imgray) # each pixel have a one value (min=0, max=255)
print 'Min : ', np.array(imgray).min()
print 'Max : ', np.array(imgray).max()


[[230 229 229 ..., 168 169 170]
 [232 231 231 ..., 169 172 174]
 [232 232 232 ..., 172 175 177]
 ..., 
 [ 38  39  22 ...,  42  38  34]
 [ 33  38  29 ...,  38  35  30]
 [ 37  38  36 ...,  36  35  30]]
Min :  0
Max :  255

In [16]:
# Samething for 2x2 image.
hist,bins = np.histogram(imgray_array.flatten(),256,[0,256])
plt.hist(imgray_array.flatten(), 256, [0,256], color = 'r'); # Plot Histogram

cdf = hist.cumsum()
cdf_normalized = cdf * hist.max()/ cdf.max()

plt.plot(cdf_normalized, color = 'b'); # Plot normalized CDF
plt.xlim([0,256]);
plt.legend(('cdf','histogram'), loc = 'upper right');


The CDF is not equal speed between 0 and 150. And the dark part is big.

So we want to make the equalization.

We can use the equalization formula from http://en.wikipedia.org/wiki/Histogram_equalization

$$ h(v) = \mathrm{round} \left( \frac {cdf(v) - cdf_{min}} {(\text{image's number of pixels}) - cdf_{min}} \times (L - 1) \right) $$

In [17]:
h = (256-1)*(cdf - cdf.min())/(cdf.max()-cdf.min()) # all values are integer. So this is automatic round.
# Each cdf value in (0, 255) has transformed value h[#].

In [35]:
transformed_img = h[imgray_array] # the value of imgray_array is index of h.

In [28]:
imgray_array # For example, the first value of imgray_array is 230.


Out[28]:
array([[230, 229, 229, ..., 168, 169, 170],
       [232, 231, 231, ..., 169, 172, 174],
       [232, 232, 232, ..., 172, 175, 177],
       ..., 
       [ 38,  39,  22, ...,  42,  38,  34],
       [ 33,  38,  29, ...,  38,  35,  30],
       [ 37,  38,  36, ...,  36,  35,  30]], dtype=uint8)

In [34]:
h[230] # h[230] is 227 which is transformed value by equalization formula.


Out[34]:
227

In [43]:
fig = plt.figure()
fig.set_size_inches(15,10)

a=fig.add_subplot(1,2,1)
plt.title('Original Image');
plt.imshow(imgray, cmap=plt.get_cmap('gray'));

a=fig.add_subplot(1,2,2)
plt.title('Transformed Image');
plt.imshow(transformed_img, cmap=plt.get_cmap('gray'));



In [47]:
fig = plt.figure()
fig.set_size_inches(15,10)

# Original Image
a=fig.add_subplot(1,2,1)
plt.title('Histogram, CDF of Original Image');
hist,bins = np.histogram(imgray_array.flatten(),256,[0,256])
plt.hist(imgray_array.flatten(), 256, [0,256], color = 'r'); # Plot Histogram
cdf = hist.cumsum()
cdf_normalized = cdf * hist.max()/ cdf.max()
plt.plot(cdf_normalized, color = 'b'); # Plot normalized CDF
plt.xlim([0,256]);
plt.legend(('cdf','histogram'), loc = 'upper right');

# Transformed Image
a=fig.add_subplot(1,2,2)
plt.title('Histogram, CDF of Transformed Image');
hist,bins = np.histogram(img2.flatten(),256,[0,256])
plt.hist(img2.flatten(), 256, [0,256], color = 'r');
cdf = hist.cumsum()
cdf_normalized = cdf * hist.max()/ cdf.max()
plt.plot(cdf_normalized, color = 'b');
plt.xlim([0,256]);
plt.legend(('cdf','histogram'), loc = 'upper right');



In [73]:
""" Total Comparison """
fig = plt.figure()
fig.set_size_inches(15,10)

a=fig.add_subplot(2,2,1)
plt.title('Original Image');
plt.imshow(imgray, cmap=plt.get_cmap('gray'));

a=fig.add_subplot(2,2,2)
plt.title('Transformed Image');
plt.imshow(transformed_img, cmap=plt.get_cmap('gray'));

# Original Image
a=fig.add_subplot(2,2,3)
plt.title('Histogram, CDF of Original Image');
hist,bins = np.histogram(imgray_array.flatten(),256,[0,256])
plt.hist(imgray_array.flatten(), 256, [0,256], color = 'r'); # Plot Histogram
cdf = hist.cumsum()
cdf_normalized = cdf * hist.max()/ cdf.max()
plt.plot(cdf_normalized, color = 'b'); # Plot normalized CDF
plt.xlim([0,256]);
plt.legend(('cdf','histogram'), loc = 'upper right');

# Transformed Image
a=fig.add_subplot(2,2,4)
plt.title('Histogram, CDF of Transformed Image');
hist,bins = np.histogram(img2.flatten(),256,[0,256])
plt.hist(img2.flatten(), 256, [0,256], color = 'r');
cdf = hist.cumsum()
cdf_normalized = cdf * hist.max()/ cdf.max()
plt.plot(cdf_normalized, color = 'b');
plt.xlim([0,256]);
plt.legend(('cdf','histogram'), loc = 'upper right');


So we make the function


In [68]:
def equalization(im):
    fig = plt.figure()
    fig.set_size_inches(15,10)
    a=fig.add_subplot(3,2,1) # Original Color Image
    imarray = np.array(im)
    plt.imshow(imarray)
    imgray = im.convert('L')
    imgray_array = np.array(imgray)
    
    a=fig.add_subplot(3,2,3)
    plt.title('Original Image');
    plt.imshow(imgray, cmap=plt.get_cmap('gray'));
    hist, bins = np.histogram(imgray_array.flatten(),256,[0,256])
    cdf = hist.cumsum()
    cdf_normalized = cdf * hist.max()/ cdf.max()
    
    a=fig.add_subplot(3,2,4)
    h = (256-1)*(cdf - cdf.min())/(cdf.max()-cdf.min())
    transformed_img = h[imgray_array]
    plt.title('Transformed Image');
    plt.imshow(transformed_img, cmap=plt.get_cmap('gray'));

    # Original Image
    a=fig.add_subplot(3,2,5)
    plt.title('Histogram, CDF of Original Image');
    plt.hist(imgray_array.flatten(), 256, [0,256], color = 'r'); # Plot Histogram
    plt.plot(cdf_normalized, color = 'b'); # Plot normalized CDF
    plt.xlim([0,256]);
    plt.legend(('cdf','histogram'), loc = 'upper right');

    # Transformed Image
    a=fig.add_subplot(3,2,6)
    plt.title('Histogram, CDF of Transformed Image');
    hist, bins = np.histogram(transformed_img.flatten(),256,[0,256])
    plt.hist(transformed_img.flatten(), 256, [0,256], color = 'r');
    cdf = hist.cumsum()
    cdf_normalized = cdf * hist.max()/ cdf.max()
    plt.plot(cdf_normalized, color = 'b');
    plt.xlim([0,256]);
    plt.legend(('cdf','histogram'), loc = 'upper right');

Demo


In [69]:
# Case 1
image = Image.open('unist.jpg')
equalization(image)



In [70]:
# Case 2
image = Image.open('IU.jpg')
equalization(image)



In [71]:
# Case 3
image = Image.open('guitar.jpg')
equalization(image)



In [72]:
# Case 4
image = Image.open('cycle.jpg')
equalization(image)


Conclusion

The general histogram equalization formula is not golden key. For the landscape photos, the performance is good. But people photo and monotone color photo(actually darked image) has a bad performance. So we need to try the various approaches with other transformation idea. Or for instance, after detect the edge of object, try the locally adaptive histogram equalization. This will be adapt method for case 4 motor cycle maybe. I'm not sure, I don't know the edge detection. But these approaches are so interesting to me.

References

  1. http://en.wikipedia.org/wiki/Histogram_equalization
  2. http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_imgproc/py_histograms/py_histogram_equalization/py_histogram_equalization.html
  3. Bradski, Gary, and Adrian Kaehler. Learning OpenCV: Computer vision with the OpenCV library. " O'Reilly Media, Inc.", 2008.
  4. Szeliski, Richard. Computer vision: algorithms and applications. Springer, 2010.