In [3]:
import h2o
h2o.init()
data = h2o.import_file("towerbridge.csv")


Checking whether there is an H2O instance running at http://localhost:54321..... not found.
Attempting to start a local H2O server...
; Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=trueed mode)
  Starting server from C:\ProgramData\Miniconda3\lib\site-packages\h2o\backend\bin\h2o.jar
  Ice root: C:\Users\edwarddi\AppData\Local\Temp\tmp1of40oys
  JVM stdout: C:\Users\edwarddi\AppData\Local\Temp\tmp1of40oys\h2o_edwarddi_started_from_python.out
  JVM stderr: C:\Users\edwarddi\AppData\Local\Temp\tmp1of40oys\h2o_edwarddi_started_from_python.err
  Server is running at http://127.0.0.1:54321
Connecting to H2O server at http://127.0.0.1:54321... successful.
H2O cluster uptime: 06 secs
H2O cluster timezone: Europe/London
H2O data parsing timezone: UTC
H2O cluster version: 3.20.0.10
H2O cluster version age: 5 days
H2O cluster name: H2O_from_python_edwarddi_eerjqv
H2O cluster total nodes: 1
H2O cluster free memory: 3.531 Gb
H2O cluster total cores: 4
H2O cluster allowed cores: 4
H2O cluster status: accepting new members, healthy
H2O connection url: http://127.0.0.1:54321
H2O connection proxy: None
H2O internal security: False
H2O API Extensions: Algos, AutoML, Core V3, Core V4
Python version: 3.7.0 final
Parse progress: |█████████████████████████████████████████████████████████| 100%

Our Data

To use it with H2O modelling, we've converted our raster image (a grid of colour values) to a list of tuples, mapping the pixel coordinates to the colour values.


In [4]:
data


C1 b g r x y
0 96 28 0-0.0233463-0.0233463
1 94 28 0-0.0231632-0.0233463
2 94 28 0-0.0229801-0.0233463
3 93 29 0-0.022797 -0.0233463
4 94 30 0-0.0226139-0.0233463
5 95 29 0-0.0224308-0.0233463
6 96 29 0-0.0222477-0.0233463
7 97 29 0-0.0220645-0.0233463
8 99 28 0-0.0218814-0.0233463
9101 28 0-0.0216983-0.0233463
Out[4]:


In [70]:
y = "b"
x = ["x","y"]

train, valid, test = data.split_frame([0.75, 0.15])

from h2o.estimators import H2ODeepLearningEstimator
m = H2ODeepLearningEstimator(model_id="DL_defaults", hidden=[20,20,20,20,20,20,20,20,20,20], activation='tanh',epochs=10000)
m.train(x,y,train)
m


deeplearning Model Build progress: |██████████████████████████████████████| 100%
Model Details
=============
H2ODeepLearningEstimator :  Deep Learning
Model Key:  DL_defaults


ModelMetricsRegression: deeplearning
** Reported on train data. **

MSE: 818.3317409616558
RMSE: 28.606498229627054
MAE: 16.85053289810434
RMSLE: 0.45929856503195216
Mean Residual Deviance: 818.3317409616558
Scoring History: 
timestamp duration training_speed epochs iterations samples training_rmse training_deviance training_mae training_r2
2018-10-23 14:07:20 0.000 sec None 0.0 0 0.0 nan nan nan nan
2018-10-23 14:07:24 4.055 sec 25656 obs/sec 2.0400821 1 100370.0 49.9216761 2492.1737447 33.9943668 0.2032764
2018-10-23 14:07:33 12.633 sec 24348 obs/sec 6.1049818 3 300359.0 48.7674719 2378.2663124 34.4528501 0.2396915
2018-10-23 14:07:41 21.055 sec 24239 obs/sec 10.1750239 5 500601.0 53.7137558 2885.1675592 33.9820562 0.0776401
2018-10-23 14:07:49 29.279 sec 24372 obs/sec 14.2462042 7 700899.0 41.1229139 1691.0940451 25.7103928 0.4593737
--- --- --- --- --- --- --- --- --- --- ---
2018-10-23 14:10:42 3 min 21.381 sec 22560 obs/sec 89.4626517 44 4401473.0 29.5796900 874.9580592 17.8693990 0.7202844
2018-10-23 14:10:51 3 min 30.742 sec 22536 obs/sec 93.5312100 46 4601642.0 30.0171724 901.0306370 17.9926054 0.7119493
2018-10-23 14:11:00 3 min 40.157 sec 22495 obs/sec 97.6032846 48 4801984.0 30.7153207 943.4309268 18.2815089 0.6983943
2018-10-23 14:11:10 3 min 49.618 sec 22450 obs/sec 101.6675136 50 5001940.0 29.3901406 863.7803663 18.3810432 0.7238578
2018-10-23 14:11:10 3 min 49.739 sec 22449 obs/sec 101.6675136 50 5001940.0 28.6064982 818.3317410 16.8505329 0.7383873
See the whole table with table.as_data_frame()
Variable Importances: 
variable relative_importance scaled_importance percentage
y 1.0 1.0 0.5408878
x 0.8488121 0.8488121 0.4591122
Out[70]:

Rendering our results

To see our model's output, we need to feed it the coordinates of the pixels we want rendered


In [86]:
import numpy as np
import pandas as pd
from h2o.frame import H2OFrame
from PIL import Image

IM_SIZE = 256
IM_CHANNELS = 1

def save_pixels(path_to_image_file, image_array, mode):
    im_out = Image.fromarray(image_array, mode )
    im_out.save(path_to_image_file)


# Takes care of clipping, casting to int8, etc.
def save_ndarray(path_to_outfile, x, width = IM_SIZE, height = IM_SIZE, channels = IM_CHANNELS):

    out_arr = np.clip(x, 0, 255)
    
    if channels == 3:
        out_arr = np.reshape(out_arr, (width, height, channels), 1)
    else:
        assert(channels == 1)
        out_arr = np.reshape(out_arr, (width, height), 1)
        
    out_arr = np.rot90(out_arr, k=3)
    out_arr = np.fliplr(out_arr)
    
    if channels == 3:
        save_pixels(path_to_outfile, out_arr.astype(np.int8), 'RGB')
    else:
        save_pixels(path_to_outfile, out_arr.astype(np.int8), 'L')


# Create suitable training matrix
def gen_input_tuples(pixels_width, pixels_height, scale, translate_x, translate_y):

    image_height = pixels_height
    image_width = pixels_width

    # One row per pixel
    X = np.zeros((image_width * image_height, 2))

    # Fill in y values
    X[:,1] = np.repeat(range(0, image_height), image_width, 0)

    # Fill in x values
    X[:,0] = np.tile(range(0, image_width), image_height)

    # Normalize X
    X = X - X.mean()
    X = X / X.var()

    X[:,0] += translate_x
    X[:,1] += translate_y
    
    X = X / scale

    return (X)
    
def render(mdl, image_size, scale, tx, ty, outfile):
    pixel_coords = gen_input_tuples(image_size, image_size, scale, tx, ty)
    df_pixels_to_render = pd.DataFrame({'x':pixel_coords[:,0], 'y':pixel_coords[:,1]})
    h2o_pixels = H2OFrame(df_pixels_to_render)
    pixel_intensities = m.predict(h2o_pixels)
    save_ndarray(outfile, pixel_intensities.as_data_frame().as_matrix(), image_size, image_size, 1)
    
render(m, IM_SIZE, 1, 0, 0, "modelled_tower_bridge.png")


Parse progress: |█████████████████████████████████████████████████████████| 100%
deeplearning prediction progress: |███████████████████████████████████████| 100%
C:\ProgramData\Miniconda3\lib\site-packages\ipykernel_launcher.py:68: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.


In [89]:
render(m, 1024, 1/8, 0, 0, "modelled_tower_bridge_x2.png")


Parse progress: |█████████████████████████████████████████████████████████| 100%
deeplearning prediction progress: |███████████████████████████████████████| 100%
C:\ProgramData\Miniconda3\lib\site-packages\ipykernel_launcher.py:68: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.

In [ ]: