Facies classification using Convolutional Neural Networks

Team StoDIG - Statoil Deep-learning Interest Group

David Wade, John Thurmond & Eskil Kulseth Dahl

In this python notebook we propose a facies classification model, building on the simple Neural Network solution proposed by LA_Team in order to outperform the prediction model proposed in the predicting facies from well logs challenge.

Given the limited size of the training data set, Deep Learning is not likely to exceed the accuracy of results from refined Machine Learning techniques (such as Gradient Boosted Trees). However, we chose to use the opportunity to advance our understanding of Deep Learning network design, and have enjoyed participating in the contest. With a substantially larger training set and perhaps more facies ambiguity, Deep Learning could be a preferred approach to this sort of problem.

We use three key innovations:

  • Inserting a convolutional layer as the first layer in the Neural Network
  • Initializing the weights of this layer to detect gradients and extrema
  • Adding Dropout regularization to prevent overfitting

Problem Modeling


The dataset we will use comes from a class excercise from The University of Kansas on Neural Networks and Fuzzy Systems. This exercise is based on a consortium project to use machine learning techniques to create a reservoir model of the largest gas fields in North America, the Hugoton and Panoma Fields. For more info on the origin of the data, see Bohling and Dubois (2003) and Dubois et al. (2007).

The dataset we will use is log data from nine wells that have been labeled with a facies type based on oberservation of core. We will use this log data to train a classifier to predict facies types.

This data is from the Council Grove gas reservoir in Southwest Kansas. The Panoma Council Grove Field is predominantly a carbonate gas reservoir encompassing 2700 square miles in Southwestern Kansas. This dataset is from nine wells (with 4149 examples), consisting of a set of seven predictor variables and a rock facies (class) for each example vector and validation (test) data (830 examples from two wells) having the same seven predictor variables in the feature vector. Facies are based on examination of cores from nine wells taken vertically at half-foot intervals. Predictor variables include five from wireline log measurements and two geologic constraining variables that are derived from geologic knowledge. These are essentially continuous variables sampled at a half-foot sample rate.

The seven predictor variables are:

The nine discrete facies (classes of rocks) are:

  1. Nonmarine sandstone
  2. Nonmarine coarse siltstone
  3. Nonmarine fine siltstone
  4. Marine siltstone and shale
  5. Mudstone (limestone)
  6. Wackestone (limestone)
  7. Dolomite
  8. Packstone-grainstone (limestone)
  9. Phylloid-algal bafflestone (limestone)

These facies aren't discrete, and gradually blend into one another. Some have neighboring facies that are rather close. Mislabeling within these neighboring facies can be expected to occur. The following table lists the facies, their abbreviated labels and their approximate neighbors.

Facies Label Adjacent Facies
1 SS 2
2 CSiS 1,3
3 FSiS 2
4 SiSh 5
5 MS 4,6
6 WS 5,7
7 D 6,8
8 PS 6,7,9
9 BS 7,8

Setup


Check we have all the libraries we need, and import the modules we require. Note that we have used the Theano backend for Keras, and to achieve a reasonable training time we have used an NVidia K20 GPU.


In [1]:
%%sh
pip install pandas
pip install scikit-learn
pip install keras
pip install sklearn


Requirement already satisfied: pandas in /home/dawad/anaconda3/lib/python3.5/site-packages
Requirement already satisfied: python-dateutil>=2 in /home/dawad/anaconda3/lib/python3.5/site-packages (from pandas)
Requirement already satisfied: pytz>=2011k in /home/dawad/anaconda3/lib/python3.5/site-packages (from pandas)
Requirement already satisfied: numpy>=1.7.0 in /home/dawad/anaconda3/lib/python3.5/site-packages (from pandas)
Requirement already satisfied: six>=1.5 in /home/dawad/anaconda3/lib/python3.5/site-packages (from python-dateutil>=2->pandas)
Requirement already satisfied: scikit-learn in /home/dawad/anaconda3/lib/python3.5/site-packages
Requirement already satisfied: keras in /home/dawad/anaconda3/lib/python3.5/site-packages
Requirement already satisfied: six in /home/dawad/anaconda3/lib/python3.5/site-packages (from keras)
Requirement already satisfied: theano in /home/dawad/anaconda3/lib/python3.5/site-packages (from keras)
Requirement already satisfied: pyyaml in /home/dawad/anaconda3/lib/python3.5/site-packages (from keras)
Requirement already satisfied: scipy>=0.11 in /home/dawad/anaconda3/lib/python3.5/site-packages (from theano->keras)
Requirement already satisfied: numpy>=1.7.1 in /home/dawad/anaconda3/lib/python3.5/site-packages (from theano->keras)
Requirement already satisfied: sklearn in /home/dawad/anaconda3/lib/python3.5/site-packages
Requirement already satisfied: scikit-learn in /home/dawad/anaconda3/lib/python3.5/site-packages (from sklearn)

In [2]:
from __future__ import print_function
import time
import numpy as np
%matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.colors as colors
from mpl_toolkits.axes_grid1 import make_axes_locatable
from keras.preprocessing import sequence
from keras.models import Model, Sequential
from keras.constraints import maxnorm, nonneg
from keras.optimizers import SGD, Adam, Adamax, Nadam
from keras.regularizers import l2, activity_l2
from keras.layers import Input, Dense, Dropout, Activation, Convolution1D, Cropping1D, Cropping2D, Permute, Flatten, MaxPooling1D, merge
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold , StratifiedKFold
from classification_utilities import display_cm, display_adj_cm
from sklearn.metrics import confusion_matrix, f1_score
from sklearn import preprocessing
from sklearn.model_selection import GridSearchCV


Using Theano backend.
Using gpu device 0: Tesla K20c (CNMeM is enabled with initial size: 80.0% of memory, cuDNN 5105)
/home/dawad/anaconda3/lib/python3.5/site-packages/theano/sandbox/cuda/__init__.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
  warnings.warn(warn)

Data ingest


We load the training and testing data to preprocess it for further analysis, filling the missing data values in the PE field with zero and proceeding to normalize the data that will be fed into our model. We now incorporate the Imputation from Paolo Bestagini via LA_Team's Submission 5.


In [3]:
filename = 'train_test_data.csv'
data = pd.read_csv(filename)
data.head(12)

# Set 'Well Name' and 'Formation' fields as categories
data['Well Name'] = data['Well Name'].astype('category')
data['Formation'] = data['Formation'].astype('category')

# Normalize the rest of fields (GR, ILD_log10, DelthaPHI, PHIND,NM_M,RELPOS)
correct_facies_labels = data['Facies'].values
feature_vectors = data.drop(['Formation'], axis=1)
well_labels = data[['Well Name', 'Facies']].values
data_vectors = feature_vectors.drop(['Well Name', 'Facies'], axis=1).values

# Fill missing values and normalize for 'PE' field
imp = preprocessing.Imputer(missing_values='NaN', strategy='mean', axis=0)
imp.fit(data_vectors)
data_vectors = imp.transform(data_vectors)

scaler = preprocessing.StandardScaler().fit(data_vectors)
scaled_features = scaler.transform(data_vectors)
data_out = np.hstack([well_labels, scaled_features])

Split data into training data and blind data, and output as Numpy arrays


In [4]:
def preprocess(data_out):
    
    data = data_out
    well_data = {}
    well_names = ['SHRIMPLIN', 'ALEXANDER D', 'SHANKLE', 'LUKE G U', 'KIMZEY A', 'CROSS H CATTLE', 
                  'NOLAN', 'Recruit F9', 'NEWBY', 'CHURCHMAN BIBLE', 'STUART', 'CRAWFORD']
    for name in well_names:
        well_data[name] = [[], []]

    for row in data:
        well_data[row[0]][1].append(row[1])
        well_data[row[0]][0].append(list(row[2::]))

    chunks = []
    chunks_test = []
    chunk_length = 1 
    chunks_facies = []
    wellID=0.0
    for name in well_names:
        
        if name not in ['STUART', 'CRAWFORD']:
            test_well_data = well_data[name]
            log_values = np.array(test_well_data[0])
            facies_values =  np.array(test_well_data[1])
            for i in range(log_values.shape[0]):
                toAppend = np.concatenate((log_values[i:i+1, :], np.asarray(wellID).reshape(1,1)), axis=1)
                chunks.append(toAppend)
                chunks_facies.append(facies_values[i])
        else:
            test_well_data = well_data[name]
            log_values = np.array(test_well_data[0])
            for i in range(log_values.shape[0]):
                toAppend = np.concatenate((log_values[i:i+1, :], np.asarray(wellID).reshape(1,1)), axis=1)
                chunks_test.append(toAppend)
        
        wellID = wellID + 1.0
    
    chunks_facies = np.array(chunks_facies, dtype=np.int32)-1
    X_ = np.array(chunks)
    X = np.zeros((len(X_),len(X_[0][0]) * len(X_[0])))
    for i in range(len(X_)):
        X[i,:] = X_[i].flatten()
        
    X_test = np.array(chunks_test)
    X_test_out = np.zeros((len(X_test),len(X_test[0][0]) * len(X_test[0])))
    for i in range(len(X_test)):
        X_test_out[i,:] = X_test[i].flatten()
    y = np_utils.to_categorical(chunks_facies)
    return X, y, X_test_out

X_train_in, y_train, X_test_in = preprocess(data_out)

Data Augmentation


We expand the input data to be acted on by the convolutional layer.


In [5]:
conv_domain = 11

# Reproducibility
np.random.seed(7) 
# Load data

def expand_dims(input):
    r = int((conv_domain-1)/2)
    l = input.shape[0]
    n_input_vars = input.shape[1]
    output = np.zeros((l, conv_domain, n_input_vars))
    for i in range(l):
        for j in range(conv_domain):
            for k in range(n_input_vars):
                output[i,j,k] = input[min(i+j-r,l-1),k]
    return output

X_train = np.empty((0,conv_domain,8), dtype=float)
X_test  = np.empty((0,conv_domain,8), dtype=float)

wellId = 0.0
for i in range(10):
    X_train_subset = X_train_in[X_train_in[:, 8] == wellId][:,0:8]
    X_train_subset = expand_dims(X_train_subset)
    X_train = np.concatenate((X_train,X_train_subset),axis=0)
    wellId = wellId + 1.0
    
for i in range(2):
    X_test_subset = X_test_in[X_test_in[:, 8] == wellId][:,0:8]
    X_test_subset = expand_dims(X_test_subset)
    X_test = np.concatenate((X_test,X_test_subset),axis=0)
    wellId = wellId + 1.0
    
print(X_train.shape)
print(X_test.shape)

# Obtain labels
y_labels = np.zeros((len(y_train),1))
for i in range(len(y_train)):
    y_labels[i] = np.argmax(y_train[i])
y_labels = y_labels.astype(int)


(4149, 11, 8)
(830, 11, 8)

Convolutional Neural Network

We build a CNN with the following layers (no longer using Sequential() model):

  • Dropout layer on input
  • One 1D convolutional layer (7-point radius)
  • One 1D cropping layer (just take actual log-value of interest)
  • Series of Merge layers re-adding result of cropping layer plus Dropout & Fully-Connected layers

Instead of running CNN with gradient features added, we initialize the Convolutional layer weights to achieve this

  • This allows the CNN to reject them, adjust them or turn them into something else if required

In [6]:
# Set parameters
input_dim = 8
output_dim = 9
n_per_batch = 128
epochs = 100
crop_factor = int(conv_domain/2)
filters_per_log = 11
n_convolutions = input_dim*filters_per_log

starting_weights = [np.zeros((conv_domain, 1, input_dim, n_convolutions)), np.ones((n_convolutions))]

norm_factor=float(conv_domain)*2.0


for i in range(input_dim):
    for j in range(conv_domain):
        starting_weights[0][j, 0, i, i*filters_per_log+0]  = j/norm_factor
        starting_weights[0][j, 0, i, i*filters_per_log+1]  = j/norm_factor
        starting_weights[0][j, 0, i, i*filters_per_log+2]  = (conv_domain-j)/norm_factor
        starting_weights[0][j, 0, i, i*filters_per_log+3]  = (conv_domain-j)/norm_factor
        starting_weights[0][j, 0, i, i*filters_per_log+4]  = (2*abs(crop_factor-j))/norm_factor
        starting_weights[0][j, 0, i, i*filters_per_log+5]  = (conv_domain-2*abs(crop_factor-j))/norm_factor
        starting_weights[0][j, 0, i, i*filters_per_log+6]  = 0.25
        starting_weights[0][j, 0, i, i*filters_per_log+7]  = 0.5  if (j%2 == 0) else 0.25
        starting_weights[0][j, 0, i, i*filters_per_log+8]  = 0.25 if (j%2 == 0) else 0.5
        starting_weights[0][j, 0, i, i*filters_per_log+9]  = 0.5  if (j%4 == 0) else 0.25
        starting_weights[0][j, 0, i, i*filters_per_log+10] = 0.25 if (j%4 == 0) else 0.5

def dnn_model(init_dropout_rate=0.2, main_dropout_rate=0.5,
              hidden_dim_1=32, hidden_dim_2=32, 
              max_norm=10, nb_conv=n_convolutions):
    # Define the model
    inputs = Input(shape=(conv_domain,input_dim,))
    inputs_dropout = Dropout(init_dropout_rate)(inputs)

    x1 = Convolution1D(nb_conv, conv_domain, border_mode='valid', weights=starting_weights, activation='tanh', input_shape=(conv_domain,input_dim), input_length=input_dim, W_constraint=nonneg())(inputs_dropout)
    x1 = Flatten()(x1)   

    xn = Cropping1D(cropping=(crop_factor,crop_factor))(inputs_dropout)
    xn = Flatten()(xn)

    xA = merge([x1, xn], mode='concat')    
    xA = Dropout(main_dropout_rate)(xA)
    xA = Dense(hidden_dim_1, init='uniform', activation='relu', W_constraint=maxnorm(max_norm))(xA)
    
    x = merge([xA, xn], mode='concat')    
    x = Dropout(main_dropout_rate)(x)
    x = Dense(hidden_dim_2, init='uniform', activation='relu', W_constraint=maxnorm(max_norm))(x)
    
    predictions = Dense(output_dim, init='uniform', activation='softmax')(x)
    
    model = Model(input=inputs, output=predictions)
    
    optimizerNadam = Nadam(lr=0.002, beta_1=0.9, beta_2=0.999, epsilon=1e-08, schedule_decay=0.004)
    model.compile(loss='categorical_crossentropy', optimizer=optimizerNadam, metrics=['accuracy'])
    return model

# Load the model
t0 = time.time()
model_dnn = dnn_model()
model_dnn.summary()
t1 = time.time()
print("Load time = %d" % (t1-t0) )

def plot_weights():
    layerID=2

    print(model_dnn.layers[layerID].get_weights()[0].shape)
    print(model_dnn.layers[layerID].get_weights()[1].shape)

    fig, ax = plt.subplots(figsize=(12,10))

    for i in range(8):
        plt.subplot(911+i)
        plt.imshow(model_dnn.layers[layerID].get_weights()[0][:,0,i,:], interpolation='none')

    fig, ax = plt.subplots(figsize=(11,9))

    plt.subplot(919)
    plt.imshow(model_dnn.layers[layerID].get_weights()[1].reshape(1,n_convolutions), interpolation='none')

    plt.show()
    
plot_weights()


____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
input_1 (InputLayer)             (None, 11, 8)         0                                            
____________________________________________________________________________________________________
dropout_1 (Dropout)              (None, 11, 8)         0           input_1[0][0]                    
____________________________________________________________________________________________________
convolution1d_1 (Convolution1D)  (None, 1, 88)         7832        dropout_1[0][0]                  
____________________________________________________________________________________________________
cropping1d_1 (Cropping1D)        (None, 1, 8)          0           dropout_1[0][0]                  
____________________________________________________________________________________________________
flatten_1 (Flatten)              (None, 88)            0           convolution1d_1[0][0]            
____________________________________________________________________________________________________
flatten_2 (Flatten)              (None, 8)             0           cropping1d_1[0][0]               
____________________________________________________________________________________________________
merge_1 (Merge)                  (None, 96)            0           flatten_1[0][0]                  
                                                                   flatten_2[0][0]                  
____________________________________________________________________________________________________
dropout_2 (Dropout)              (None, 96)            0           merge_1[0][0]                    
____________________________________________________________________________________________________
dense_1 (Dense)                  (None, 32)            3104        dropout_2[0][0]                  
____________________________________________________________________________________________________
merge_2 (Merge)                  (None, 40)            0           dense_1[0][0]                    
                                                                   flatten_2[0][0]                  
____________________________________________________________________________________________________
dropout_3 (Dropout)              (None, 40)            0           merge_2[0][0]                    
____________________________________________________________________________________________________
dense_2 (Dense)                  (None, 32)            1312        dropout_3[0][0]                  
____________________________________________________________________________________________________
dense_3 (Dense)                  (None, 9)             297         dense_2[0][0]                    
====================================================================================================
Total params: 12,545
Trainable params: 12,545
Non-trainable params: 0
____________________________________________________________________________________________________
Load time = 0
(11, 1, 8, 88)
(88,)

We train the CNN and evaluate it on precision/recall.


In [7]:
#Train model
t0 = time.time()
model_dnn.fit(X_train, y_train, batch_size=n_per_batch, nb_epoch=epochs, verbose=2)
t1 = time.time()
print("Train time = %d seconds" % (t1-t0) )


# Predict Values on Training set
t0 = time.time()
y_predicted = model_dnn.predict( X_train , batch_size=n_per_batch, verbose=2)
t1 = time.time()
print("Test time = %d seconds" % (t1-t0) )

# Print Report

# Format output [0 - 8 ]
y_ = np.zeros((len(y_train),1))
for i in range(len(y_train)):
    y_[i] = np.argmax(y_train[i])

y_predicted_ = np.zeros((len(y_predicted), 1))
for i in range(len(y_predicted)):
    y_predicted_[i] = np.argmax( y_predicted[i] )
    
# Confusion Matrix
conf = confusion_matrix(y_, y_predicted_)

def accuracy(conf):
    total_correct = 0.
    nb_classes = conf.shape[0]
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
    acc = total_correct/sum(sum(conf))
    return acc

adjacent_facies = np.array([[1], [0,2], [1], [4], [3,5], [4,6,7], [5,7], [5,6,8], [6,7]])
facies_labels = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS','WS', 'D','PS', 'BS']

def accuracy_adjacent(conf, adjacent_facies):
    nb_classes = conf.shape[0]
    total_correct = 0.
    for i in np.arange(0,nb_classes):
        total_correct += conf[i][i]
        for j in adjacent_facies[i]:
            total_correct += conf[i][j]
    return total_correct / sum(sum(conf))

# Print Results
print ("\nModel Report")
print ("-Accuracy: %.6f" % ( accuracy(conf) ))
print ("-Adjacent Accuracy: %.6f" % ( accuracy_adjacent(conf, adjacent_facies) ))
print ("\nConfusion Matrix")
display_cm(conf, facies_labels, display_metrics=True, hide_zeros=True)


Epoch 1/100
0s - loss: 1.8302 - acc: 0.3651
Epoch 2/100
0s - loss: 1.3809 - acc: 0.4146
Epoch 3/100
0s - loss: 1.2878 - acc: 0.4531
Epoch 4/100
0s - loss: 1.2265 - acc: 0.4743
Epoch 5/100
0s - loss: 1.2020 - acc: 0.4919
Epoch 6/100
0s - loss: 1.1843 - acc: 0.5066
Epoch 7/100
0s - loss: 1.1738 - acc: 0.5155
Epoch 8/100
0s - loss: 1.1379 - acc: 0.5331
Epoch 9/100
0s - loss: 1.1397 - acc: 0.5300
Epoch 10/100
0s - loss: 1.1423 - acc: 0.5305
Epoch 11/100
0s - loss: 1.1255 - acc: 0.5276
Epoch 12/100
0s - loss: 1.1289 - acc: 0.5283
Epoch 13/100
0s - loss: 1.1171 - acc: 0.5358
Epoch 14/100
0s - loss: 1.1199 - acc: 0.5382
Epoch 15/100
0s - loss: 1.1125 - acc: 0.5430
Epoch 16/100
0s - loss: 1.0953 - acc: 0.5418
Epoch 17/100
0s - loss: 1.0946 - acc: 0.5527
Epoch 18/100
0s - loss: 1.0941 - acc: 0.5534
Epoch 19/100
0s - loss: 1.0874 - acc: 0.5515
Epoch 20/100
0s - loss: 1.0887 - acc: 0.5452
Epoch 21/100
0s - loss: 1.0812 - acc: 0.5565
Epoch 22/100
0s - loss: 1.0856 - acc: 0.5519
Epoch 23/100
0s - loss: 1.0789 - acc: 0.5584
Epoch 24/100
0s - loss: 1.0749 - acc: 0.5495
Epoch 25/100
0s - loss: 1.0704 - acc: 0.5488
Epoch 26/100
0s - loss: 1.0738 - acc: 0.5483
Epoch 27/100
0s - loss: 1.0527 - acc: 0.5678
Epoch 28/100
0s - loss: 1.0622 - acc: 0.5611
Epoch 29/100
0s - loss: 1.0599 - acc: 0.5659
Epoch 30/100
0s - loss: 1.0340 - acc: 0.5719
Epoch 31/100
0s - loss: 1.0519 - acc: 0.5628
Epoch 32/100
0s - loss: 1.0615 - acc: 0.5563
Epoch 33/100
0s - loss: 1.0484 - acc: 0.5688
Epoch 34/100
0s - loss: 1.0591 - acc: 0.5519
Epoch 35/100
0s - loss: 1.0465 - acc: 0.5570
Epoch 36/100
0s - loss: 1.0403 - acc: 0.5705
Epoch 37/100
0s - loss: 1.0371 - acc: 0.5635
Epoch 38/100
0s - loss: 1.0347 - acc: 0.5698
Epoch 39/100
0s - loss: 1.0276 - acc: 0.5780
Epoch 40/100
0s - loss: 1.0227 - acc: 0.5828
Epoch 41/100
0s - loss: 1.0419 - acc: 0.5703
Epoch 42/100
0s - loss: 1.0355 - acc: 0.5669
Epoch 43/100
0s - loss: 1.0273 - acc: 0.5645
Epoch 44/100
0s - loss: 1.0237 - acc: 0.5662
Epoch 45/100
0s - loss: 1.0351 - acc: 0.5606
Epoch 46/100
0s - loss: 1.0223 - acc: 0.5654
Epoch 47/100
0s - loss: 1.0264 - acc: 0.5722
Epoch 48/100
0s - loss: 1.0194 - acc: 0.5748
Epoch 49/100
0s - loss: 1.0185 - acc: 0.5736
Epoch 50/100
0s - loss: 1.0279 - acc: 0.5719
Epoch 51/100
0s - loss: 1.0189 - acc: 0.5794
Epoch 52/100
0s - loss: 1.0036 - acc: 0.5715
Epoch 53/100
0s - loss: 1.0167 - acc: 0.5715
Epoch 54/100
0s - loss: 1.0069 - acc: 0.5879
Epoch 55/100
0s - loss: 1.0121 - acc: 0.5693
Epoch 56/100
0s - loss: 1.0185 - acc: 0.5804
Epoch 57/100
0s - loss: 1.0062 - acc: 0.5780
Epoch 58/100
0s - loss: 1.0045 - acc: 0.5801
Epoch 59/100
0s - loss: 1.0052 - acc: 0.5835
Epoch 60/100
0s - loss: 1.0186 - acc: 0.5828
Epoch 61/100
0s - loss: 1.0187 - acc: 0.5707
Epoch 62/100
0s - loss: 1.0072 - acc: 0.5830
Epoch 63/100
0s - loss: 1.0161 - acc: 0.5840
Epoch 64/100
0s - loss: 1.0179 - acc: 0.5830
Epoch 65/100
0s - loss: 1.0044 - acc: 0.5862
Epoch 66/100
0s - loss: 1.0047 - acc: 0.5801
Epoch 67/100
0s - loss: 1.0000 - acc: 0.5826
Epoch 68/100
0s - loss: 0.9966 - acc: 0.5854
Epoch 69/100
0s - loss: 1.0028 - acc: 0.5780
Epoch 70/100
0s - loss: 0.9994 - acc: 0.5780
Epoch 71/100
0s - loss: 1.0076 - acc: 0.5782
Epoch 72/100
0s - loss: 1.0083 - acc: 0.5770
Epoch 73/100
0s - loss: 1.0047 - acc: 0.5765
Epoch 74/100
0s - loss: 0.9740 - acc: 0.5936
Epoch 75/100
0s - loss: 0.9927 - acc: 0.5883
Epoch 76/100
0s - loss: 1.0058 - acc: 0.5965
Epoch 77/100
0s - loss: 0.9883 - acc: 0.5866
Epoch 78/100
0s - loss: 0.9831 - acc: 0.5835
Epoch 79/100
0s - loss: 1.0019 - acc: 0.5852
Epoch 80/100
0s - loss: 1.0002 - acc: 0.5847
Epoch 81/100
0s - loss: 0.9884 - acc: 0.5919
Epoch 82/100
0s - loss: 0.9883 - acc: 0.5871
Epoch 83/100
0s - loss: 0.9939 - acc: 0.5869
Epoch 84/100
0s - loss: 0.9766 - acc: 0.5973
Epoch 85/100
0s - loss: 0.9854 - acc: 0.5929
Epoch 86/100
0s - loss: 0.9909 - acc: 0.5963
Epoch 87/100
0s - loss: 0.9940 - acc: 0.5869
Epoch 88/100
0s - loss: 0.9898 - acc: 0.5813
Epoch 89/100
0s - loss: 0.9868 - acc: 0.5862
Epoch 90/100
0s - loss: 0.9769 - acc: 0.5900
Epoch 91/100
0s - loss: 0.9889 - acc: 0.5893
Epoch 92/100
0s - loss: 0.9808 - acc: 0.5866
Epoch 93/100
0s - loss: 0.9811 - acc: 0.5912
Epoch 94/100
0s - loss: 0.9860 - acc: 0.5922
Epoch 95/100
0s - loss: 0.9819 - acc: 0.5951
Epoch 96/100
0s - loss: 0.9776 - acc: 0.5903
Epoch 97/100
0s - loss: 0.9940 - acc: 0.5835
Epoch 98/100
0s - loss: 0.9650 - acc: 0.5944
Epoch 99/100
0s - loss: 0.9589 - acc: 0.5956
Epoch 100/100
0s - loss: 0.9823 - acc: 0.5811
Train time = 28 seconds
Test time = 1 seconds

Model Report
-Accuracy: 0.672933
-Adjacent Accuracy: 0.935406

Confusion Matrix
     Pred    SS  CSiS  FSiS  SiSh    MS    WS     D    PS    BS Total
     True
       SS   206    53     9                                       268
     CSiS    86   581   273                                       940
     FSiS     4   129   639     1           1           6         780
     SiSh           3     3   213          50           2         271
       MS           5     9    55    21   145     2    59         296
       WS                 2    49     8   413    11    93     6   582
        D                 1     7     7     4    97    25         141
       PS           1     9    14     5   150     3   478    26   686
       BS           2     1     1     1     8          28   144   185

Precision  0.70  0.75  0.68  0.63  0.50  0.54  0.86  0.69  0.82  0.67
   Recall  0.77  0.62  0.82  0.79  0.07  0.71  0.69  0.70  0.78  0.67
       F1  0.73  0.68  0.74  0.70  0.12  0.61  0.76  0.69  0.80  0.66

We display the learned 1D convolution kernels


In [8]:
plot_weights()


(11, 1, 8, 88)
(88,)

In order to avoid overfitting, we evaluate our model by running a 5-fold stratified cross-validation routine.


In [9]:
# Cross Validation
def cross_validate():
    t0 = time.time()
    estimator = KerasClassifier(build_fn=dnn_model, nb_epoch=epochs, batch_size=n_per_batch, verbose=0)
    skf = StratifiedKFold(n_splits=5, shuffle=True)
    results_dnn = cross_val_score(estimator, X_train, y_train, cv= skf.get_n_splits(X_train, y_train))
    t1 = time.time()
    print("Cross Validation time = %d" % (t1-t0) )
    print(' Cross Validation Results')
    print( results_dnn )
    print(np.mean(results_dnn))

cross_validate()


Cross Validation time = 106
 Cross Validation Results
[ 0.5626506   0.52650602  0.53373494  0.50361446  0.49577805]
0.524456814166

Prediction


To predict the STUART and CRAWFORD blind wells we do the following:

Set up a plotting function to display the logs & facies.


In [10]:
# 1=sandstone  2=c_siltstone   3=f_siltstone 
# 4=marine_silt_shale 5=mudstone 6=wackestone 7=dolomite
# 8=packstone 9=bafflestone
facies_colors = ['#F4D03F', '#F5B041','#DC7633','#6E2C00', '#1B4F72','#2E86C1', '#AED6F1', '#A569BD', '#196F3D']

#facies_color_map is a dictionary that maps facies labels
#to their respective colors
facies_color_map = {}
for ind, label in enumerate(facies_labels):
    facies_color_map[label] = facies_colors[ind]

def label_facies(row, labels):
    return labels[ row['Facies'] -1]

def make_facies_log_plot(logs, facies_colors):
    #make sure logs are sorted by depth
    logs = logs.sort_values(by='Depth')
    cmap_facies = colors.ListedColormap(
            facies_colors[0:len(facies_colors)], 'indexed')
    
    ztop=logs.Depth.min(); zbot=logs.Depth.max()
    
    cluster=np.repeat(np.expand_dims(logs['Facies'].values,1), 100, 1)
    
    f, ax = plt.subplots(nrows=1, ncols=6, figsize=(8, 12))
    ax[0].plot(logs.GR, logs.Depth, '-g')
    ax[1].plot(logs.ILD_log10, logs.Depth, '-')
    ax[2].plot(logs.DeltaPHI, logs.Depth, '-', color='0.5')
    ax[3].plot(logs.PHIND, logs.Depth, '-', color='r')
    ax[4].plot(logs.PE, logs.Depth, '-', color='black')
    im=ax[5].imshow(cluster, interpolation='none', aspect='auto',
                    cmap=cmap_facies,vmin=1,vmax=9)
    
    divider = make_axes_locatable(ax[5])
    cax = divider.append_axes("right", size="20%", pad=0.05)
    cbar=plt.colorbar(im, cax=cax)
    cbar.set_label((17*' ').join([' SS ', 'CSiS', 'FSiS', 
                                'SiSh', ' MS ', ' WS ', ' D  ', 
                                ' PS ', ' BS ']))
    cbar.set_ticks(range(0,1)); cbar.set_ticklabels('')
    
    for i in range(len(ax)-1):
        ax[i].set_ylim(ztop,zbot)
        ax[i].invert_yaxis()
        ax[i].grid()
        ax[i].locator_params(axis='x', nbins=3)
    
    ax[0].set_xlabel("GR")
    ax[0].set_xlim(logs.GR.min(),logs.GR.max())
    ax[1].set_xlabel("ILD_log10")
    ax[1].set_xlim(logs.ILD_log10.min(),logs.ILD_log10.max())
    ax[2].set_xlabel("DeltaPHI")
    ax[2].set_xlim(logs.DeltaPHI.min(),logs.DeltaPHI.max())
    ax[3].set_xlabel("PHIND")
    ax[3].set_xlim(logs.PHIND.min(),logs.PHIND.max())
    ax[4].set_xlabel("PE")
    ax[4].set_xlim(logs.PE.min(),logs.PE.max())
    ax[5].set_xlabel('Facies')
    
    ax[1].set_yticklabels([]); ax[2].set_yticklabels([]); ax[3].set_yticklabels([])
    ax[4].set_yticklabels([]); ax[5].set_yticklabels([])
    ax[5].set_xticklabels([])
    f.suptitle('Well: %s'%logs.iloc[0]['Well Name'], fontsize=14,y=0.94)

Run the model on the blind data

  • Output a CSV
  • Plot the wells in the notebook

In [11]:
# DNN model Prediction
y_test = model_dnn.predict( X_test , batch_size=n_per_batch, verbose=0)
predictions_dnn = np.zeros((len(y_test),1))
for i in range(len(y_test)):
    predictions_dnn[i] = np.argmax(y_test[i]) + 1 
predictions_dnn = predictions_dnn.astype(int)
# Store results
test_data = pd.read_csv('../validation_data_nofacies.csv')
test_data['Facies'] = predictions_dnn
test_data.to_csv('Prediction_StoDIG_2.csv')

make_facies_log_plot(
    test_data[test_data['Well Name'] == 'STUART'],
    facies_colors=facies_colors)

make_facies_log_plot(
    test_data[test_data['Well Name'] == 'CRAWFORD'],
    facies_colors=facies_colors)