In [1]:
-- deps
require 'nn'
Plot = require 'itorch.Plot'
Faye isn't wonderful at reading common data formats (what do you think Torch is? Python?). I swapped out the CSV files provided in this contest for Faye's favorite file type, T7. If you love your robot, you'll do this for him or her, too. My instructions are here; they're based on a script built by Scott Locklin.
In [2]:
-- load data
file = torch.DiskFile('dat/facies_vectors.t7', 'r')
facies = file:readObject()
file:close()
First, let's build these data into Lua tables.
In [3]:
-- build tables
print("facies size: ", facies:size()[1], "x", facies:size()[2])
-- initialize
training_data = {}
depth = {}
-- build the training wells into the table
training_data["shrimplin"] = facies[{{1,471},{3,9}}]
training_data["alexander"] = facies[{{472,937},{3,9}}]
training_data["shankle"] = facies[{{938,1386},{3,9}}]
training_data["luke"] = facies[{{1387,1847},{3,9}}]
training_data["kimzey"] = facies[{{1848,2286},{3,9}}]
training_data["cross"] = facies[{{2287,2787},{3,9}}]
training_data["nolan"] = facies[{{2788,3202},{3,9}}]
training_data["recruit"] = facies[{{3203,3282},{3,9}}]
training_data["newby"] = facies[{{3283,3745},{3,9}}]
training_data["churchman"] = facies[{{3746,4149},{3,9}}]
-- build a depth log for plotting
depth["shrimplin"] = facies[{{1,471},{2}}]
depth["alexander"] = facies[{{472,937},{2}}]
depth["shankle"] = facies[{{938,1386},{2}}]
depth["luke"] = facies[{{1387,1847},{2}}]
depth["kimzey"] = facies[{{1848,2286},{2}}]
depth["cross"] = facies[{{2287,2787},{2}}]
depth["nolan"] = facies[{{2788,3202},{2}}]
depth["recruit"] = facies[{{3203,3282},{2}}]
depth["newby"] = facies[{{3283,3745},{2}}]
depth["churchman"] = facies[{{3746,4149},{2}}]
Out[3]:
In [4]:
-- normalize the data
-- training data
mean = {}
stdv = {}
for key,value in pairs(training_data) do --over each well
mean[key] = torch.Tensor(7)
stdv[key] = torch.Tensor(7)
for i = 1, 7 do --over each log
mean[key][i] = training_data[key][{{},{i}}]:mean()
training_data[key][{{},{i}}]:add(-mean[key][i])
stdv[key][i] = training_data[key][{{},{i}}]:std()
training_data[key][{{},{i}}]:div(stdv[key][i])
end
end
-- facies labels for training
facies_labels = {}
facies_labels["shrimplin"] = facies[{{1,471},{1}}]
facies_labels["alexander"] = facies[{{472,937},{1}}]
facies_labels["shankle"] = facies[{{938,1386},{1}}]
facies_labels["luke"] = facies[{{1387,1847},{1}}]
facies_labels["kimzey"] = facies[{{1848,2286},{1}}]
facies_labels["cross"] = facies[{{2287,2787},{1}}]
facies_labels["nolan"] = facies[{{2788,3202},{1}}]
facies_labels["recruit"] = facies[{{3203,3282},{1}}]
facies_labels["newby"] = facies[{{3283,3745},{1}}]
facies_labels["churchman"] = facies[{{3746,4149},{1}}]
Just to make sure we're feeding Faye the best data possible, let's plot one well's log suite.
In [5]:
-- plot out a log suite for fun
plot = Plot():line(training_data["luke"][{{},{1}}]:reshape(461), depth["luke"]:reshape(461),'red','gamma'):draw()
plot:line(training_data["luke"][{{},{2}}]:reshape(461)+4, depth["luke"]:reshape(461),'blue','res'):redraw()
plot:line(training_data["luke"][{{},{3}}]:reshape(461)+8, depth["luke"]:reshape(461),'green','por'):redraw()
plot:line(training_data["luke"][{{},{4}}]:reshape(461)+12, depth["luke"]:reshape(461),'brown','neuDens'):redraw()
plot:line(training_data["luke"][{{},{5}}]:reshape(461)+16, depth["luke"]:reshape(461),'grey','photo'):redraw()
plot:title('Luke Logs'):redraw()
plot:yaxis('TVD (m)'):redraw()
plot:legend(true)
plot:redraw()
In [6]:
-- chop out blind well
blind_well = {}
blind_labels = {}
blind_well["newby"] = training_data["newby"][{{},{}}]
training_data["newby"] = nil
blind_labels["newby"] = facies_labels["newby"][{{},{}}]
facies_labels["newby"] = nil
In [7]:
-- build the neural net
net = nil
net = nn.Sequential()
net:add(nn.Linear(7,200))
net:add(nn.ReLU())
net:add(nn.Linear(200,50))
net:add(nn.ReLU())
net:add(nn.Linear(50,9))
net:add(nn.LogSoftMax())
In [8]:
-- test the net -> forward
temp = torch.Tensor(7)
for i = 1,7 do
temp[i] = training_data["shrimplin"][1][i]
end
input = temp
output = net:forward(input)
-- zero gradients and initialize
net:zeroGradParameters()
gradInput = net:backward(input, torch.rand(9))
-- untrained prediction
temp1, temp2 = torch.sort(output, true)
print("predicted facies = ", temp2[1])
print("actual facies = ", facies_labels["shrimplin"][1][1])
Out[8]:
Oh no! She's predicted the wrong facies. Let's not judge to harshly; Faye still needs to go through facies prediction training school.
You may have noticed Faye's output layer isn't a simple transfer function. That's because her training utilizes a negative log likelihood loss function. Keep in mind, if your (multi-class) classification robot has a logrithmic activation function in the output layer, you'll need an exponential function to recover class probabilities.
In [9]:
-- define the loss function
criterion = nn.ClassNLLCriterion()
criterion:forward(output,facies_labels["shrimplin"][1])
gradients = criterion:backward(output, facies_labels["shrimplin"][1])
gradInput = net:backward(input, gradients)
As mentioned above, Faye is quite picky about her data. Though Torch will put Faye through training school automatically, it does requires three things for the robots it trains: 1. the data is input as a torch.DoubleTensor, 2. the data has a size function capable of operating on the entire Lua table, 3. the table has an index function.
In [10]:
-- condition the data
trainset = {}
-- the data
trainset["data"] = torch.Tensor(facies:size()[1]-blind_well["newby"]:size()[1],7)
idx = 0
for key,value in pairs(training_data) do
for i = 1,training_data[key]:size()[1] do
trainset["data"][i + idx] = training_data[key][i]
end
idx = idx + training_data[key]:size()[1]
end
-- the answers
trainset["facies"] = torch.Tensor(facies:size()[1]-blind_labels["newby"]:size()[1])
idx = 0
for key,value in pairs(facies_labels) do
for i = 1, facies_labels[key]:size()[1] do
trainset["facies"][i + idx] = facies_labels[key][i]
end
idx = idx + facies_labels[key]:size()[1]
end
-- write index() and size() functions
setmetatable(trainset,
{__index = function(t, i)
return {t.data[i], t.facies[i]}
end}
);
function trainset:size()
return self.data:size(1)
end
-- condition the testing data
testset = {}
-- the data
testset["data"] = torch.Tensor(blind_well["newby"]:size()[1],7)
for i = 1,blind_well["newby"]:size()[1] do
testset["data"][i] = blind_well["newby"][i]
end
-- the answers
testset["facies"] = torch.Tensor(blind_labels["newby"]:size()[1])
for i = 1, blind_labels["newby"]:size()[1] do
testset["facies"][i] = blind_labels["newby"][i]
end
setmetatable(testset,
{__index = function(t, i)
return {t.data[i], t.facies[i]}
end}
);
function testset:size()
return self.data:size(1)
end
-- eliminate NaNs
nan_mask = trainset.data:ne(trainset.data)
trainset.data[nan_mask] = 0
nan_mask = testset.data:ne(testset.data)
testset.data[nan_mask] = 0
In [11]:
-- train the net
trainer = nn.StochasticGradient(net, criterion)
trainer.learningRate = .001
trainer.maxIteration = 20
print("starting training")
timer = torch.Timer()
trainer:train(trainset)
print("training time =", timer:time().real)
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
Out[11]:
In [12]:
-- overall performance
correct = 0
for i=1,testset:size() do
local groundtruth = testset.facies[i]
local prediction = net:forward(testset.data[i])
local confidences, indices = torch.sort(prediction, true)
if groundtruth == indices[1] then
correct = correct + 1
end
end
print("\ncorrect: ", correct, 100*correct/testset:size() .. ' % \n')
Out[12]:
And now we'll see which facies she classifies accurately, and which ones she struggles with.
In [13]:
-- class performance
counts = {0,0,0,0,0,0,0,0,0}
for i = 1,testset.facies:size()[1] do
temp = testset.facies[i]
counts[temp] = counts[temp] + 1
end
--print(counts)
classes = {'SS', 'CSiS', 'FSiS', 'SiSh', 'MS', 'WS', 'D', 'PS', 'BS'}
class_performance = {0, 0, 0, 0, 0, 0, 0, 0, 0}
for i=1,testset.facies:size()[1] do
local groundtruth = testset.facies[i]
local prediction = net:forward(testset.data[i])
local confidences, indices = torch.sort(prediction, true)
if groundtruth == indices[1] then
class_performance[groundtruth] = class_performance[groundtruth] + 1
end
end
for i = 1, #classes do
print(classes[i], torch.round(100 * class_performance[i] / counts[i]) .. ' %')
end
Out[13]:
In [14]:
preds = torch.Tensor(testset.facies:size()[1])
for i = 1, testset.facies:size()[1] do
local prediction = net:forward(testset.data[i])
confidences, indices = torch.sort(prediction, true)
preds[i] = indices[1]
end
In [15]:
plot = Plot():circle(preds:reshape(463), depth["newby"]:reshape(463),'red','predicted'):draw()
plot:circle(testset.facies:reshape(463), depth["newby"]:reshape(463), 'blue','true')
plot:title('Predictions v. Truths'):redraw()
plot:yaxis('TVD (m)'):redraw()
plot:legend(true)
plot:redraw()
Next we'll train Faye several times, holding a different well blind each iteration and record her accuracy as before.
Since only one or two of you will want to run xval I'm not putting it in this notebook; you can access it in the crossValidation_cpu.lua script in this directory.
Don't tell her I said so, but Faye is pretty basic. She could definitely be optimized by way of complexity. The training data in this case is not perfectly spatially stochastic. Which means that the simple stochastic gradient descent training algorithm I've used isn't the best choice for this classifier. This problem could be somewhat mitigated by weighting the input feature vectors inverse proportionally to the frequency of the facies appearance.
Additionally, using early stopping or a regularization algorithm during Faye's training would certainly improve her accuracy.
Another question to be answered is whether or not the training feature vectors span the testing data field. This question does have an answer in this case, since we have access to the testing data facies labels.
Though I've had fun playing with Faye it's time to move on to another, more sophisticated robot. Stay tuned in to my GitHub page for more artificially intelligent critters.
It's not easy to read other people's code. It's really not easy to read MY buggy code. So thanks to the TLE team for herding me like a cat. Thanks especially to the Hall boys, Matt and Brendon (no relation, I guess) for encouraging me to participate in this wing-ding.
In [ ]: