Behavioral Cloning Project
The goals / steps of this project are the following:
My project includes the following files:
Using the Udacity provided simulator and my drive.py file, the car can be driven autonomously around the track by executing
python drive.py model.h5
The main.py file contains the code for training and saving the convolution neural network. The file shows the pipeline I used for training and validating the model, and it contains comments to explain how the code works.
I used NVIDIA End to End module accroding my understanding. The model croped, resized and normalized the data at begining. and then have 5 convolutional neural networks with 5x5 and 3x3 filter, after the flatten lay and a dropout layer followed 5 fully connected layers.
The model contains dropout layers in order to reduce overfitting (model.py lines 43).
The model was trained and validated on different data sets to ensure that the model was not overfitting (code line main.py 17-22).
And in the generator, each epoch create a set of random image for training, always be different can help the training out of overfit.
The model was tested by running it through the simulator and ensuring that the vehicle could stay on the track.
all model parameter tuning are located in datafield.py.
At begining I use a big set of data(20000+), then 3 epoch are enough to reach nearly overfiting. Afterwards I use a small random dataset(3000+), it always have potential to reduce the loss, so I take 10~20 epoch.
I was tried to use my own data, which collecting by keyboard control method. The resluts are not good enough. So after it in this task, I use Udacity data
The overall strategy for deriving a model architecture was to use the Nivida end to end architecture.
My first step was to use a convolution neural network model similar to the Nivida's papier I thought this model might be appropriate because it was proved by others.
In order to gauge how well the model was working, I split my image and steering angle data into a training and validation set. I found that my first model had a low mean squared error on the training set but a high mean squared error on the validation set. This implied that the model was overfitting.
To combat the overfitting, I modified the model with drop out layer so that result seems better.
Then I try to do next step was to run the simulator to see how well the car was driving around track one. There were a few spots where the vehicle fell off the track. to improve the driving behavior in these cases, I used a lambda layer, which calling a resize and normalize function.
At the end of the process, the vehicle is able to drive autonomously around the track without leaving the road.
The final model architecture (model.py lines 14-50) consisted of a convolution neural network with the following layers and layer sizes. It is builded according nivida end2end Architecture.
model.add(Cropping2D(cropping=((df.cropTop, df.cropBottom), (0, 0)), input_shape=df.ImgShape, name='input'))
model.add(Lambda(resize))
model.add(Lambda(normalize))
# In: 64x64
model.add(Conv2D(24,5,5,subsample=(2, 2),activation = 'elu') )
model.add(Conv2D(36,5,5,subsample=(2, 2),activation = 'elu'))
model.add(Conv2D(48,5,5,subsample=(2, 2),activation = 'elu'))
model.add(Conv2D(64,3,3,activation = 'elu'))
model.add(Conv2D(64,3,3,activation = 'elu'))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(1164))
model.add(Dense(100))
model.add(Dense(50))
model.add(Dense(10))
model.add(Dense(1))
For the dataset, I used Udacity data as the bais. this dataset contained a mount of steering Zero Frame. So in the getTrainDate function, I dropout a part of this kind Frame.
and then to augment the data set, the following techincal are used:
Here is an example image of center lane driving:
I also used the vehicle recovering from the left side and right sides of the road. with these two different camera, I can have more data for model training.
There are three image come from different camera, I randomly take one of it, if it from left will adjust with a positv offset. If from right, then give a negative offset(in this task I use 0.25).
In [48]:
import numpy as np
import cv2
import utils
import datafield as df
import matplotlib.pyplot as plt
image = cv2.imread("./pic/center.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
plt.subplots(figsize=(10, 20))
plt.subplot(4,1,1)
plt.imshow(image)
plt.title('Original')
for i in range(1, 4):
plt.subplot(4,1,i+1)
plt.imshow(utils.random_bright(image))
plt.title('Brightness {}'.format(i))
plt.show()
as a reference, I readed the Blog of Yadav, I find the idea of translate could be useful, then I implement a simliar solution for this point, and cenificante improve the behavior of the driving.
In [72]:
image = cv2.imread("./pic/center.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
angle =0.4
plt.subplots(figsize=(10, 20))
plt.subplot(4,1,1)
plt.imshow(image)
plt.title('Original and angle is {}'.format(angle))
for i in range(1, 3):
plt.subplot(4,1,i+1)
image_tr, angle_tr = utils.random_translate(image,angle)
plt.imshow(image_tr)
plt.title('translate {} and angle is {}'.format(i,angle_tr))
plt.show()
In [73]:
image = cv2.imread("./pic/center.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
angle =0.4
plt.subplots(figsize=(10, 20))
plt.subplot(4,1,1)
plt.imshow(image)
plt.title('Original and angle is {}'.format(angle))
for i in range(1, 2):
plt.subplot(4,1,i+1)
image_tr, angle_tr = utils.random_flip(image,angle)
plt.imshow(image_tr)
plt.title('translate {} and angle is {}'.format(i,angle_tr))
plt.show()
I finally randomly shuffled the data set. for the validation set (20%) is already split at begining. I used an adam optimizer so that manually training the learning rate wasn't necessary.