The goal of this project is to analyse brain MRI images in order to diagnose Alzheimer's disease.
To do so, a dataset constituted of 186 806 brain MRI images from 3013 scans will be used. Each scan consists of 62 layers of the brain. They are already classified in 3 diagnosis:
The dataset is divided between a training, validation and a test set of respectively 2019, 435 and 469 scans. The dataset is retrieved from Kaggle website and was located on Kaggle Alzheimers MRI dataset, however due to unknown reasons, the dataset was removed from the website at the end of december but a complete description can be found on this github. Hoewever pepole can apply here to get the data directly from University of Southern California.
Our project aims to use the dataset to train a model that will be able to detect if a patient is affected by Alzheimer's disease or not.
To prevent users to store a huge amount of data on their own laptop an USB key is needed to store all data. User create settings.ini file in the local folder (AlzheimersMRI) as shown :
AlzheimersMRI (local folder)
│ settings.ini
│ ...
└───run
│ ...The user MUST set path in settings.ini, according to his local settings. The structure of file settings.ini is :
[dataset]
path = /path/to/USB_KEY
Concerning USB_KEY, it must have the following structure :
USB_KEY
│ adni_demographic_master_kaggle.csv
│
└───MRI
│ img_array_train_6k_1.npy
│ ...
│ img_array_train_6k_22.npy
│ img_array_valid_6k_1.npy
│ ...
│ img_array_valid_6k_5.npy
│ img_array_test_6k_1.npy
│ ...
│ img_array_test_6k_5.npyCSV File : Load data and display first elements of the lookup table associated to the problem. File adni_demographic_master_kaggle.csv contains multiples fields.
Data cleaning in this case means that column are converted to the corresponding type (time or int).
MRI folder : It contains all MRI scans of patients. The data set was pre-splitted in 3 subdata sets : Train, Validation and Test sets. Due to large amount of data each sets are stored as *.npy file. Each *.npy file contains around 6000 images. Each patient is represented by 62 scans of 96x96 pixels. The scans goes from bottom to top of the brain.
Note that laptops are not built to handle such huge amout of data. Therefore training is performed with only a part of the scans (one to five scan(s) per patient).
In [1]:
%load_ext autoreload
%autoreload 2
%matplotlib notebook
%reload_ext autoreload
%matplotlib inline
from DataLoader import DataLoader
from CNN import CNN
import configparser
# Read the dataset path according to user settings
settings = configparser.ConfigParser()
settings.read('settings.ini')
os_path = settings.get('dataset', 'path')
In [2]:
# Load Patient and file preview
patients = DataLoader.load_table(os_path=os_path)
DataLoader.clean_data(patients=patients)
patients.head()
Out[2]:
In [3]:
# Load MRIS (train, validation and test) form dataset and display sizes
id_image = [30, 35, 40, 45, 50]
mris_train = DataLoader.load_MRI_dataset_part_id(os_path=os_path, nbr=22, id_image=id_image, set_type='train')
mris_valid = DataLoader.load_MRI_dataset_part_id(os_path=os_path, nbr=5, id_image=id_image, set_type='valid')
mris_test = DataLoader.load_MRI_dataset_part_id(os_path=os_path, nbr=5, id_image=id_image, set_type='test')
print('Final shape train=', mris_train.shape)
print('Final shape valid=', mris_valid.shape)
print('Final shape test=', mris_test.shape)
Data is analysed according to:
In [4]:
# (a) Gender and Diagnosis repartitions
DataLoader.plot_pie_info(patients=patients)
In [5]:
# (b) Patients age repartition for each diagnosis
DataLoader.plot_age_distrib(patients=patients)
In [6]:
# (c) Scans frequencies over time as an histogram
DataLoader.plot_scan_time_hist(patients=patients)
In [7]:
# (d) Display age of patient over time and color indicationg gender
DataLoader.plot_scan_time_age(patients=patients)
In [8]:
# (e) Correlation between diagnosis and age/sex
DataLoader.print_correlation(patients=patients)
In [9]:
# (f) Sets vizualizations
CNN.plot_sets_preview(train_data=mris_train, valid_data=mris_valid, n_sample = 8)
The convolutional neural network classifier used in this case is defined as:
$$ y=\textrm{softmax}(ReLU(x\ast W_{CL}+b_{CL})W_{FC}+b_{FC}) $$Regularized convolutional neural network classifier with the dropout technique (some neurons are randomly set to 0 in the forward pass)
Here is a demo of the first model that we run with its results displayed. Note that the computation time is high. The results are saved to dummy file 'model_basic_demo.npy'. The result of the run are already displayed below.
In [10]:
# (a) Get training/validation/traing labels and normalize images
patients = DataLoader.load_table(os_path=os_path)
train_data, train_labels = CNN.get_data(patients, mris_train, batch_size=100, id_train=0, is_one_hot=True)
valid_data, valid_labels = CNN.get_data(patients, mris_valid, batch_size=100, id_train=1, is_one_hot=True)
test_data, test_labels = CNN.get_data(patients, mris_test, batch_size=1, id_train=2, is_one_hot=True)
# Run model
CNN.run_basic_cnn(train_data, train_labels, valid_data, valid_labels,
batch_size=100, reg_par=1e-7, learning_rate=1e-3, F=20,
n_iter=15000, print_iter=1000, run_name = 'model_basic_demo.npy')
In [11]:
# (b) Plot probability results and accuracy evolution
CNN.plot_cnn_results('model_basic_demo.npy')
In [12]:
# (c) Test learned model on test data
accuracy_test = CNN.apply_basic_cnn(test_data, test_labels, 'model_basic_demo.npy')
print('Test accuracy=', accuracy_test)
Model was tested with different settings. All plots show the evolution of the validation accuracy over iteration.
In [13]:
# Run results - plotting sweep settings
CNN.plot_run_resume('basic')
To see what the CNN learned a deconvolution is used. It allows to see the activation on the image (e.g. zones of interest learned by the network). Using this method two information shows off. The activation are displayed in red. To classify the images, the NN seems to use information on the outer shell of the brain.
References for deconvolution :
[1] Deconvolutional Networks. Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor and Rob Fergus. Dept. of Computer Science, Courant Institute, New York University.
[2] Visualizing and Understanding Convolutional Networks. Matthew D. Zeiler and Rob Fergus Dept. of Computer Science, New York University, USA
In [14]:
test_data, test_labels = CNN.get_data(patients, mris_test, batch_size=1, id_train=2, is_one_hot=True)
CNN.apply_base_cnn_deconv(test_data, patients, 'model_basic_demo.npy')
Here is a demo of the 'LeNet5' model that we implented which can be run and result will be displayed. Not that the computation time is high. The results are saved to dummy file 'model_lenet_demo.npy'. The result of the run are visible below
(b) - First plot shows the probality of each point to belong to the coresponding class (ground truth indicated by the colors). This is the plot of the outcome of our 'LeNet5' like CNN model. Note that some point are stilly badly classified but there is no more esitation between AD and Normal cassification. This is really good since the model does not mix up the two state. There is a 'smooth' transision from Normal to MCI and then from MCI to AD.
(c) - Print the accuracy of model on test data
Note that this model reaches the same accuracy that the basic model, however it needs fewer iterations to do so.
In [15]:
# (a) Get training/validation/traing labels and normalize images
patients = DataLoader.load_table(os_path=os_path)
train_data, train_labels = CNN.get_data(patients, mris_train, batch_size=100, id_train=0, is_one_hot=False)
valid_data, valid_labels = CNN.get_data(patients, mris_valid, batch_size=100, id_train=1, is_one_hot=False)
test_data, test_labels = CNN.get_data(patients, mris_test, batch_size=1, id_train=2, is_one_hot=False)
# Run basic CNN model using loaded data
CNN.run_leNet5_cnn(train_data, train_labels, valid_data, valid_labels,
batch_size=100, F1=10, F2=20, reg_par=1e-5, learning_rate=1e-3,
n_iter=5000, print_iter=500, drop=0.5,
run_name = 'model_lenet_demo.npy')
In [16]:
# Plot probability results and accuracy evolution
CNN.plot_cnn_results('model_lenet_demo.npy')
In [17]:
# Test learned model on test data
accuracy_test = CNN.apply_lenet5_cnn(test_data, test_labels, 'model_lenet_demo.npy')
print('Test accuracy =', accuracy_test)
Model was tested with different settings. All plots show the evolution of the validation accuracy over iteration.
In [18]:
# Run results - plotting sweep settings
CNN.plot_run_resume('lenet5')
Same deconvolution technique is used on the second CNN. Convolution layer 1 and 2 are shows on (a) and (b).
In [19]:
# (a) Shows activation of first convolutional layer
test_data, test_labels = CNN.get_data(patients, mris_test, batch_size=1, id_train=2, is_one_hot=False)
CNN.apply_lenet5_cnn_deconv(test_data, patients, 'model_lenet_demo.npy', id_layer_conv=1)
In [20]:
# (b) Shows activation of second convolutional layer
CNN.apply_lenet5_cnn_deconv(test_data, patients, 'model_lenet_demo.npy', id_layer_conv=2)
As stated previously, we conducted several run with many parameters to find the best one to use. Also, due to computer limitation, we could only use selected images to perform our experiment. For this reason, we took a step by step approach to conduct this project. We started with a basic model and conducted experiment with one image first to see if we had relevant results. Then we added some more images and tried to implement a more complex model in order to achieve the best result possible with the limitation we had.
Below is a table regrouping the best results we could achieve with the basic model we implemented. Many run were conducted using only one image, and the limit of an accuracy value of 0.75 was quickly reached. In order to surpass this limit, a run with 3 images was conducted. The latter proved to be better and atteined a slightly better accuracy value. We implemented then a more complex model on which we focused all our efforts.
Note that the difference of time needed to compute is due to the fact that 3 different computers with different spech were used.
In [21]:
res = CNN.get_basic_resume()
res.head(8)
Out[21]:
We apprenhended this model knowing that better results were achievable when using more images. We tried using 3 images, 5 and even 7. Again, we conducted many experiments to find the best parameters to use. In this case there are no significant difference between the two models except for the computation times. However this information is not relevant since run were performed on different laptops.
In [22]:
res = CNN.get_lenet_resume()
res.head(8)
Out[22]:
Even if the results are good (around 88% accuracy on validation set), the modely can still be improved.