Once you've trained a model, you might like to get more information about how it performs on the various targets you asked it to predict.

To run this tutorial, you'll need to either download the pre-trained model from https://www.dropbox.com/s/rguytuztemctkf8/pretrained_model.th.gz and preprocess the consortium data, or just substitute your own files here:



In [1]:

    
model_file = '../data/models/pretrained_model.th'
seqs_file = '../data/encode_roadmap.h5'

As long as your HDF5 file has test data set aside, run:



In [3]:

    
import subprocess

cmd = 'basset_test.lua %s %s test_out' % (model_file, seqs_file)
subprocess.call(cmd, shell=True)









    Out[3]:





0

In the output directory, you'll find a table specifying the AUC for each target.



In [4]:

    
!head test_eg/aucs.txt









    



1     0.8820
2     0.9141
3     0.8492
4     0.9062
5     0.7646
6     0.8155
7     0.9008
8     0.9195
9     0.9140
10    0.9058

We can also make receiver operating characteristic curves for each target with the following command.



In [8]:

    
targets_file = '../data/sample_beds.txt'

cmd = 'plot_roc.py -t %s test_out' % (targets_file)
subprocess.call(cmd, shell=True)









    Out[8]:





0



In [9]:

    
# actual file is test_out/roc1.pdf

from IPython.display import Image
Image(filename='test_eg/roc1.png')









    Out[9]: