In [1]:
import glob, os, subprocess
from IPython.display import IFrame
To train a model, you first need to convert your sequences and targets into the input HDF5 format. Check out my tutorials for how to do that; they're linked from the main page.
For this tutorial, grab a small example HDF5 that I constructed here with 10% of the training sequences and only GM12878 targets for various DNase-seq, ChIP-seq, and CAGE experiments.
In [2]:
if len(glob.glob('data/heart_l131k/tfrecords/*.tfr')) == 0:
subprocess.call('curl -o data/heart_l131k.tgz https://storage.googleapis.com/basenji_tutorial_data/heart_l131k.tgz', shell=True)
subprocess.call('tar -xzvf data/heart_l131k.tgz', shell=True)
Next, you need to decide what sort of architecture to use. This grammar probably needs work; my goal was to enable hyperparameter searches to write the parameters to file so that I could run parallel training jobs to explore the hyperparameter space. I included an example set of parameters that will work well with this data in models/params_small.txt.
Then, run basenji_train.py to train a model. The program will offer training feedback via stdout and write the model output files to the prefix given by the -s parameter.
The most relevant options here are:
| Option/Argument | Value | Note |
|---|---|---|
| --augment_rc | True | Process even-numbered epochs as forward, odd-numbered as reverse complemented. |
| --ensemble_rc | True | Average forward and reverse complemented predictions on validation set. |
| --augment_shifts | "1,0,-1" | Rotate epochs over small sequence shifts. |
| --logdir | models/heart | Directory to save training logs and model checkpoints. |
| --params | models/params_small.txt | Table of parameters to setup the model architecture and optimization. |
| --train_data | data/heart_l131k/tfrecords/train*.tfr | Training set TFRecords. |
| --test_data | data/heart_l131k/tfrecords/valid*.tfr | Validation set TFRecords. |
If you want to train, uncomment the following line and run it. Depending on your hardware, it may require several hours.
In [12]:
! basenji_train.py -o models/heart models/params_small.json data/heart_l131k
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
2019-11-10 17:59:08.768155: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-11-10 17:59:08.793392: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_Dataset_flat_map_read_one_file_37
data/heart_l131k/tfrecords/train-*.tfr has 1499 sequences with 3/3 targets
2019-11-10 17:59:10.581209: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_Dataset_flat_map_read_one_file_6087
2019-11-10 17:59:10.632040: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_Dataset_flat_map_read_one_file_6157
data/heart_l131k/tfrecords/valid-*.tfr has 180 sequences with 3/3 targets
2019-11-10 17:59:10.874229: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_Dataset_flat_map_read_one_file_6929
2019-11-10 17:59:10.986618: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_6978
2019-11-10 17:59:10.986647: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_6979
2019-11-10 17:59:11.250601: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_7038
2019-11-10 17:59:11.250659: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_7039
2019-11-10 17:59:11.255172: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_7019
2019-11-10 17:59:11.255236: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_7038
2019-11-10 17:59:11.255273: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_7039
2019-11-10 17:59:11.255305: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_7020
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
sequence (InputLayer) [(None, 131072, 4)] 0
__________________________________________________________________________________________________
stochastic_reverse_complement ( ((None, 131072, 4), 0 sequence[0][0]
__________________________________________________________________________________________________
stochastic_shift (StochasticShi (None, 131072, 4) 0 stochastic_reverse_complement[0][
__________________________________________________________________________________________________
gelu (GELU) (None, 131072, 4) 0 stochastic_shift[0][0]
__________________________________________________________________________________________________
conv1d (Conv1D) (None, 131072, 64) 3840 gelu[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 131072, 64) 256 conv1d[0][0]
__________________________________________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 16384, 64) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
gelu_1 (GELU) (None, 16384, 64) 0 max_pooling1d[0][0]
__________________________________________________________________________________________________
conv1d_1 (Conv1D) (None, 16384, 64) 20480 gelu_1[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 16384, 64) 256 conv1d_1[0][0]
__________________________________________________________________________________________________
max_pooling1d_1 (MaxPooling1D) (None, 4096, 64) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
gelu_2 (GELU) (None, 4096, 64) 0 max_pooling1d_1[0][0]
__________________________________________________________________________________________________
conv1d_2 (Conv1D) (None, 4096, 72) 23040 gelu_2[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 4096, 72) 288 conv1d_2[0][0]
__________________________________________________________________________________________________
max_pooling1d_2 (MaxPooling1D) (None, 1024, 72) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
gelu_3 (GELU) (None, 1024, 72) 0 max_pooling1d_2[0][0]
__________________________________________________________________________________________________
conv1d_3 (Conv1D) (None, 1024, 32) 6912 gelu_3[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 1024, 32) 128 conv1d_3[0][0]
__________________________________________________________________________________________________
gelu_4 (GELU) (None, 1024, 32) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv1d_4 (Conv1D) (None, 1024, 72) 2304 gelu_4[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 1024, 72) 288 conv1d_4[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 1024, 72) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
add (Add) (None, 1024, 72) 0 max_pooling1d_2[0][0]
dropout[0][0]
__________________________________________________________________________________________________
gelu_5 (GELU) (None, 1024, 72) 0 add[0][0]
__________________________________________________________________________________________________
conv1d_5 (Conv1D) (None, None, 32) 6912 gelu_5[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, None, 32) 128 conv1d_5[0][0]
__________________________________________________________________________________________________
gelu_6 (GELU) (None, None, 32) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv1d_6 (Conv1D) (None, None, 72) 2304 gelu_6[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, None, 72) 288 conv1d_6[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, None, 72) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 1024, 72) 0 add[0][0]
dropout_1[0][0]
__________________________________________________________________________________________________
gelu_7 (GELU) (None, 1024, 72) 0 add_1[0][0]
__________________________________________________________________________________________________
conv1d_7 (Conv1D) (None, None, 32) 6912 gelu_7[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, None, 32) 128 conv1d_7[0][0]
__________________________________________________________________________________________________
gelu_8 (GELU) (None, None, 32) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv1d_8 (Conv1D) (None, None, 72) 2304 gelu_8[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, None, 72) 288 conv1d_8[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, None, 72) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 1024, 72) 0 add_1[0][0]
dropout_2[0][0]
__________________________________________________________________________________________________
gelu_9 (GELU) (None, 1024, 72) 0 add_2[0][0]
__________________________________________________________________________________________________
conv1d_9 (Conv1D) (None, None, 32) 6912 gelu_9[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, None, 32) 128 conv1d_9[0][0]
__________________________________________________________________________________________________
gelu_10 (GELU) (None, None, 32) 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
conv1d_10 (Conv1D) (None, None, 72) 2304 gelu_10[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, None, 72) 288 conv1d_10[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, None, 72) 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 1024, 72) 0 add_2[0][0]
dropout_3[0][0]
__________________________________________________________________________________________________
gelu_11 (GELU) (None, 1024, 72) 0 add_3[0][0]
__________________________________________________________________________________________________
conv1d_11 (Conv1D) (None, None, 32) 6912 gelu_11[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, None, 32) 128 conv1d_11[0][0]
__________________________________________________________________________________________________
gelu_12 (GELU) (None, None, 32) 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
conv1d_12 (Conv1D) (None, None, 72) 2304 gelu_12[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, None, 72) 288 conv1d_12[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, None, 72) 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 1024, 72) 0 add_3[0][0]
dropout_4[0][0]
__________________________________________________________________________________________________
gelu_13 (GELU) (None, 1024, 72) 0 add_4[0][0]
__________________________________________________________________________________________________
conv1d_13 (Conv1D) (None, None, 32) 6912 gelu_13[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, None, 32) 128 conv1d_13[0][0]
__________________________________________________________________________________________________
gelu_14 (GELU) (None, None, 32) 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
conv1d_14 (Conv1D) (None, None, 72) 2304 gelu_14[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, None, 72) 288 conv1d_14[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, None, 72) 0 batch_normalization_14[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 1024, 72) 0 add_4[0][0]
dropout_5[0][0]
__________________________________________________________________________________________________
gelu_15 (GELU) (None, 1024, 72) 0 add_5[0][0]
__________________________________________________________________________________________________
conv1d_15 (Conv1D) (None, 1024, 64) 4608 gelu_15[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 1024, 64) 256 conv1d_15[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout) (None, 1024, 64) 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
gelu_16 (GELU) (None, 1024, 64) 0 dropout_6[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 1024, 3) 195 gelu_16[0][0]
__________________________________________________________________________________________________
switch_reverse (SwitchReverse) (None, 1024, 3) 0 dense[0][0]
stochastic_reverse_complement[0][
==================================================================================================
Total params: 111,011
Trainable params: 109,235
Non-trainable params: 1,776
__________________________________________________________________________________________________
None
Epoch 1/20
2019-11-10 17:59:12.530932: W ./tensorflow/core/framework/model.h:213] Encountered a stop event that was not preceded by a start event.
WARNING:tensorflow:From //anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/clip_ops.py:157: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
2019-11-10 17:59:22.030164: I tensorflow/core/profiler/lib/profiler_session.cc:174] Profiler session started.
373/374 [============================>.] - ETA: 0s - loss: 0.4433 - pearsonr: 0.1656 - r2: 0.0206
Epoch 00001: val_loss improved from inf to 0.38308, saving model to models/heart/model_best.h5
374/374 [==============================] - 257s 688ms/step - loss: 0.4433 - pearsonr: 0.1659 - r2: 0.0207 - val_loss: 0.3831 - val_pearsonr: 0.4126 - val_r2: 0.1556
Epoch 2/20
373/374 [============================>.] - ETA: 0s - loss: 0.3876 - pearsonr: 0.2163 - r2: 0.0432
Epoch 00002: val_loss improved from 0.38308 to 0.37163, saving model to models/heart/model_best.h5
374/374 [==============================] - 251s 672ms/step - loss: 0.3875 - pearsonr: 0.2164 - r2: 0.0433 - val_loss: 0.3716 - val_pearsonr: 0.4074 - val_r2: 0.1034
Epoch 3/20
373/374 [============================>.] - ETA: 0s - loss: 0.3811 - pearsonr: 0.2263 - r2: 0.0417
Epoch 00003: val_loss did not improve from 0.37163
374/374 [==============================] - 259s 692ms/step - loss: 0.3810 - pearsonr: 0.2263 - r2: 0.0416 - val_loss: 0.4004 - val_pearsonr: 0.4259 - val_r2: 0.0473
Epoch 4/20
373/374 [============================>.] - ETA: 0s - loss: 0.3723 - pearsonr: 0.2450 - r2: 0.0563
Epoch 00004: val_loss did not improve from 0.37163
374/374 [==============================] - 255s 683ms/step - loss: 0.3721 - pearsonr: 0.2455 - r2: 0.0565 - val_loss: 0.3778 - val_pearsonr: 0.4443 - val_r2: 0.1897
Epoch 5/20
373/374 [============================>.] - ETA: 0s - loss: 0.3640 - pearsonr: 0.2704 - r2: 0.0726
Epoch 00005: val_loss did not improve from 0.37163
374/374 [==============================] - 259s 693ms/step - loss: 0.3641 - pearsonr: 0.2704 - r2: 0.0726 - val_loss: 0.3732 - val_pearsonr: 0.4495 - val_r2: 0.1897
Epoch 6/20
373/374 [============================>.] - ETA: 0s - loss: 0.3560 - pearsonr: 0.2924 - r2: 0.0853
Epoch 00006: val_loss did not improve from 0.37163
374/374 [==============================] - 257s 688ms/step - loss: 0.3562 - pearsonr: 0.2926 - r2: 0.0854 - val_loss: 0.3751 - val_pearsonr: 0.4572 - val_r2: 0.1946
Epoch 7/20
373/374 [============================>.] - ETA: 0s - loss: 0.3573 - pearsonr: 0.3025 - r2: 0.0914
Epoch 00007: val_loss did not improve from 0.37163
374/374 [==============================] - 261s 698ms/step - loss: 0.3574 - pearsonr: 0.3027 - r2: 0.0915 - val_loss: 0.3723 - val_pearsonr: 0.4576 - val_r2: 0.2036
Epoch 8/20
373/374 [============================>.] - ETA: 0s - loss: 0.3582 - pearsonr: 0.2886 - r2: 0.0810
Epoch 00008: val_loss did not improve from 0.37163
374/374 [==============================] - 262s 700ms/step - loss: 0.3583 - pearsonr: 0.2885 - r2: 0.0810 - val_loss: 0.3858 - val_pearsonr: 0.4383 - val_r2: 0.1887
Epoch 9/20
373/374 [============================>.] - ETA: 0s - loss: 0.3491 - pearsonr: 0.3051 - r2: 0.0932
Epoch 00009: val_loss improved from 0.37163 to 0.36718, saving model to models/heart/model_best.h5
374/374 [==============================] - 264s 707ms/step - loss: 0.3491 - pearsonr: 0.3051 - r2: 0.0932 - val_loss: 0.3672 - val_pearsonr: 0.4285 - val_r2: 0.1713
Epoch 10/20
373/374 [============================>.] - ETA: 0s - loss: 0.3517 - pearsonr: 0.3122 - r2: 0.0955
Epoch 00010: val_loss improved from 0.36718 to 0.35215, saving model to models/heart/model_best.h5
374/374 [==============================] - 258s 690ms/step - loss: 0.3517 - pearsonr: 0.3123 - r2: 0.0956 - val_loss: 0.3522 - val_pearsonr: 0.4528 - val_r2: 0.1807
Epoch 11/20
373/374 [============================>.] - ETA: 0s - loss: 0.3502 - pearsonr: 0.3098 - r2: 0.0959
Epoch 00011: val_loss did not improve from 0.35215
374/374 [==============================] - 257s 688ms/step - loss: 0.3502 - pearsonr: 0.3099 - r2: 0.0960 - val_loss: 0.3875 - val_pearsonr: 0.3933 - val_r2: 0.0193
Epoch 12/20
373/374 [============================>.] - ETA: 0s - loss: 0.3455 - pearsonr: 0.3314 - r2: 0.1098
Epoch 00012: val_loss did not improve from 0.35215
374/374 [==============================] - 256s 686ms/step - loss: 0.3458 - pearsonr: 0.3312 - r2: 0.1097 - val_loss: 0.3792 - val_pearsonr: 0.4253 - val_r2: 0.1468
Epoch 13/20
373/374 [============================>.] - ETA: 1s - loss: 0.3520 - pearsonr: 0.3204 - r2: 0.1015
Epoch 00013: val_loss did not improve from 0.35215
374/374 [==============================] - 497s 1s/step - loss: 0.3522 - pearsonr: 0.3202 - r2: 0.1015 - val_loss: 0.3632 - val_pearsonr: 0.4507 - val_r2: 0.0719
Epoch 14/20
373/374 [============================>.] - ETA: 0s - loss: 0.3426 - pearsonr: 0.3551 - r2: 0.1257
Epoch 00014: val_loss improved from 0.35215 to 0.34557, saving model to models/heart/model_best.h5
374/374 [==============================] - 258s 691ms/step - loss: 0.3426 - pearsonr: 0.3554 - r2: 0.1259 - val_loss: 0.3456 - val_pearsonr: 0.4801 - val_r2: 0.2311
Epoch 15/20
373/374 [============================>.] - ETA: 18s - loss: 0.3378 - pearsonr: 0.3843 - r2: 0.1459
Epoch 00015: val_loss did not improve from 0.34557
374/374 [==============================] - 6723s 18s/step - loss: 0.3378 - pearsonr: 0.3843 - r2: 0.1459 - val_loss: 0.3495 - val_pearsonr: 0.4716 - val_r2: 0.1875
Epoch 16/20
373/374 [============================>.] - ETA: 0s - loss: 0.3265 - pearsonr: 0.4003 - r2: 0.1594
Epoch 00016: val_loss did not improve from 0.34557
374/374 [==============================] - 260s 695ms/step - loss: 0.3266 - pearsonr: 0.4003 - r2: 0.1594 - val_loss: 0.3712 - val_pearsonr: 0.4846 - val_r2: 0.2337
Epoch 17/20
373/374 [============================>.] - ETA: 0s - loss: 0.3253 - pearsonr: 0.3961 - r2: 0.1551
Epoch 00017: val_loss did not improve from 0.34557
374/374 [==============================] - 261s 697ms/step - loss: 0.3248 - pearsonr: 0.3970 - r2: 0.1559 - val_loss: 0.3550 - val_pearsonr: 0.4630 - val_r2: 0.0185
Epoch 18/20
373/374 [============================>.] - ETA: 0s - loss: 0.3246 - pearsonr: 0.4538 - r2: 0.2030
Epoch 00018: val_loss did not improve from 0.34557
374/374 [==============================] - 260s 695ms/step - loss: 0.3248 - pearsonr: 0.4535 - r2: 0.2028 - val_loss: 0.3508 - val_pearsonr: 0.4651 - val_r2: 0.2215
Epoch 19/20
373/374 [============================>.] - ETA: 0s - loss: 0.3218 - pearsonr: 0.4874 - r2: 0.2272
Epoch 00019: val_loss did not improve from 0.34557
374/374 [==============================] - 262s 700ms/step - loss: 0.3217 - pearsonr: 0.4872 - r2: 0.2272 - val_loss: 0.3603 - val_pearsonr: 0.4710 - val_r2: 0.1676
Epoch 20/20
373/374 [============================>.] - ETA: 0s - loss: 0.3310 - pearsonr: 0.4510 - r2: 0.2015
Epoch 00020: val_loss did not improve from 0.34557
374/374 [==============================] - 259s 693ms/step - loss: 0.3303 - pearsonr: 0.4494 - r2: 0.1999 - val_loss: 0.3972 - val_pearsonr: 0.4722 - val_r2: 0.2000
Alternatively, you can just download a trained model.
In [4]:
if not os.path.isdir('models/heart'):
os.mkdir('models/heart')
if not os.path.isfile('models/heart/model_best.tf.meta'):
subprocess.call('curl -o models/heart/model_best.tf.index https://storage.googleapis.com/basenji_tutorial_data/model_best.tf.index', shell=True)
subprocess.call('curl -o models/heart/model_best.tf.meta https://storage.googleapis.com/basenji_tutorial_data/model_best.tf.meta', shell=True)
subprocess.call('curl -o models/heart/model_best.tf.data-00000-of-00001 https://storage.googleapis.com/basenji_tutorial_data/model_best.tf.data-00000-of-00001', shell=True)
models/heart/model_best.tf will now specify the name of your saved model to be provided to other programs.
To further benchmark the accuracy (e.g. computing significant "peak" accuracy), use basenji_test.py.
The most relevant options here are:
| Option/Argument | Value | Note |
|---|---|---|
| --rc | Average the forward and reverse complement to form prediction. | |
| -o | output/heart_test | Output directory. |
| --ai | 0,1,2 | Make accuracy scatter plots for targets 0, 1, and 2. |
| --ti | 3,4,5 | Make BigWig tracks for targets 3, 4, and 5. |
| -t | data/heart_l131k.bed | BED file describing sequence regions for BigWig track output. |
| params_file | models/params_small.txt | Table of parameters to setup the model architecture and optimization. |
| model_file | models/heart/model_best.tf | Trained saved model prefix. |
| data_file | data/heart_l131k.h5 | HDF5 file containing the test input and output datasets as generated by basenji_hdf5_single.py |
In [13]:
! basenji_test.py --ai 0,1,2 -o output/heart_test --rc --shifts "1,0,-1" models/params_small.json models/heart/model_best.h5 data/heart_l131k
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
//anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
//anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
2019-11-10 21:45:43.530838: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-11-10 21:45:43.572807: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_Dataset_flat_map_read_one_file_37
data/heart_l131k/tfrecords/test-*.tfr has 179 sequences with 3/3 targets
2019-11-10 21:45:43.855273: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_Dataset_flat_map_read_one_file_805
2019-11-10 21:45:43.971565: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_854
2019-11-10 21:45:43.971595: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_855
2019-11-10 21:45:44.222221: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_914
2019-11-10 21:45:44.222282: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_915
2019-11-10 21:45:44.226511: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_895
2019-11-10 21:45:44.226592: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_914
2019-11-10 21:45:44.226642: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_915
2019-11-10 21:45:44.226673: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_896
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
sequence (InputLayer) [(None, 131072, 4)] 0
__________________________________________________________________________________________________
stochastic_reverse_complement ( ((None, 131072, 4), 0 sequence[0][0]
__________________________________________________________________________________________________
stochastic_shift (StochasticShi (None, 131072, 4) 0 stochastic_reverse_complement[0][
__________________________________________________________________________________________________
gelu (GELU) (None, 131072, 4) 0 stochastic_shift[0][0]
__________________________________________________________________________________________________
conv1d (Conv1D) (None, 131072, 64) 3840 gelu[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 131072, 64) 256 conv1d[0][0]
__________________________________________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 16384, 64) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
gelu_1 (GELU) (None, 16384, 64) 0 max_pooling1d[0][0]
__________________________________________________________________________________________________
conv1d_1 (Conv1D) (None, 16384, 64) 20480 gelu_1[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 16384, 64) 256 conv1d_1[0][0]
__________________________________________________________________________________________________
max_pooling1d_1 (MaxPooling1D) (None, 4096, 64) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
gelu_2 (GELU) (None, 4096, 64) 0 max_pooling1d_1[0][0]
__________________________________________________________________________________________________
conv1d_2 (Conv1D) (None, 4096, 72) 23040 gelu_2[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 4096, 72) 288 conv1d_2[0][0]
__________________________________________________________________________________________________
max_pooling1d_2 (MaxPooling1D) (None, 1024, 72) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
gelu_3 (GELU) (None, 1024, 72) 0 max_pooling1d_2[0][0]
__________________________________________________________________________________________________
conv1d_3 (Conv1D) (None, 1024, 32) 6912 gelu_3[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 1024, 32) 128 conv1d_3[0][0]
__________________________________________________________________________________________________
gelu_4 (GELU) (None, 1024, 32) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv1d_4 (Conv1D) (None, 1024, 72) 2304 gelu_4[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 1024, 72) 288 conv1d_4[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 1024, 72) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
add (Add) (None, 1024, 72) 0 max_pooling1d_2[0][0]
dropout[0][0]
__________________________________________________________________________________________________
gelu_5 (GELU) (None, 1024, 72) 0 add[0][0]
__________________________________________________________________________________________________
conv1d_5 (Conv1D) (None, None, 32) 6912 gelu_5[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, None, 32) 128 conv1d_5[0][0]
__________________________________________________________________________________________________
gelu_6 (GELU) (None, None, 32) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv1d_6 (Conv1D) (None, None, 72) 2304 gelu_6[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, None, 72) 288 conv1d_6[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, None, 72) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 1024, 72) 0 add[0][0]
dropout_1[0][0]
__________________________________________________________________________________________________
gelu_7 (GELU) (None, 1024, 72) 0 add_1[0][0]
__________________________________________________________________________________________________
conv1d_7 (Conv1D) (None, None, 32) 6912 gelu_7[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, None, 32) 128 conv1d_7[0][0]
__________________________________________________________________________________________________
gelu_8 (GELU) (None, None, 32) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv1d_8 (Conv1D) (None, None, 72) 2304 gelu_8[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, None, 72) 288 conv1d_8[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, None, 72) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 1024, 72) 0 add_1[0][0]
dropout_2[0][0]
__________________________________________________________________________________________________
gelu_9 (GELU) (None, 1024, 72) 0 add_2[0][0]
__________________________________________________________________________________________________
conv1d_9 (Conv1D) (None, None, 32) 6912 gelu_9[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, None, 32) 128 conv1d_9[0][0]
__________________________________________________________________________________________________
gelu_10 (GELU) (None, None, 32) 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
conv1d_10 (Conv1D) (None, None, 72) 2304 gelu_10[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, None, 72) 288 conv1d_10[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, None, 72) 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 1024, 72) 0 add_2[0][0]
dropout_3[0][0]
__________________________________________________________________________________________________
gelu_11 (GELU) (None, 1024, 72) 0 add_3[0][0]
__________________________________________________________________________________________________
conv1d_11 (Conv1D) (None, None, 32) 6912 gelu_11[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, None, 32) 128 conv1d_11[0][0]
__________________________________________________________________________________________________
gelu_12 (GELU) (None, None, 32) 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
conv1d_12 (Conv1D) (None, None, 72) 2304 gelu_12[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, None, 72) 288 conv1d_12[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, None, 72) 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 1024, 72) 0 add_3[0][0]
dropout_4[0][0]
__________________________________________________________________________________________________
gelu_13 (GELU) (None, 1024, 72) 0 add_4[0][0]
__________________________________________________________________________________________________
conv1d_13 (Conv1D) (None, None, 32) 6912 gelu_13[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, None, 32) 128 conv1d_13[0][0]
__________________________________________________________________________________________________
gelu_14 (GELU) (None, None, 32) 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
conv1d_14 (Conv1D) (None, None, 72) 2304 gelu_14[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, None, 72) 288 conv1d_14[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, None, 72) 0 batch_normalization_14[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 1024, 72) 0 add_4[0][0]
dropout_5[0][0]
__________________________________________________________________________________________________
gelu_15 (GELU) (None, 1024, 72) 0 add_5[0][0]
__________________________________________________________________________________________________
conv1d_15 (Conv1D) (None, 1024, 64) 4608 gelu_15[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 1024, 64) 256 conv1d_15[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout) (None, 1024, 64) 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
gelu_16 (GELU) (None, 1024, 64) 0 dropout_6[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 1024, 3) 195 gelu_16[0][0]
__________________________________________________________________________________________________
switch_reverse (SwitchReverse) (None, 1024, 3) 0 dense[0][0]
stochastic_reverse_complement[0][
==================================================================================================
Total params: 111,011
Trainable params: 109,235
Non-trainable params: 1,776
__________________________________________________________________________________________________
None
2019-11-10 21:45:45.437792: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_3594
2019-11-10 21:45:45.437821: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_3595
2019-11-10 21:45:45.539367: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_3650
2019-11-10 21:45:45.539471: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_3651
2019-11-10 21:45:45.547124: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_3631
2019-11-10 21:45:45.547229: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_3650
2019-11-10 21:45:45.547291: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_3651
2019-11-10 21:45:45.547345: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_3632
2019-11-10 21:45:46.200085: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_5178
2019-11-10 21:45:46.200114: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_5179
2019-11-10 21:45:46.229871: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_5234
2019-11-10 21:45:46.229926: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_5235
2019-11-10 21:45:46.234356: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_5215
2019-11-10 21:45:46.234419: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_5234
2019-11-10 21:45:46.234457: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_5235
2019-11-10 21:45:46.234491: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_5216
2019-11-10 21:45:46.966716: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_6762
2019-11-10 21:45:46.966746: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_6763
2019-11-10 21:45:46.997261: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_6818
2019-11-10 21:45:46.997321: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_6819
2019-11-10 21:45:47.001883: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_6799
2019-11-10 21:45:47.001951: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_6818
2019-11-10 21:45:47.001988: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_6819
2019-11-10 21:45:47.002022: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_6800
2019-11-10 21:45:47.642678: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_8346
2019-11-10 21:45:47.642709: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_8347
2019-11-10 21:45:47.673226: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_8402
2019-11-10 21:45:47.673285: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_8403
2019-11-10 21:45:47.677688: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_8383
2019-11-10 21:45:47.677819: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_8402
2019-11-10 21:45:47.677873: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_8403
2019-11-10 21:45:47.677907: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_8384
2019-11-10 21:45:48.439626: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_9930
2019-11-10 21:45:48.439657: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_9931
2019-11-10 21:45:48.468684: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_9986
2019-11-10 21:45:48.468737: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_9987
2019-11-10 21:45:48.472988: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_9967
2019-11-10 21:45:48.473053: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_9986
2019-11-10 21:45:48.473091: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_9987
2019-11-10 21:45:48.473125: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_9968
2019-11-10 21:45:49.111117: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_11514
2019-11-10 21:45:49.111146: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_11515
2019-11-10 21:45:49.140789: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_11570
2019-11-10 21:45:49.140846: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_11571
2019-11-10 21:45:49.145178: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_11551
2019-11-10 21:45:49.145242: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_true_11570
2019-11-10 21:45:49.145278: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_11571
2019-11-10 21:45:49.145310: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: cond_false_11552
45/Unknown - 42s 927ms/step - loss: 0.3288 - pearsonr: 0.4365 - r2: 0.1875
Test Loss: 0.32880
Test R2: 0.18752
Test PearsonR: 0.43646
2019-11-10 21:47:13.574264: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_Dataset_flat_map_read_one_file_25395
data/heart_test/acc.txt is a table specifiying the loss function value, R2, R2 after log2, and Spearman correlation for each dataset.
In [14]:
! cat output/heart_test/acc.txt
index r2 pearsonr identifier description
0 0.13260 0.36457 CNhs11760 aorta
1 0.23704 0.50558 CNhs12843 artery
2 0.19291 0.43924 CNhs12856 pulmonic_valve
The directories pr, roc, violin, and scatter in data/heart_test contain plots for the targets indexed by 0, 1, and 2 as specified by the --ai option above.
E.g.
In [15]:
IFrame('output/heart_test/pr/t0.pdf', width=600, height=500)
Out[15]:
Content source: calico/basenji
Similar notebooks: