notebook.community

Edit and run

This colab file is created by Pragnakalp Techlabs.

You can copy this colab in your drive and then execute the command in given order. For more details check our blog NLP Tutorial: Setup Question Answering System using BERT + SQuAD on Colab TPU

Check our BERT based Question and Answering system demo for English and other 8 languages.

You can also purchase the Demo of our BERT based QnA system including fine-tuned models.

BERT Fine-tuning and Prediction on SQUAD 2.0 using Cloud TPU!

Overview

BERT, or Bidirectional Embedding Representations from Transformers, is a new method of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks. The academic paper can be found here: https://arxiv.org/abs/1810.04805.

SQuAD Stanford Question Answering Dataset is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.

This colab file shows how to fine-tune BERT on SQuAD dataset, and then how to perform the prediction. Using this you can create your own Question Answering System.

Prerequisite : You will need a GCP (Google Compute Engine) account and a GCS (Google Cloud Storage) bucket to run this colab file.

Please follow the Google Cloud for how to create GCP account and GCS bucket. You have $300 free credit to get started with any GCP product. You can learn more about it at https://cloud.google.com/tpu/docs/setup-gcp-account

You can create your GCS bucket from here http://console.cloud.google.com/storage.

Change Runtime to TPU

On the main menu, click on Runtime and select Change runtime type. Set "TPU" as the hardware accelerator.

Clone the BERT github repository

First Step is to Clone the BERT github repository, below is the way by which you can clone the repo from github.





In [0]:



    


!git clone https://github.com/google-research/bert.git



    


















    






Cloning into 'bert'...
remote: Enumerating objects: 340, done.
remote: Total 340 (delta 0), reused 0 (delta 0), pack-reused 340
Receiving objects: 100% (340/340), 300.28 KiB | 4.06 MiB/s, done.
Resolving deltas: 100% (185/185), done.

Confirm that BERT repo is cloned properly.

"ls -l" is used for long listing, if BERT repo is cloned properly you can see the BERT folder in current directory.





In [0]:



    


ls -l



    


















    






total 8
drwxr-xr-x 3 root root 4096 Mar 17 10:10 bert/
drwxr-xr-x 1 root root 4096 Mar  3 18:11 sample_data/

















In [0]:



    


cd bert



    


















    






/content/bert

BERT repository files

use ls -l to check the content inside BERT folder, you can see all files related to BERT.





In [0]:



    


ls -l



    


















    






total 400
-rw-r--r-- 1 root root  1323 Mar 17 10:10 CONTRIBUTING.md
-rw-r--r-- 1 root root 16475 Mar 17 10:10 create_pretraining_data.py
-rw-r--r-- 1 root root 13898 Mar 17 10:10 extract_features.py
-rw-r--r-- 1 root root   616 Mar 17 10:10 __init__.py
-rw-r--r-- 1 root root 11358 Mar 17 10:10 LICENSE
-rw-r--r-- 1 root root 37922 Mar 17 10:10 modeling.py
-rw-r--r-- 1 root root  9191 Mar 17 10:10 modeling_test.py
-rw-r--r-- 1 root root 11242 Mar 17 10:10 multilingual.md
-rw-r--r-- 1 root root  6258 Mar 17 10:10 optimization.py
-rw-r--r-- 1 root root  1721 Mar 17 10:10 optimization_test.py
-rw-r--r-- 1 root root 66488 Mar 17 10:10 predicting_movie_reviews_with_bert_on_tf_hub.ipynb
-rw-r--r-- 1 root root 50519 Mar 17 10:10 README.md
-rw-r--r-- 1 root root   110 Mar 17 10:10 requirements.txt
-rw-r--r-- 1 root root 34783 Mar 17 10:10 run_classifier.py
-rw-r--r-- 1 root root 11426 Mar 17 10:10 run_classifier_with_tfhub.py
-rw-r--r-- 1 root root 18667 Mar 17 10:10 run_pretraining.py
-rw-r--r-- 1 root root 46532 Mar 17 10:10 run_squad.py
-rw-r--r-- 1 root root  4394 Mar 17 10:10 sample_text.txt
-rw-r--r-- 1 root root 12257 Mar 17 10:10 tokenization.py
-rw-r--r-- 1 root root  4589 Mar 17 10:10 tokenization_test.py

Download the BERT PRETRAINED MODEL

BERT Pretrained Model List :

BERT has release BERT-Base and BERT-Large models. Uncased means that the text has been lowercased before WordPiece tokenization, e.g., John Smith becomes john smith, whereas Cased means that the true case and accent markers are preserved.

When using a cased model, make sure to pass --do_lower=False at the time of training.

You can download any model of your choice. We have used BERT-Large-Uncased Model.





In [0]:



    


!wget https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-24_H-1024_A-16.zip



    


















    






--2020-03-17 10:11:11--  https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-24_H-1024_A-16.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.214.128, 2607:f8b0:4001:c05::80
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.214.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1247797031 (1.2G) [application/zip]
Saving to: ‘uncased_L-24_H-1024_A-16.zip’

uncased_L-24_H-1024 100%[===================>]   1.16G  95.0MB/s    in 9.9s    

2020-03-17 10:11:22 (120 MB/s) - ‘uncased_L-24_H-1024_A-16.zip’ saved [1247797031/1247797031]


















In [0]:



    


# Unzip the pretrained model
!unzip uncased_L-24_H-1024_A-16.zip



    


















    






Archive:  uncased_L-24_H-1024_A-16.zip
   creating: uncased_L-24_H-1024_A-16/
  inflating: uncased_L-24_H-1024_A-16/bert_model.ckpt.meta  
  inflating: uncased_L-24_H-1024_A-16/bert_model.ckpt.data-00000-of-00001  
  inflating: uncased_L-24_H-1024_A-16/vocab.txt  
  inflating: uncased_L-24_H-1024_A-16/bert_model.ckpt.index  
  inflating: uncased_L-24_H-1024_A-16/bert_config.json

Download the SQUAD 2.0 Dataset





In [0]:



    


#Download the SQUAD train and dev dataset
!wget https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json
!wget https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json



    


















    






--2020-03-17 10:11:42--  https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json
Resolving rajpurkar.github.io (rajpurkar.github.io)... 185.199.108.153, 185.199.111.153, 185.199.110.153, ...
Connecting to rajpurkar.github.io (rajpurkar.github.io)|185.199.108.153|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 42123633 (40M) [application/json]
Saving to: ‘train-v2.0.json’

train-v2.0.json     100%[===================>]  40.17M   158MB/s    in 0.3s    

2020-03-17 10:11:42 (158 MB/s) - ‘train-v2.0.json’ saved [42123633/42123633]

--2020-03-17 10:11:45--  https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json
Resolving rajpurkar.github.io (rajpurkar.github.io)... 185.199.108.153, 185.199.111.153, 185.199.110.153, ...
Connecting to rajpurkar.github.io (rajpurkar.github.io)|185.199.108.153|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4370528 (4.2M) [application/json]
Saving to: ‘dev-v2.0.json’

dev-v2.0.json       100%[===================>]   4.17M  22.3MB/s    in 0.2s    

2020-03-17 10:11:45 (22.3 MB/s) - ‘dev-v2.0.json’ saved [4370528/4370528]

Set up your TPU environment

  • Verify that you are connected to a TPU device
  • You will get know your TPU Address that is used at time of fine-tuning
  • Perform Google Authentication to access your bucket
  • Upload your credentials to TPU to access your GCS bucket




In [0]:



    


import datetime
import json
import os
import pprint
import random
import string
import sys
import tensorflow as tf

assert 'COLAB_TPU_ADDR' in os.environ, 'ERROR: Not connected to a TPU runtime; please see the first cell in this notebook for instructions!'
TPU_ADDRESS = 'grpc://' + os.environ['COLAB_TPU_ADDR']
print('TPU address is => ', TPU_ADDRESS)

from google.colab import auth
auth.authenticate_user()
with tf.Session(TPU_ADDRESS) as session:
  print('TPU devices:')
  pprint.pprint(session.list_devices())

  # Upload credentials to TPU.
  with open('/content/adc.json', 'r') as f:
    auth_info = json.load(f)
  tf.contrib.cloud.configure_gcs(session, credentials=auth_info)
  # Now credentials are set for all future sessions on this TPU.



    


















    






TPU address is =>  grpc://10.29.165.74:8470
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

TPU devices:
[_DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:CPU:0, CPU, -1, 7635328332459542216),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 2874832984123831798),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 4040080567667641550),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:1, TPU, 17179869184, 5679213459339138745),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:2, TPU, 17179869184, 13541815098229104680),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:3, TPU, 17179869184, 11341280136594724657),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:4, TPU, 17179869184, 15504123199676842572),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:5, TPU, 17179869184, 8451755694666277343),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:6, TPU, 17179869184, 8025377445115626534),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU:7, TPU, 17179869184, 8053071906858085354),
 _DeviceAttributes(/job:tpu_worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 8589934592, 9885964334607432210)]

Create output directory

Need to create a output directory at GCS (Google Cloud Storage) bucket, where you will get your fine_tuned model after training completion. For that you need to provide your BUCKET name and OUPUT DIRECTORY name.

Also need to move Pre-trained Model at GCS (Google Cloud Storage) bucket, as Local File System is not Supported on TPU. If you don't move your pretrained model to TPU you may face an error.





In [0]:



    


BUCKET = 'bertnlpdemo' #@param {type:"string"}
assert BUCKET, '*** Must specify an existing GCS bucket name ***'
output_dir_name = 'bert_output' #@param {type:"string"}
BUCKET_NAME = 'gs://{}'.format(BUCKET)
OUTPUT_DIR = 'gs://{}/{}'.format(BUCKET,output_dir_name)
tf.gfile.MakeDirs(OUTPUT_DIR)
print('***** Model output directory: {} *****'.format(OUTPUT_DIR))



    


















    






***** Model output directory: gs://bertnlpdemo/bert_output *****

Move Pretrained Model to GCS Bucket

Need to move Pre-trained Model at GCS (Google Cloud Storage) bucket, as Local File System is not Supported on TPU. If you don't move your pretrained model to TPU you may face the error.

The gsutil mv command allows you to move data between your local file system and the cloud, move data within the cloud, and move data between cloud storage providers.





In [0]:



    


!gsutil mv /content/bert/uncased_L-24_H-1024_A-16 $BUCKET_NAME



    


















    






Copying file:///content/bert/uncased_L-24_H-1024_A-16/bert_config.json [Content-Type=application/json]...
Removing file:///content/bert/uncased_L-24_H-1024_A-16/bert_config.json...
Copying file:///content/bert/uncased_L-24_H-1024_A-16/bert_model.ckpt.index [Content-Type=application/octet-stream]...
Removing file:///content/bert/uncased_L-24_H-1024_A-16/bert_model.ckpt.index...
Copying file:///content/bert/uncased_L-24_H-1024_A-16/vocab.txt [Content-Type=text/plain]...
Removing file:///content/bert/uncased_L-24_H-1024_A-16/vocab.txt...
Copying file:///content/bert/uncased_L-24_H-1024_A-16/bert_model.ckpt.meta [Content-Type=application/octet-stream]...
Removing file:///content/bert/uncased_L-24_H-1024_A-16/bert_model.ckpt.meta...

==> NOTE: You are performing a sequence of gsutil operations that may
run significantly faster if you instead use gsutil -m cp ... Please
see the -m section under "gsutil help options" for further information
about when gsutil -m can be advantageous.

Copying file:///content/bert/uncased_L-24_H-1024_A-16/bert_model.ckpt.data-00000-of-00001 [Content-Type=application/octet-stream]...
==> NOTE: You are uploading one or more large file(s), which would run
significantly faster if you enable parallel composite uploads. This
feature can be enabled by editing the
"parallel_composite_upload_threshold" value in your .boto
configuration file. However, note that if you do this large files will
be uploaded as `composite objects
<https://cloud.google.com/storage/docs/composite-objects>`_,which
means that any user who downloads such objects will need to have a
compiled crcmod installed (see "gsutil help crcmod"). This is because
without a compiled crcmod, computing checksums on composite objects is
so slow that gsutil disables downloads of composite objects.

Removing file:///content/bert/uncased_L-24_H-1024_A-16/bert_model.ckpt.data-00000-of-00001...

Operation completed over 5 objects/1.3 GiB.

Training

Below is the command to run the training. To run the training on TPU you need to make sure about below Hyperparameter, that is tpu must be true and provide the tpu_address that we have find out above.

  1. --use_tpu=True
  2. --tpu_name=YOUR_TPU_ADDRESS




In [0]:



    


!python run_squad.py \
  --vocab_file=$BUCKET_NAME/uncased_L-24_H-1024_A-16/vocab.txt \
  --bert_config_file=$BUCKET_NAME/uncased_L-24_H-1024_A-16/bert_config.json \
  --init_checkpoint=$BUCKET_NAME/uncased_L-24_H-1024_A-16/bert_model.ckpt \
  --do_train=True \
  --train_file=train-v2.0.json \
  --do_predict=True \
  --predict_file=dev-v2.0.json \
  --train_batch_size=24 \
  --learning_rate=3e-5 \
  --num_train_epochs=2.0 \
  --use_tpu=True \
  --tpu_name=grpc://10.1.118.82:8470 \
  --max_seq_length=384 \
  --doc_stride=128 \
  --version_2_with_negative=True \
  --output_dir=$OUTPUT_DIR



    


















    






WARNING:tensorflow:From /content/bert/optimization.py:87: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING:tensorflow:From run_squad.py:1283: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

WARNING:tensorflow:From run_squad.py:1127: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

W1203 11:55:59.995234 139660688828288 module_wrapper.py:139] From run_squad.py:1127: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From run_squad.py:1127: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

W1203 11:55:59.995468 139660688828288 module_wrapper.py:139] From run_squad.py:1127: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /content/bert/modeling.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W1203 11:55:59.995662 139660688828288 module_wrapper.py:139] From /content/bert/modeling.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From run_squad.py:1133: The name tf.gfile.MakeDirs is deprecated. Please use tf.io.gfile.makedirs instead.

W1203 11:56:01.214218 139660688828288 module_wrapper.py:139] From run_squad.py:1133: The name tf.gfile.MakeDirs is deprecated. Please use tf.io.gfile.makedirs instead.

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W1203 11:56:01.495515 139660688828288 lazy_loader.py:50] 
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

I1203 11:56:02.063556 139660688828288 utils.py:141] NumExpr defaulting to 2 threads.
WARNING:tensorflow:From run_squad.py:229: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.

W1203 11:56:03.502187 139660688828288 module_wrapper.py:139] From run_squad.py:229: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f04e61ac048>) includes params argument, but params are not passed to Estimator.
W1203 11:56:12.588263 139660688828288 estimator.py:1994] Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f04e61ac048>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:Using config: {'_model_dir': 'gs://bertnlpdemo/bert_output/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
cluster_def {
  job {
    name: "worker"
    tasks {
      key: 0
      value: "10.1.118.82:8470"
    }
  }
}
isolate_session_state: true
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f04e621bba8>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': 'grpc://10.1.118.82:8470', '_evaluation_master': 'grpc://10.1.118.82:8470', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': <tensorflow.python.distribute.cluster_resolver.tpu_cluster_resolver.TPUClusterResolver object at 0x7f04f5260eb8>}
I1203 11:56:12.590006 139660688828288 estimator.py:212] Using config: {'_model_dir': 'gs://bertnlpdemo/bert_output/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
cluster_def {
  job {
    name: "worker"
    tasks {
      key: 0
      value: "10.1.118.82:8470"
    }
  }
}
isolate_session_state: true
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f04e621bba8>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': 'grpc://10.1.118.82:8470', '_evaluation_master': 'grpc://10.1.118.82:8470', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': <tensorflow.python.distribute.cluster_resolver.tpu_cluster_resolver.TPUClusterResolver object at 0x7f04f5260eb8>}
INFO:tensorflow:_TPUContext: eval_on_tpu True
I1203 11:56:12.590440 139660688828288 tpu_context.py:220] _TPUContext: eval_on_tpu True
WARNING:tensorflow:From run_squad.py:1065: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

W1203 11:56:12.591245 139660688828288 module_wrapper.py:139] From run_squad.py:1065: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

WARNING:tensorflow:From run_squad.py:431: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

W1203 11:56:12.595412 139660688828288 module_wrapper.py:139] From run_squad.py:431: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

INFO:tensorflow:*** Example ***
I1203 11:56:12.595659 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000000
I1203 11:56:12.595808 139660688828288 run_squad.py:432] unique_id: 1000000000
INFO:tensorflow:example_index: 0
I1203 11:56:12.595922 139660688828288 run_squad.py:433] example_index: 0
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.596024 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what is the name of the act that was a success in creating boundaries for the crown and the e ##ic for being subjective ? [SEP] pitt ' s act was deemed a failure because it quickly became apparent that the boundaries between government control and the company ' s powers were ne ##bu ##lous and highly subjective . the government felt obliged to respond to humanitarian calls for better treatment of local peoples in british - occupied territories . edmund burke , a former east india company shareholder and diplomat , was moved to address the situation and introduced a new regulating bill in 1783 . the bill was defeated amid lobbying by company loyalists and accusations of ne ##pot ##ism in the bill ' s recommendations for the appointment of councillors . [SEP]
I1203 11:56:12.596178 139660688828288 run_squad.py:436] tokens: [CLS] what is the name of the act that was a success in creating boundaries for the crown and the e ##ic for being subjective ? [SEP] pitt ' s act was deemed a failure because it quickly became apparent that the boundaries between government control and the company ' s powers were ne ##bu ##lous and highly subjective . the government felt obliged to respond to humanitarian calls for better treatment of local peoples in british - occupied territories . edmund burke , a former east india company shareholder and diplomat , was moved to address the situation and introduced a new regulating bill in 1783 . the bill was defeated amid lobbying by company loyalists and accusations of ne ##pot ##ism in the bill ' s recommendations for the appointment of councillors . [SEP]
INFO:tensorflow:token_to_orig_map: 27:0 28:0 29:0 30:1 31:2 32:3 33:4 34:5 35:6 36:7 37:8 38:9 39:10 40:11 41:12 42:13 43:14 44:15 45:16 46:17 47:18 48:19 49:19 50:19 51:20 52:21 53:22 54:22 55:22 56:23 57:24 58:25 59:25 60:26 61:27 62:28 63:29 64:30 65:31 66:32 67:33 68:34 69:35 70:36 71:37 72:38 73:39 74:40 75:41 76:42 77:42 78:42 79:43 80:43 81:44 82:45 83:45 84:46 85:47 86:48 87:49 88:50 89:51 90:52 91:53 92:53 93:54 94:55 95:56 96:57 97:58 98:59 99:60 100:61 101:62 102:63 103:64 104:65 105:66 106:67 107:67 108:68 109:69 110:70 111:71 112:72 113:73 114:74 115:75 116:76 117:77 118:78 119:79 120:80 121:80 122:80 123:81 124:82 125:83 126:83 127:83 128:84 129:85 130:86 131:87 132:88 133:89 134:89
I1203 11:56:12.596332 139660688828288 run_squad.py:438] token_to_orig_map: 27:0 28:0 29:0 30:1 31:2 32:3 33:4 34:5 35:6 36:7 37:8 38:9 39:10 40:11 41:12 42:13 43:14 44:15 45:16 46:17 47:18 48:19 49:19 50:19 51:20 52:21 53:22 54:22 55:22 56:23 57:24 58:25 59:25 60:26 61:27 62:28 63:29 64:30 65:31 66:32 67:33 68:34 69:35 70:36 71:37 72:38 73:39 74:40 75:41 76:42 77:42 78:42 79:43 80:43 81:44 82:45 83:45 84:46 85:47 86:48 87:49 88:50 89:51 90:52 91:53 92:53 93:54 94:55 95:56 96:57 97:58 98:59 99:60 100:61 101:62 102:63 103:64 104:65 105:66 106:67 107:67 108:68 109:69 110:70 111:71 112:72 113:73 114:74 115:75 116:76 117:77 118:78 119:79 120:80 121:80 122:80 123:81 124:82 125:83 126:83 127:83 128:84 129:85 130:86 131:87 132:88 133:89 134:89
INFO:tensorflow:token_is_max_context: 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True
I1203 11:56:12.596476 139660688828288 run_squad.py:440] token_is_max_context: 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True
INFO:tensorflow:input_ids: 101 2054 2003 1996 2171 1997 1996 2552 2008 2001 1037 3112 1999 4526 7372 2005 1996 4410 1998 1996 1041 2594 2005 2108 20714 1029 102 15091 1005 1055 2552 2001 8357 1037 4945 2138 2009 2855 2150 6835 2008 1996 7372 2090 2231 2491 1998 1996 2194 1005 1055 4204 2020 11265 8569 15534 1998 3811 20714 1012 1996 2231 2371 14723 2000 6869 2000 11470 4455 2005 2488 3949 1997 2334 7243 1999 2329 1011 4548 6500 1012 9493 9894 1010 1037 2280 2264 2634 2194 18668 1998 11125 1010 2001 2333 2000 4769 1996 3663 1998 3107 1037 2047 21575 3021 1999 15331 1012 1996 3021 2001 3249 13463 19670 2011 2194 26590 1998 13519 1997 11265 11008 2964 1999 1996 3021 1005 1055 11433 2005 1996 6098 1997 13189 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.596688 139660688828288 run_squad.py:442] input_ids: 101 2054 2003 1996 2171 1997 1996 2552 2008 2001 1037 3112 1999 4526 7372 2005 1996 4410 1998 1996 1041 2594 2005 2108 20714 1029 102 15091 1005 1055 2552 2001 8357 1037 4945 2138 2009 2855 2150 6835 2008 1996 7372 2090 2231 2491 1998 1996 2194 1005 1055 4204 2020 11265 8569 15534 1998 3811 20714 1012 1996 2231 2371 14723 2000 6869 2000 11470 4455 2005 2488 3949 1997 2334 7243 1999 2329 1011 4548 6500 1012 9493 9894 1010 1037 2280 2264 2634 2194 18668 1998 11125 1010 2001 2333 2000 4769 1996 3663 1998 3107 1037 2047 21575 3021 1999 15331 1012 1996 3021 2001 3249 13463 19670 2011 2194 26590 1998 13519 1997 11265 11008 2964 1999 1996 3021 1005 1055 11433 2005 1996 6098 1997 13189 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.596898 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.597087 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:impossible example
I1203 11:56:12.597178 139660688828288 run_squad.py:448] impossible example
INFO:tensorflow:*** Example ***
I1203 11:56:12.602271 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000001
I1203 11:56:12.602504 139660688828288 run_squad.py:432] unique_id: 1000000001
INFO:tensorflow:example_index: 1
I1203 11:56:12.602625 139660688828288 run_squad.py:433] example_index: 1
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.602713 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what do winners of the continental competition get to do ? [SEP] after the world cup , the most important international football competitions are the continental championships , which are organised by each continental confederation and contested between national teams . these are the european championship ( uefa ) , the copa america ( con ##me ##bol ) , african cup of nations ( caf ) , the asian cup ( afc ) , the concacaf gold cup ( concacaf ) and the of ##c nations cup ( of ##c ) . the fifa confederation ##s cup is contested by the winners of all six continental championships , the current fifa world cup champions and the country which is hosting the confederation ##s cup . this is generally regarded as a warm - up tournament for the upcoming fifa world cup and does not carry the same prestige as the world cup itself . the most prestigious competitions in club football are the respective continental championships , which are generally contested between national champions , for example the uefa champions league in europe and the copa libertadores in south america . the winners of each continental competition contest the fifa club world cup . [SEP]
I1203 11:56:12.602903 139660688828288 run_squad.py:436] tokens: [CLS] what do winners of the continental competition get to do ? [SEP] after the world cup , the most important international football competitions are the continental championships , which are organised by each continental confederation and contested between national teams . these are the european championship ( uefa ) , the copa america ( con ##me ##bol ) , african cup of nations ( caf ) , the asian cup ( afc ) , the concacaf gold cup ( concacaf ) and the of ##c nations cup ( of ##c ) . the fifa confederation ##s cup is contested by the winners of all six continental championships , the current fifa world cup champions and the country which is hosting the confederation ##s cup . this is generally regarded as a warm - up tournament for the upcoming fifa world cup and does not carry the same prestige as the world cup itself . the most prestigious competitions in club football are the respective continental championships , which are generally contested between national champions , for example the uefa champions league in europe and the copa libertadores in south america . the winners of each continental competition contest the fifa club world cup . [SEP]
INFO:tensorflow:token_to_orig_map: 13:0 14:1 15:2 16:3 17:3 18:4 19:5 20:6 21:7 22:8 23:9 24:10 25:11 26:12 27:13 28:13 29:14 30:15 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:25 42:26 43:27 44:28 45:29 46:30 47:31 48:31 49:31 50:31 51:32 52:33 53:34 54:35 55:35 56:35 57:35 58:35 59:35 60:36 61:37 62:38 63:39 64:40 65:40 66:40 67:40 68:41 69:42 70:43 71:44 72:44 73:44 74:44 75:45 76:46 77:47 78:48 79:49 80:49 81:49 82:50 83:51 84:52 85:52 86:53 87:54 88:55 89:55 90:55 91:55 92:55 93:56 94:57 95:58 96:58 97:59 98:60 99:61 100:62 101:63 102:64 103:65 104:66 105:67 106:68 107:69 108:69 109:70 110:71 111:72 112:73 113:74 114:75 115:76 116:77 117:78 118:79 119:80 120:81 121:82 122:83 123:83 124:84 125:84 126:85 127:86 128:87 129:88 130:89 131:90 132:91 133:91 134:91 135:92 136:93 137:94 138:95 139:96 140:97 141:98 142:99 143:100 144:101 145:102 146:103 147:104 148:105 149:106 150:107 151:108 152:109 153:110 154:110 155:111 156:112 157:113 158:114 159:115 160:116 161:117 162:118 163:119 164:120 165:121 166:122 167:122 168:123 169:124 170:125 171:126 172:127 173:128 174:129 175:129 176:130 177:131 178:132 179:133 180:134 181:135 182:136 183:137 184:138 185:139 186:140 187:141 188:142 189:143 190:144 191:144 192:145 193:146 194:147 195:148 196:149 197:150 198:151 199:152 200:153 201:154 202:155 203:156 204:156
I1203 11:56:12.664989 139660688828288 run_squad.py:438] token_to_orig_map: 13:0 14:1 15:2 16:3 17:3 18:4 19:5 20:6 21:7 22:8 23:9 24:10 25:11 26:12 27:13 28:13 29:14 30:15 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:25 42:26 43:27 44:28 45:29 46:30 47:31 48:31 49:31 50:31 51:32 52:33 53:34 54:35 55:35 56:35 57:35 58:35 59:35 60:36 61:37 62:38 63:39 64:40 65:40 66:40 67:40 68:41 69:42 70:43 71:44 72:44 73:44 74:44 75:45 76:46 77:47 78:48 79:49 80:49 81:49 82:50 83:51 84:52 85:52 86:53 87:54 88:55 89:55 90:55 91:55 92:55 93:56 94:57 95:58 96:58 97:59 98:60 99:61 100:62 101:63 102:64 103:65 104:66 105:67 106:68 107:69 108:69 109:70 110:71 111:72 112:73 113:74 114:75 115:76 116:77 117:78 118:79 119:80 120:81 121:82 122:83 123:83 124:84 125:84 126:85 127:86 128:87 129:88 130:89 131:90 132:91 133:91 134:91 135:92 136:93 137:94 138:95 139:96 140:97 141:98 142:99 143:100 144:101 145:102 146:103 147:104 148:105 149:106 150:107 151:108 152:109 153:110 154:110 155:111 156:112 157:113 158:114 159:115 160:116 161:117 162:118 163:119 164:120 165:121 166:122 167:122 168:123 169:124 170:125 171:126 172:127 173:128 174:129 175:129 176:130 177:131 178:132 179:133 180:134 181:135 182:136 183:137 184:138 185:139 186:140 187:141 188:142 189:143 190:144 191:144 192:145 193:146 194:147 195:148 196:149 197:150 198:151 199:152 200:153 201:154 202:155 203:156 204:156
INFO:tensorflow:token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True
I1203 11:56:12.665372 139660688828288 run_squad.py:440] token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True
INFO:tensorflow:input_ids: 101 2054 2079 4791 1997 1996 6803 2971 2131 2000 2079 1029 102 2044 1996 2088 2452 1010 1996 2087 2590 2248 2374 6479 2024 1996 6803 3219 1010 2029 2024 7362 2011 2169 6803 11078 1998 7259 2090 2120 2780 1012 2122 2024 1996 2647 2528 1006 6663 1007 1010 1996 10613 2637 1006 9530 4168 14956 1007 1010 3060 2452 1997 3741 1006 24689 1007 1010 1996 4004 2452 1006 10511 1007 1010 1996 22169 2751 2452 1006 22169 1007 1998 1996 1997 2278 3741 2452 1006 1997 2278 1007 1012 1996 5713 11078 2015 2452 2003 7259 2011 1996 4791 1997 2035 2416 6803 3219 1010 1996 2783 5713 2088 2452 3966 1998 1996 2406 2029 2003 9936 1996 11078 2015 2452 1012 2023 2003 3227 5240 2004 1037 4010 1011 2039 2977 2005 1996 9046 5713 2088 2452 1998 2515 2025 4287 1996 2168 14653 2004 1996 2088 2452 2993 1012 1996 2087 8919 6479 1999 2252 2374 2024 1996 7972 6803 3219 1010 2029 2024 3227 7259 2090 2120 3966 1010 2005 2742 1996 6663 3966 2223 1999 2885 1998 1996 10613 27968 1999 2148 2637 1012 1996 4791 1997 2169 6803 2971 5049 1996 5713 2252 2088 2452 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.665721 139660688828288 run_squad.py:442] input_ids: 101 2054 2079 4791 1997 1996 6803 2971 2131 2000 2079 1029 102 2044 1996 2088 2452 1010 1996 2087 2590 2248 2374 6479 2024 1996 6803 3219 1010 2029 2024 7362 2011 2169 6803 11078 1998 7259 2090 2120 2780 1012 2122 2024 1996 2647 2528 1006 6663 1007 1010 1996 10613 2637 1006 9530 4168 14956 1007 1010 3060 2452 1997 3741 1006 24689 1007 1010 1996 4004 2452 1006 10511 1007 1010 1996 22169 2751 2452 1006 22169 1007 1998 1996 1997 2278 3741 2452 1006 1997 2278 1007 1012 1996 5713 11078 2015 2452 2003 7259 2011 1996 4791 1997 2035 2416 6803 3219 1010 1996 2783 5713 2088 2452 3966 1998 1996 2406 2029 2003 9936 1996 11078 2015 2452 1012 2023 2003 3227 5240 2004 1037 4010 1011 2039 2977 2005 1996 9046 5713 2088 2452 1998 2515 2025 4287 1996 2168 14653 2004 1996 2088 2452 2993 1012 1996 2087 8919 6479 1999 2252 2374 2024 1996 7972 6803 3219 1010 2029 2024 3227 7259 2090 2120 3966 1010 2005 2742 1996 6663 3966 2223 1999 2885 1998 1996 10613 27968 1999 2148 2637 1012 1996 4791 1997 2169 6803 2971 5049 1996 5713 2252 2088 2452 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.666256 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.666547 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 198
I1203 11:56:12.666700 139660688828288 run_squad.py:451] start_position: 198
INFO:tensorflow:end_position: 203
I1203 11:56:12.666842 139660688828288 run_squad.py:452] end_position: 203
INFO:tensorflow:answer: contest the fifa club world cup
I1203 11:56:12.666953 139660688828288 run_squad.py:454] answer: contest the fifa club world cup
INFO:tensorflow:*** Example ***
I1203 11:56:12.671008 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000002
I1203 11:56:12.671200 139660688828288 run_squad.py:432] unique_id: 1000000002
INFO:tensorflow:example_index: 2
I1203 11:56:12.671302 139660688828288 run_squad.py:433] example_index: 2
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.671390 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what did frederick zu ##gi ##be study in detail for this book ? [SEP] in his book the cr ##uc ##if ##ix ##ion of jesus , physician and forensic path ##ologist frederick zu ##gi ##be studied the likely circumstances of the death of jesus in great detail . zu ##gi ##be carried out a number of experiments over several years to test his theories while he was a medical examiner . these studies included experiments in which volunteers with specific weights were hanging at specific angles and the amount of pull on each hand was measured , in cases where the feet were also secured or not . in these cases the amount of pull and the corresponding pain was found to be significant . [SEP]
I1203 11:56:12.671549 139660688828288 run_squad.py:436] tokens: [CLS] what did frederick zu ##gi ##be study in detail for this book ? [SEP] in his book the cr ##uc ##if ##ix ##ion of jesus , physician and forensic path ##ologist frederick zu ##gi ##be studied the likely circumstances of the death of jesus in great detail . zu ##gi ##be carried out a number of experiments over several years to test his theories while he was a medical examiner . these studies included experiments in which volunteers with specific weights were hanging at specific angles and the amount of pull on each hand was measured , in cases where the feet were also secured or not . in these cases the amount of pull and the corresponding pain was found to be significant . [SEP]
INFO:tensorflow:token_to_orig_map: 15:0 16:1 17:2 18:3 19:4 20:4 21:4 22:4 23:4 24:5 25:6 26:6 27:7 28:8 29:9 30:10 31:10 32:11 33:12 34:12 35:12 36:13 37:14 38:15 39:16 40:17 41:18 42:19 43:20 44:21 45:22 46:23 47:24 48:24 49:25 50:25 51:25 52:26 53:27 54:28 55:29 56:30 57:31 58:32 59:33 60:34 61:35 62:36 63:37 64:38 65:39 66:40 67:41 68:42 69:43 70:44 71:44 72:45 73:46 74:47 75:48 76:49 77:50 78:51 79:52 80:53 81:54 82:55 83:56 84:57 85:58 86:59 87:60 88:61 89:62 90:63 91:64 92:65 93:66 94:67 95:68 96:69 97:69 98:70 99:71 100:72 101:73 102:74 103:75 104:76 105:77 106:78 107:79 108:79 109:80 110:81 111:82 112:83 113:84 114:85 115:86 116:87 117:88 118:89 119:90 120:91 121:92 122:93 123:94 124:95 125:95
I1203 11:56:12.671710 139660688828288 run_squad.py:438] token_to_orig_map: 15:0 16:1 17:2 18:3 19:4 20:4 21:4 22:4 23:4 24:5 25:6 26:6 27:7 28:8 29:9 30:10 31:10 32:11 33:12 34:12 35:12 36:13 37:14 38:15 39:16 40:17 41:18 42:19 43:20 44:21 45:22 46:23 47:24 48:24 49:25 50:25 51:25 52:26 53:27 54:28 55:29 56:30 57:31 58:32 59:33 60:34 61:35 62:36 63:37 64:38 65:39 66:40 67:41 68:42 69:43 70:44 71:44 72:45 73:46 74:47 75:48 76:49 77:50 78:51 79:52 80:53 81:54 82:55 83:56 84:57 85:58 86:59 87:60 88:61 89:62 90:63 91:64 92:65 93:66 94:67 95:68 96:69 97:69 98:70 99:71 100:72 101:73 102:74 103:75 104:76 105:77 106:78 107:79 108:79 109:80 110:81 111:82 112:83 113:84 114:85 115:86 116:87 117:88 118:89 119:90 120:91 121:92 122:93 123:94 124:95 125:95
INFO:tensorflow:token_is_max_context: 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True
I1203 11:56:12.671871 139660688828288 run_squad.py:440] token_is_max_context: 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True
INFO:tensorflow:input_ids: 101 2054 2106 5406 16950 5856 4783 2817 1999 6987 2005 2023 2338 1029 102 1999 2010 2338 1996 13675 14194 10128 7646 3258 1997 4441 1010 7522 1998 15359 4130 8662 5406 16950 5856 4783 3273 1996 3497 6214 1997 1996 2331 1997 4441 1999 2307 6987 1012 16950 5856 4783 3344 2041 1037 2193 1997 7885 2058 2195 2086 2000 3231 2010 8106 2096 2002 2001 1037 2966 19684 1012 2122 2913 2443 7885 1999 2029 7314 2007 3563 15871 2020 5689 2012 3563 12113 1998 1996 3815 1997 4139 2006 2169 2192 2001 7594 1010 1999 3572 2073 1996 2519 2020 2036 7119 2030 2025 1012 1999 2122 3572 1996 3815 1997 4139 1998 1996 7978 3255 2001 2179 2000 2022 3278 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.672083 139660688828288 run_squad.py:442] input_ids: 101 2054 2106 5406 16950 5856 4783 2817 1999 6987 2005 2023 2338 1029 102 1999 2010 2338 1996 13675 14194 10128 7646 3258 1997 4441 1010 7522 1998 15359 4130 8662 5406 16950 5856 4783 3273 1996 3497 6214 1997 1996 2331 1997 4441 1999 2307 6987 1012 16950 5856 4783 3344 2041 1037 2193 1997 7885 2058 2195 2086 2000 3231 2010 8106 2096 2002 2001 1037 2966 19684 1012 2122 2913 2443 7885 1999 2029 7314 2007 3563 15871 2020 5689 2012 3563 12113 1998 1996 3815 1997 4139 2006 2169 2192 2001 7594 1010 1999 3572 2073 1996 2519 2020 2036 7119 2030 2025 1012 1999 2122 3572 1996 3815 1997 4139 1998 1996 7978 3255 2001 2179 2000 2022 3278 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.672275 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.672459 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 37
I1203 11:56:12.672558 139660688828288 run_squad.py:451] start_position: 37
INFO:tensorflow:end_position: 44
I1203 11:56:12.672637 139660688828288 run_squad.py:452] end_position: 44
INFO:tensorflow:answer: the likely circumstances of the death of jesus
I1203 11:56:12.672711 139660688828288 run_squad.py:454] answer: the likely circumstances of the death of jesus
INFO:tensorflow:*** Example ***
I1203 11:56:12.676122 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000003
I1203 11:56:12.676302 139660688828288 run_squad.py:432] unique_id: 1000000003
INFO:tensorflow:example_index: 3
I1203 11:56:12.676403 139660688828288 run_squad.py:433] example_index: 3
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.676487 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] how many millions of tourists does greece ban each year ? [SEP] greece attracts more than 16 million tourists each year , thus contributing 18 . 2 % to the nation ' s gdp in 2008 according to an o ##ec ##d report . the same survey showed that the average tourist expenditure while in greece was $ 1 , 07 ##3 , ranking greece 10th in the world . the number of jobs directly or indirectly related to the tourism sector were 840 , 000 in 2008 and represented 19 % of the country ' s total labor force . in 2009 , greece welcomed over 19 . 3 million tourists , a major increase from the 17 . 7 million tourists the country welcomed in 2008 . [SEP]
I1203 11:56:12.676638 139660688828288 run_squad.py:436] tokens: [CLS] how many millions of tourists does greece ban each year ? [SEP] greece attracts more than 16 million tourists each year , thus contributing 18 . 2 % to the nation ' s gdp in 2008 according to an o ##ec ##d report . the same survey showed that the average tourist expenditure while in greece was $ 1 , 07 ##3 , ranking greece 10th in the world . the number of jobs directly or indirectly related to the tourism sector were 840 , 000 in 2008 and represented 19 % of the country ' s total labor force . in 2009 , greece welcomed over 19 . 3 million tourists , a major increase from the 17 . 7 million tourists the country welcomed in 2008 . [SEP]
INFO:tensorflow:token_to_orig_map: 13:0 14:1 15:2 16:3 17:4 18:5 19:6 20:7 21:8 22:8 23:9 24:10 25:11 26:11 27:11 28:11 29:12 30:13 31:14 32:14 33:14 34:15 35:16 36:17 37:18 38:19 39:20 40:21 41:21 42:21 43:22 44:22 45:23 46:24 47:25 48:26 49:27 50:28 51:29 52:30 53:31 54:32 55:33 56:34 57:35 58:36 59:36 60:36 61:36 62:36 63:36 64:37 65:38 66:39 67:40 68:41 69:42 70:42 71:43 72:44 73:45 74:46 75:47 76:48 77:49 78:50 79:51 80:52 81:53 82:54 83:55 84:56 85:56 86:56 87:57 88:58 89:59 90:60 91:61 92:61 93:62 94:63 95:64 96:64 97:64 98:65 99:66 100:67 101:67 102:68 103:69 104:69 105:70 106:71 107:72 108:73 109:73 110:73 111:74 112:75 113:75 114:76 115:77 116:78 117:79 118:80 119:81 120:81 121:81 122:82 123:83 124:84 125:85 126:86 127:87 128:88 129:88
I1203 11:56:12.676805 139660688828288 run_squad.py:438] token_to_orig_map: 13:0 14:1 15:2 16:3 17:4 18:5 19:6 20:7 21:8 22:8 23:9 24:10 25:11 26:11 27:11 28:11 29:12 30:13 31:14 32:14 33:14 34:15 35:16 36:17 37:18 38:19 39:20 40:21 41:21 42:21 43:22 44:22 45:23 46:24 47:25 48:26 49:27 50:28 51:29 52:30 53:31 54:32 55:33 56:34 57:35 58:36 59:36 60:36 61:36 62:36 63:36 64:37 65:38 66:39 67:40 68:41 69:42 70:42 71:43 72:44 73:45 74:46 75:47 76:48 77:49 78:50 79:51 80:52 81:53 82:54 83:55 84:56 85:56 86:56 87:57 88:58 89:59 90:60 91:61 92:61 93:62 94:63 95:64 96:64 97:64 98:65 99:66 100:67 101:67 102:68 103:69 104:69 105:70 106:71 107:72 108:73 109:73 110:73 111:74 112:75 113:75 114:76 115:77 116:78 117:79 118:80 119:81 120:81 121:81 122:82 123:83 124:84 125:85 126:86 127:87 128:88 129:88
INFO:tensorflow:token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True
I1203 11:56:12.676955 139660688828288 run_squad.py:440] token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True
INFO:tensorflow:input_ids: 101 2129 2116 8817 1997 9045 2515 5483 7221 2169 2095 1029 102 5483 17771 2062 2084 2385 2454 9045 2169 2095 1010 2947 8020 2324 1012 1016 1003 2000 1996 3842 1005 1055 14230 1999 2263 2429 2000 2019 1051 8586 2094 3189 1012 1996 2168 5002 3662 2008 1996 2779 7538 20700 2096 1999 5483 2001 1002 1015 1010 5718 2509 1010 5464 5483 6049 1999 1996 2088 1012 1996 2193 1997 5841 3495 2030 17351 3141 2000 1996 6813 4753 2020 28122 1010 2199 1999 2263 1998 3421 2539 1003 1997 1996 2406 1005 1055 2561 4450 2486 1012 1999 2268 1010 5483 10979 2058 2539 1012 1017 2454 9045 1010 1037 2350 3623 2013 1996 2459 1012 1021 2454 9045 1996 2406 10979 1999 2263 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.769079 139660688828288 run_squad.py:442] input_ids: 101 2129 2116 8817 1997 9045 2515 5483 7221 2169 2095 1029 102 5483 17771 2062 2084 2385 2454 9045 2169 2095 1010 2947 8020 2324 1012 1016 1003 2000 1996 3842 1005 1055 14230 1999 2263 2429 2000 2019 1051 8586 2094 3189 1012 1996 2168 5002 3662 2008 1996 2779 7538 20700 2096 1999 5483 2001 1002 1015 1010 5718 2509 1010 5464 5483 6049 1999 1996 2088 1012 1996 2193 1997 5841 3495 2030 17351 3141 2000 1996 6813 4753 2020 28122 1010 2199 1999 2263 1998 3421 2539 1003 1997 1996 2406 1005 1055 2561 4450 2486 1012 1999 2268 1010 5483 10979 2058 2539 1012 1017 2454 9045 1010 1037 2350 3623 2013 1996 2459 1012 1021 2454 9045 1996 2406 10979 1999 2263 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.769501 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.769765 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:impossible example
I1203 11:56:12.769923 139660688828288 run_squad.py:448] impossible example
INFO:tensorflow:*** Example ***
I1203 11:56:12.774670 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000004
I1203 11:56:12.775086 139660688828288 run_squad.py:432] unique_id: 1000000004
INFO:tensorflow:example_index: 4
I1203 11:56:12.775199 139660688828288 run_squad.py:433] example_index: 4
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.775286 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what is the main role of the cabinet of government ministers ? [SEP] when the party is represented by members in the lower house of parliament , the party leader simultaneously serves as the leader of the parliamentary group of that full party representation ; depending on a minimum number of seats held , westminster - based parties typically allow for leaders to form front ##ben ##ch teams of senior fellow members of the parliamentary group to serve as critics of aspects of government policy . when a party becomes the largest party not part of the government , the party ' s parliamentary group forms the official opposition , with official opposition front ##ben ##ch team members often forming the official opposition shadow cabinet . when a party achieve ##s enough seats in an election to form a majority , the party ' s front ##ben ##ch becomes the cabinet of government ministers . [SEP]
I1203 11:56:12.775447 139660688828288 run_squad.py:436] tokens: [CLS] what is the main role of the cabinet of government ministers ? [SEP] when the party is represented by members in the lower house of parliament , the party leader simultaneously serves as the leader of the parliamentary group of that full party representation ; depending on a minimum number of seats held , westminster - based parties typically allow for leaders to form front ##ben ##ch teams of senior fellow members of the parliamentary group to serve as critics of aspects of government policy . when a party becomes the largest party not part of the government , the party ' s parliamentary group forms the official opposition , with official opposition front ##ben ##ch team members often forming the official opposition shadow cabinet . when a party achieve ##s enough seats in an election to form a majority , the party ' s front ##ben ##ch becomes the cabinet of government ministers . [SEP]
INFO:tensorflow:token_to_orig_map: 14:0 15:1 16:2 17:3 18:4 19:5 20:6 21:7 22:8 23:9 24:10 25:11 26:12 27:12 28:13 29:14 30:15 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:26 42:27 43:28 44:29 45:29 46:30 47:31 48:32 49:33 50:34 51:35 52:36 53:37 54:37 55:38 56:38 57:38 58:39 59:40 60:41 61:42 62:43 63:44 64:45 65:46 66:46 67:46 68:47 69:48 70:49 71:50 72:51 73:52 74:53 75:54 76:55 77:56 78:57 79:58 80:59 81:60 82:61 83:62 84:63 85:64 86:64 87:65 88:66 89:67 90:68 91:69 92:70 93:71 94:72 95:73 96:74 97:75 98:76 99:76 100:77 101:78 102:78 103:78 104:79 105:80 106:81 107:82 108:83 109:84 110:84 111:85 112:86 113:87 114:88 115:88 116:88 117:89 118:90 119:91 120:92 121:93 122:94 123:95 124:96 125:97 126:97 127:98 128:99 129:100 130:101 131:101 132:102 133:103 134:104 135:105 136:106 137:107 138:108 139:109 140:110 141:110 142:111 143:112 144:112 145:112 146:113 147:113 148:113 149:114 150:115 151:116 152:117 153:118 154:119 155:119
I1203 11:56:12.775628 139660688828288 run_squad.py:438] token_to_orig_map: 14:0 15:1 16:2 17:3 18:4 19:5 20:6 21:7 22:8 23:9 24:10 25:11 26:12 27:12 28:13 29:14 30:15 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:26 42:27 43:28 44:29 45:29 46:30 47:31 48:32 49:33 50:34 51:35 52:36 53:37 54:37 55:38 56:38 57:38 58:39 59:40 60:41 61:42 62:43 63:44 64:45 65:46 66:46 67:46 68:47 69:48 70:49 71:50 72:51 73:52 74:53 75:54 76:55 77:56 78:57 79:58 80:59 81:60 82:61 83:62 84:63 85:64 86:64 87:65 88:66 89:67 90:68 91:69 92:70 93:71 94:72 95:73 96:74 97:75 98:76 99:76 100:77 101:78 102:78 103:78 104:79 105:80 106:81 107:82 108:83 109:84 110:84 111:85 112:86 113:87 114:88 115:88 116:88 117:89 118:90 119:91 120:92 121:93 122:94 123:95 124:96 125:97 126:97 127:98 128:99 129:100 130:101 131:101 132:102 133:103 134:104 135:105 136:106 137:107 138:108 139:109 140:110 141:110 142:111 143:112 144:112 145:112 146:113 147:113 148:113 149:114 150:115 151:116 152:117 153:118 154:119 155:119
INFO:tensorflow:token_is_max_context: 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True
I1203 11:56:12.775805 139660688828288 run_squad.py:440] token_is_max_context: 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True
INFO:tensorflow:input_ids: 101 2054 2003 1996 2364 2535 1997 1996 5239 1997 2231 7767 1029 102 2043 1996 2283 2003 3421 2011 2372 1999 1996 2896 2160 1997 3323 1010 1996 2283 3003 7453 4240 2004 1996 3003 1997 1996 5768 2177 1997 2008 2440 2283 6630 1025 5834 2006 1037 6263 2193 1997 4272 2218 1010 9434 1011 2241 4243 4050 3499 2005 4177 2000 2433 2392 10609 2818 2780 1997 3026 3507 2372 1997 1996 5768 2177 2000 3710 2004 4401 1997 5919 1997 2231 3343 1012 2043 1037 2283 4150 1996 2922 2283 2025 2112 1997 1996 2231 1010 1996 2283 1005 1055 5768 2177 3596 1996 2880 4559 1010 2007 2880 4559 2392 10609 2818 2136 2372 2411 5716 1996 2880 4559 5192 5239 1012 2043 1037 2283 6162 2015 2438 4272 1999 2019 2602 2000 2433 1037 3484 1010 1996 2283 1005 1055 2392 10609 2818 4150 1996 5239 1997 2231 7767 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.776037 139660688828288 run_squad.py:442] input_ids: 101 2054 2003 1996 2364 2535 1997 1996 5239 1997 2231 7767 1029 102 2043 1996 2283 2003 3421 2011 2372 1999 1996 2896 2160 1997 3323 1010 1996 2283 3003 7453 4240 2004 1996 3003 1997 1996 5768 2177 1997 2008 2440 2283 6630 1025 5834 2006 1037 6263 2193 1997 4272 2218 1010 9434 1011 2241 4243 4050 3499 2005 4177 2000 2433 2392 10609 2818 2780 1997 3026 3507 2372 1997 1996 5768 2177 2000 3710 2004 4401 1997 5919 1997 2231 3343 1012 2043 1037 2283 4150 1996 2922 2283 2025 2112 1997 1996 2231 1010 1996 2283 1005 1055 5768 2177 3596 1996 2880 4559 1010 2007 2880 4559 2392 10609 2818 2136 2372 2411 5716 1996 2880 4559 5192 5239 1012 2043 1037 2283 6162 2015 2438 4272 1999 2019 2602 2000 2433 1037 3484 1010 1996 2283 1005 1055 2392 10609 2818 4150 1996 5239 1997 2231 7767 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.776233 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.776414 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:impossible example
I1203 11:56:12.776512 139660688828288 run_squad.py:448] impossible example
INFO:tensorflow:*** Example ***
I1203 11:56:12.781704 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000005
I1203 11:56:12.781977 139660688828288 run_squad.py:432] unique_id: 1000000005
INFO:tensorflow:example_index: 5
I1203 11:56:12.782079 139660688828288 run_squad.py:433] example_index: 5
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.782165 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what does the fe ##pc stand for ? [SEP] f ##dr ' s new deal programs often contained equal opportunity clauses stating " no discrimination shall be made on account of race , color or creed " , : 11 but the true forerunner to affirmative action was the interior secretary of the time , harold l . ic ##kes . ic ##kes prohibited discrimination in hiring for public works administration funded projects and oversaw not only the institution of a quota system , where contractors were required to employ a fixed percentage of black workers , by robert c . weaver and clark foreman , : 12 but also the equal pay of women proposed by harry hopkins . : 14 ##f ##dr ' s largest contribution to affirmative action , however , lay in his executive order 880 ##2 which prohibited discrimination in the defense industry or government . : 22 the executive order promoted the idea that if taxpayer funds were accepted through a government contract , then all taxpayers should have an equal opportunity to work through the contractor . : 23 – 4 to enforce this idea , roosevelt created the fair employment practices committee ( fe ##pc ) with the power to investigate hiring practices by government contractors . : 22 [SEP]
I1203 11:56:12.782344 139660688828288 run_squad.py:436] tokens: [CLS] what does the fe ##pc stand for ? [SEP] f ##dr ' s new deal programs often contained equal opportunity clauses stating " no discrimination shall be made on account of race , color or creed " , : 11 but the true forerunner to affirmative action was the interior secretary of the time , harold l . ic ##kes . ic ##kes prohibited discrimination in hiring for public works administration funded projects and oversaw not only the institution of a quota system , where contractors were required to employ a fixed percentage of black workers , by robert c . weaver and clark foreman , : 12 but also the equal pay of women proposed by harry hopkins . : 14 ##f ##dr ' s largest contribution to affirmative action , however , lay in his executive order 880 ##2 which prohibited discrimination in the defense industry or government . : 22 the executive order promoted the idea that if taxpayer funds were accepted through a government contract , then all taxpayers should have an equal opportunity to work through the contractor . : 23 – 4 to enforce this idea , roosevelt created the fair employment practices committee ( fe ##pc ) with the power to investigate hiring practices by government contractors . : 22 [SEP]
INFO:tensorflow:token_to_orig_map: 10:0 11:0 12:0 13:0 14:1 15:2 16:3 17:4 18:5 19:6 20:7 21:8 22:9 23:10 24:10 25:11 26:12 27:13 28:14 29:15 30:16 31:17 32:18 33:18 34:19 35:20 36:21 37:21 38:21 39:21 40:21 41:22 42:23 43:24 44:25 45:26 46:27 47:28 48:29 49:30 50:31 51:32 52:33 53:34 54:35 55:35 56:36 57:37 58:37 59:38 60:38 61:38 62:39 63:39 64:40 65:41 66:42 67:43 68:44 69:45 70:46 71:47 72:48 73:49 74:50 75:51 76:52 77:53 78:54 79:55 80:56 81:57 82:58 83:59 84:59 85:60 86:61 87:62 88:63 89:64 90:65 91:66 92:67 93:68 94:69 95:70 96:71 97:71 98:72 99:73 100:74 101:74 102:75 103:76 104:77 105:78 106:78 107:78 108:78 109:79 110:80 111:81 112:82 113:83 114:84 115:85 116:86 117:87 118:88 119:89 120:89 121:89 122:89 123:89 124:89 125:89 126:89 127:90 128:91 129:92 130:93 131:94 132:94 133:95 134:95 135:96 136:97 137:98 138:99 139:100 140:101 141:101 142:102 143:103 144:104 145:105 146:106 147:107 148:108 149:109 150:110 151:110 152:110 153:110 154:111 155:112 156:113 157:114 158:115 159:116 160:117 161:118 162:119 163:120 164:121 165:122 166:123 167:124 168:125 169:126 170:126 171:127 172:128 173:129 174:130 175:131 176:132 177:133 178:134 179:135 180:136 181:137 182:138 183:139 184:139 185:139 186:139 187:139 188:139 189:140 190:141 191:142 192:143 193:143 194:144 195:145 196:146 197:147 198:148 199:149 200:150 201:151 202:151 203:151 204:151 205:152 206:153 207:154 208:155 209:156 210:157 211:158 212:159 213:160 214:161 215:161 216:161 217:161
I1203 11:56:12.782551 139660688828288 run_squad.py:438] token_to_orig_map: 10:0 11:0 12:0 13:0 14:1 15:2 16:3 17:4 18:5 19:6 20:7 21:8 22:9 23:10 24:10 25:11 26:12 27:13 28:14 29:15 30:16 31:17 32:18 33:18 34:19 35:20 36:21 37:21 38:21 39:21 40:21 41:22 42:23 43:24 44:25 45:26 46:27 47:28 48:29 49:30 50:31 51:32 52:33 53:34 54:35 55:35 56:36 57:37 58:37 59:38 60:38 61:38 62:39 63:39 64:40 65:41 66:42 67:43 68:44 69:45 70:46 71:47 72:48 73:49 74:50 75:51 76:52 77:53 78:54 79:55 80:56 81:57 82:58 83:59 84:59 85:60 86:61 87:62 88:63 89:64 90:65 91:66 92:67 93:68 94:69 95:70 96:71 97:71 98:72 99:73 100:74 101:74 102:75 103:76 104:77 105:78 106:78 107:78 108:78 109:79 110:80 111:81 112:82 113:83 114:84 115:85 116:86 117:87 118:88 119:89 120:89 121:89 122:89 123:89 124:89 125:89 126:89 127:90 128:91 129:92 130:93 131:94 132:94 133:95 134:95 135:96 136:97 137:98 138:99 139:100 140:101 141:101 142:102 143:103 144:104 145:105 146:106 147:107 148:108 149:109 150:110 151:110 152:110 153:110 154:111 155:112 156:113 157:114 158:115 159:116 160:117 161:118 162:119 163:120 164:121 165:122 166:123 167:124 168:125 169:126 170:126 171:127 172:128 173:129 174:130 175:131 176:132 177:133 178:134 179:135 180:136 181:137 182:138 183:139 184:139 185:139 186:139 187:139 188:139 189:140 190:141 191:142 192:143 193:143 194:144 195:145 196:146 197:147 198:148 199:149 200:150 201:151 202:151 203:151 204:151 205:152 206:153 207:154 208:155 209:156 210:157 211:158 212:159 213:160 214:161 215:161 216:161 217:161
INFO:tensorflow:token_is_max_context: 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True 205:True 206:True 207:True 208:True 209:True 210:True 211:True 212:True 213:True 214:True 215:True 216:True 217:True
I1203 11:56:12.874029 139660688828288 run_squad.py:440] token_is_max_context: 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True 205:True 206:True 207:True 208:True 209:True 210:True 211:True 212:True 213:True 214:True 215:True 216:True 217:True
INFO:tensorflow:input_ids: 101 2054 2515 1996 10768 15042 3233 2005 1029 102 1042 13626 1005 1055 2047 3066 3454 2411 4838 5020 4495 24059 5517 1000 2053 9147 4618 2022 2081 2006 4070 1997 2679 1010 3609 2030 16438 1000 1010 1024 2340 2021 1996 2995 23993 2000 27352 2895 2001 1996 4592 3187 1997 1996 2051 1010 7157 1048 1012 24582 9681 1012 24582 9681 10890 9147 1999 14763 2005 2270 2573 3447 6787 3934 1998 14105 2025 2069 1996 5145 1997 1037 20563 2291 1010 2073 16728 2020 3223 2000 12666 1037 4964 7017 1997 2304 3667 1010 2011 2728 1039 1012 14077 1998 5215 18031 1010 1024 2260 2021 2036 1996 5020 3477 1997 2308 3818 2011 4302 10239 1012 1024 2403 2546 13626 1005 1055 2922 6691 2000 27352 2895 1010 2174 1010 3913 1999 2010 3237 2344 26839 2475 2029 10890 9147 1999 1996 3639 3068 2030 2231 1012 1024 2570 1996 3237 2344 3755 1996 2801 2008 2065 26980 5029 2020 3970 2083 1037 2231 3206 1010 2059 2035 26457 2323 2031 2019 5020 4495 2000 2147 2083 1996 13666 1012 1024 2603 1516 1018 2000 16306 2023 2801 1010 8573 2580 1996 4189 6107 6078 2837 1006 10768 15042 1007 2007 1996 2373 2000 8556 14763 6078 2011 2231 16728 1012 1024 2570 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.874902 139660688828288 run_squad.py:442] input_ids: 101 2054 2515 1996 10768 15042 3233 2005 1029 102 1042 13626 1005 1055 2047 3066 3454 2411 4838 5020 4495 24059 5517 1000 2053 9147 4618 2022 2081 2006 4070 1997 2679 1010 3609 2030 16438 1000 1010 1024 2340 2021 1996 2995 23993 2000 27352 2895 2001 1996 4592 3187 1997 1996 2051 1010 7157 1048 1012 24582 9681 1012 24582 9681 10890 9147 1999 14763 2005 2270 2573 3447 6787 3934 1998 14105 2025 2069 1996 5145 1997 1037 20563 2291 1010 2073 16728 2020 3223 2000 12666 1037 4964 7017 1997 2304 3667 1010 2011 2728 1039 1012 14077 1998 5215 18031 1010 1024 2260 2021 2036 1996 5020 3477 1997 2308 3818 2011 4302 10239 1012 1024 2403 2546 13626 1005 1055 2922 6691 2000 27352 2895 1010 2174 1010 3913 1999 2010 3237 2344 26839 2475 2029 10890 9147 1999 1996 3639 3068 2030 2231 1012 1024 2570 1996 3237 2344 3755 1996 2801 2008 2065 26980 5029 2020 3970 2083 1037 2231 3206 1010 2059 2035 26457 2323 2031 2019 5020 4495 2000 2147 2083 1996 13666 1012 1024 2603 1516 1018 2000 16306 2023 2801 1010 8573 2580 1996 4189 6107 6078 2837 1006 10768 15042 1007 2007 1996 2373 2000 8556 14763 6078 2011 2231 16728 1012 1024 2570 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.875903 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.876454 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 197
I1203 11:56:12.882365 139660688828288 run_squad.py:451] start_position: 197
INFO:tensorflow:end_position: 200
I1203 11:56:12.882695 139660688828288 run_squad.py:452] end_position: 200
INFO:tensorflow:answer: fair employment practices committee
I1203 11:56:12.883368 139660688828288 run_squad.py:454] answer: fair employment practices committee
INFO:tensorflow:*** Example ***
I1203 11:56:12.886117 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000006
I1203 11:56:12.886294 139660688828288 run_squad.py:432] unique_id: 1000000006
INFO:tensorflow:example_index: 6
I1203 11:56:12.886396 139660688828288 run_squad.py:433] example_index: 6
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.886490 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] the structures that still remain are open to whom ? [SEP] because it was designated as the national capital , many structures were built around that time . even today , some of them still remain which are open to tourists . [SEP]
I1203 11:56:12.886582 139660688828288 run_squad.py:436] tokens: [CLS] the structures that still remain are open to whom ? [SEP] because it was designated as the national capital , many structures were built around that time . even today , some of them still remain which are open to tourists . [SEP]
INFO:tensorflow:token_to_orig_map: 12:0 13:1 14:2 15:3 16:4 17:5 18:6 19:7 20:7 21:8 22:9 23:10 24:11 25:12 26:13 27:14 28:14 29:15 30:16 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:26 42:26
I1203 11:56:12.886659 139660688828288 run_squad.py:438] token_to_orig_map: 12:0 13:1 14:2 15:3 16:4 17:5 18:6 19:7 20:7 21:8 22:9 23:10 24:11 25:12 26:13 27:14 28:14 29:15 30:16 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:26 42:26
INFO:tensorflow:token_is_max_context: 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True
I1203 11:56:12.886725 139660688828288 run_squad.py:440] token_is_max_context: 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True
INFO:tensorflow:input_ids: 101 1996 5090 2008 2145 3961 2024 2330 2000 3183 1029 102 2138 2009 2001 4351 2004 1996 2120 3007 1010 2116 5090 2020 2328 2105 2008 2051 1012 2130 2651 1010 2070 1997 2068 2145 3961 2029 2024 2330 2000 9045 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.886914 139660688828288 run_squad.py:442] input_ids: 101 1996 5090 2008 2145 3961 2024 2330 2000 3183 1029 102 2138 2009 2001 4351 2004 1996 2120 3007 1010 2116 5090 2020 2328 2105 2008 2051 1012 2130 2651 1010 2070 1997 2068 2145 3961 2029 2024 2330 2000 9045 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.887069 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.887193 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 41
I1203 11:56:12.887250 139660688828288 run_squad.py:451] start_position: 41
INFO:tensorflow:end_position: 41
I1203 11:56:12.887299 139660688828288 run_squad.py:452] end_position: 41
INFO:tensorflow:answer: tourists
I1203 11:56:12.887348 139660688828288 run_squad.py:454] answer: tourists
INFO:tensorflow:*** Example ***
I1203 11:56:12.890726 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000007
I1203 11:56:12.890910 139660688828288 run_squad.py:432] unique_id: 1000000007
INFO:tensorflow:example_index: 7
I1203 11:56:12.890972 139660688828288 run_squad.py:433] example_index: 7
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.891025 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] when was the mann act passed ? [SEP] the bureau ' s first official task was visiting and making surveys of the houses of prostitution in preparation for enforcing the " white slave traffic act , " or mann act , passed on june 25 , 1910 . in 1932 , it was renamed the united states bureau of investigation . the following year it was linked to the bureau of prohibition and rec ##hri ##sten ##ed the division of investigation ( doi ) before finally becoming an independent service within the department of justice in 1935 . in the same year , its name was officially changed from the division of investigation to the present - day federal bureau of investigation , or fbi . [SEP]
I1203 11:56:12.891119 139660688828288 run_squad.py:436] tokens: [CLS] when was the mann act passed ? [SEP] the bureau ' s first official task was visiting and making surveys of the houses of prostitution in preparation for enforcing the " white slave traffic act , " or mann act , passed on june 25 , 1910 . in 1932 , it was renamed the united states bureau of investigation . the following year it was linked to the bureau of prohibition and rec ##hri ##sten ##ed the division of investigation ( doi ) before finally becoming an independent service within the department of justice in 1935 . in the same year , its name was officially changed from the division of investigation to the present - day federal bureau of investigation , or fbi . [SEP]
INFO:tensorflow:token_to_orig_map: 9:0 10:1 11:1 12:1 13:2 14:3 15:4 16:5 17:6 18:7 19:8 20:9 21:10 22:11 23:12 24:13 25:14 26:15 27:16 28:17 29:18 30:19 31:20 32:20 33:21 34:22 35:23 36:23 37:23 38:24 39:25 40:26 41:26 42:27 43:28 44:29 45:30 46:30 47:31 48:31 49:32 50:33 51:33 52:34 53:35 54:36 55:37 56:38 57:39 58:40 59:41 60:42 61:42 62:43 63:44 64:45 65:46 66:47 67:48 68:49 69:50 70:51 71:52 72:53 73:54 74:55 75:55 76:55 77:55 78:56 79:57 80:58 81:59 82:60 83:60 84:60 85:61 86:62 87:63 88:64 89:65 90:66 91:67 92:68 93:69 94:70 95:71 96:72 97:73 98:73 99:74 100:75 101:76 102:77 103:77 104:78 105:79 106:80 107:81 108:82 109:83 110:84 111:85 112:86 113:87 114:88 115:89 116:90 117:90 118:90 119:91 120:92 121:93 122:94 123:94 124:95 125:96 126:96
I1203 11:56:12.891217 139660688828288 run_squad.py:438] token_to_orig_map: 9:0 10:1 11:1 12:1 13:2 14:3 15:4 16:5 17:6 18:7 19:8 20:9 21:10 22:11 23:12 24:13 25:14 26:15 27:16 28:17 29:18 30:19 31:20 32:20 33:21 34:22 35:23 36:23 37:23 38:24 39:25 40:26 41:26 42:27 43:28 44:29 45:30 46:30 47:31 48:31 49:32 50:33 51:33 52:34 53:35 54:36 55:37 56:38 57:39 58:40 59:41 60:42 61:42 62:43 63:44 64:45 65:46 66:47 67:48 68:49 69:50 70:51 71:52 72:53 73:54 74:55 75:55 76:55 77:55 78:56 79:57 80:58 81:59 82:60 83:60 84:60 85:61 86:62 87:63 88:64 89:65 90:66 91:67 92:68 93:69 94:70 95:71 96:72 97:73 98:73 99:74 100:75 101:76 102:77 103:77 104:78 105:79 106:80 107:81 108:82 109:83 110:84 111:85 112:86 113:87 114:88 115:89 116:90 117:90 118:90 119:91 120:92 121:93 122:94 123:94 124:95 125:96 126:96
INFO:tensorflow:token_is_max_context: 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True
I1203 11:56:12.891305 139660688828288 run_squad.py:440] token_is_max_context: 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True
INFO:tensorflow:input_ids: 101 2043 2001 1996 10856 2552 2979 1029 102 1996 4879 1005 1055 2034 2880 4708 2001 5873 1998 2437 12265 1997 1996 3506 1997 15016 1999 7547 2005 27455 1996 1000 2317 6658 4026 2552 1010 1000 2030 10856 2552 1010 2979 2006 2238 2423 1010 4976 1012 1999 4673 1010 2009 2001 4096 1996 2142 2163 4879 1997 4812 1012 1996 2206 2095 2009 2001 5799 2000 1996 4879 1997 13574 1998 28667 26378 16173 2098 1996 2407 1997 4812 1006 9193 1007 2077 2633 3352 2019 2981 2326 2306 1996 2533 1997 3425 1999 4437 1012 1999 1996 2168 2095 1010 2049 2171 2001 3985 2904 2013 1996 2407 1997 4812 2000 1996 2556 1011 2154 2976 4879 1997 4812 1010 2030 8495 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.891451 139660688828288 run_squad.py:442] input_ids: 101 2043 2001 1996 10856 2552 2979 1029 102 1996 4879 1005 1055 2034 2880 4708 2001 5873 1998 2437 12265 1997 1996 3506 1997 15016 1999 7547 2005 27455 1996 1000 2317 6658 4026 2552 1010 1000 2030 10856 2552 1010 2979 2006 2238 2423 1010 4976 1012 1999 4673 1010 2009 2001 4096 1996 2142 2163 4879 1997 4812 1012 1996 2206 2095 2009 2001 5799 2000 1996 4879 1997 13574 1998 28667 26378 16173 2098 1996 2407 1997 4812 1006 9193 1007 2077 2633 3352 2019 2981 2326 2306 1996 2533 1997 3425 1999 4437 1012 1999 1996 2168 2095 1010 2049 2171 2001 3985 2904 2013 1996 2407 1997 4812 2000 1996 2556 1011 2154 2976 4879 1997 4812 1010 2030 8495 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.891578 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.985524 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 44
I1203 11:56:12.985891 139660688828288 run_squad.py:451] start_position: 44
INFO:tensorflow:end_position: 47
I1203 11:56:12.986228 139660688828288 run_squad.py:452] end_position: 47
INFO:tensorflow:answer: june 25 , 1910
I1203 11:56:12.986340 139660688828288 run_squad.py:454] answer: june 25 , 1910
INFO:tensorflow:*** Example ***
I1203 11:56:12.990844 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000008
I1203 11:56:12.991119 139660688828288 run_squad.py:432] unique_id: 1000000008
INFO:tensorflow:example_index: 8
I1203 11:56:12.991503 139660688828288 run_squad.py:433] example_index: 8
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.991618 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what is the result of / e / being the same in central , western and bel ##ear ##ic ? [SEP] central , western , and bal ##ear ##ic differ in the lexi ##cal incidence of stressed / e / and / ɛ / . usually , words with / ɛ / in central catalan correspond to / ə / in bal ##ear ##ic and / e / in western catalan . words with / e / in bal ##ear ##ic almost always have / e / in central and western catalan as well . [ vague ] as a result , central catalan has a much higher incidence of / e / . [SEP]
I1203 11:56:12.991807 139660688828288 run_squad.py:436] tokens: [CLS] what is the result of / e / being the same in central , western and bel ##ear ##ic ? [SEP] central , western , and bal ##ear ##ic differ in the lexi ##cal incidence of stressed / e / and / ɛ / . usually , words with / ɛ / in central catalan correspond to / ə / in bal ##ear ##ic and / e / in western catalan . words with / e / in bal ##ear ##ic almost always have / e / in central and western catalan as well . [ vague ] as a result , central catalan has a much higher incidence of / e / . [SEP]
INFO:tensorflow:token_to_orig_map: 22:0 23:0 24:1 25:1 26:2 27:3 28:3 29:3 30:4 31:5 32:6 33:7 34:7 35:8 36:9 37:10 38:11 39:11 40:11 41:12 42:13 43:13 44:13 45:13 46:14 47:14 48:15 49:16 50:17 51:17 52:17 53:18 54:19 55:20 56:21 57:22 58:23 59:23 60:23 61:24 62:25 63:25 64:25 65:26 66:27 67:27 68:27 69:28 70:29 71:30 72:30 73:31 74:32 75:33 76:33 77:33 78:34 79:35 80:35 81:35 82:36 83:37 84:38 85:39 86:39 87:39 88:40 89:41 90:42 91:43 92:44 93:45 94:46 95:46 96:46 97:46 98:46 99:47 100:48 101:49 102:49 103:50 104:51 105:52 106:53 107:54 108:55 109:56 110:57 111:58 112:58 113:58 114:58
I1203 11:56:12.991982 139660688828288 run_squad.py:438] token_to_orig_map: 22:0 23:0 24:1 25:1 26:2 27:3 28:3 29:3 30:4 31:5 32:6 33:7 34:7 35:8 36:9 37:10 38:11 39:11 40:11 41:12 42:13 43:13 44:13 45:13 46:14 47:14 48:15 49:16 50:17 51:17 52:17 53:18 54:19 55:20 56:21 57:22 58:23 59:23 60:23 61:24 62:25 63:25 64:25 65:26 66:27 67:27 68:27 69:28 70:29 71:30 72:30 73:31 74:32 75:33 76:33 77:33 78:34 79:35 80:35 81:35 82:36 83:37 84:38 85:39 86:39 87:39 88:40 89:41 90:42 91:43 92:44 93:45 94:46 95:46 96:46 97:46 98:46 99:47 100:48 101:49 102:49 103:50 104:51 105:52 106:53 107:54 108:55 109:56 110:57 111:58 112:58 113:58 114:58
INFO:tensorflow:token_is_max_context: 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True
I1203 11:56:12.992122 139660688828288 run_squad.py:440] token_is_max_context: 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True
INFO:tensorflow:input_ids: 101 2054 2003 1996 2765 1997 1013 1041 1013 2108 1996 2168 1999 2430 1010 2530 1998 19337 14644 2594 1029 102 2430 1010 2530 1010 1998 28352 14644 2594 11234 1999 1996 16105 9289 18949 1997 13233 1013 1041 1013 1998 1013 1115 1013 1012 2788 1010 2616 2007 1013 1115 1013 1999 2430 13973 17254 2000 1013 1114 1013 1999 28352 14644 2594 1998 1013 1041 1013 1999 2530 13973 1012 2616 2007 1013 1041 1013 1999 28352 14644 2594 2471 2467 2031 1013 1041 1013 1999 2430 1998 2530 13973 2004 2092 1012 1031 13727 1033 2004 1037 2765 1010 2430 13973 2038 1037 2172 3020 18949 1997 1013 1041 1013 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.992342 139660688828288 run_squad.py:442] input_ids: 101 2054 2003 1996 2765 1997 1013 1041 1013 2108 1996 2168 1999 2430 1010 2530 1998 19337 14644 2594 1029 102 2430 1010 2530 1010 1998 28352 14644 2594 11234 1999 1996 16105 9289 18949 1997 13233 1013 1041 1013 1998 1013 1115 1013 1012 2788 1010 2616 2007 1013 1115 1013 1999 2430 13973 17254 2000 1013 1114 1013 1999 28352 14644 2594 1998 1013 1041 1013 1999 2530 13973 1012 2616 2007 1013 1041 1013 1999 28352 14644 2594 2471 2467 2031 1013 1041 1013 1999 2430 1998 2530 13973 2004 2092 1012 1031 13727 1033 2004 1037 2765 1010 2430 13973 2038 1037 2172 3020 18949 1997 1013 1041 1013 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.992523 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:12.992708 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 108
I1203 11:56:12.993147 139660688828288 run_squad.py:451] start_position: 108
INFO:tensorflow:end_position: 109
I1203 11:56:12.993295 139660688828288 run_squad.py:452] end_position: 109
INFO:tensorflow:answer: higher incidence
I1203 11:56:12.993401 139660688828288 run_squad.py:454] answer: higher incidence
INFO:tensorflow:*** Example ***
I1203 11:56:12.997355 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000009
I1203 11:56:12.997809 139660688828288 run_squad.py:432] unique_id: 1000000009
INFO:tensorflow:example_index: 9
I1203 11:56:12.998009 139660688828288 run_squad.py:433] example_index: 9
INFO:tensorflow:doc_span_index: 0
I1203 11:56:12.998144 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what did the old letter ⟨ [UNK] ⟩ become ? [SEP] older letters of the russian alphabet include ⟨ [UNK] ⟩ , which merged to ⟨ е ⟩ ( / je / or / ʲ ##e / ) ; ⟨ і ⟩ and ⟨ [UNK] ⟩ , which both merged to ⟨ и ⟩ ( / i / ) ; ⟨ [UNK] ⟩ , which merged to ⟨ ф ⟩ ( / f / ) ; ⟨ [UNK] ⟩ , which merged to ⟨ у ⟩ ( / u / ) ; ⟨ [UNK] ⟩ , which merged to ⟨ ю ⟩ ( / ju / or / ʲ ##u / ) ; and ⟨ [UNK] / ⟨ [UNK] ⟩ ⟩ , which later were graphical ##ly res ##ha ##ped into ⟨ я ⟩ and merged phonetic ##ally to / ja / or / ʲ ##a / . while these older letters have been abandoned at one time or another , they may be used in this and related articles . the yer ##s ⟨ ъ ⟩ and ⟨ ь ⟩ originally indicated the pronunciation of ultra - short or reduced / u / , / i / . [SEP]
I1203 11:56:12.998329 139660688828288 run_squad.py:436] tokens: [CLS] what did the old letter ⟨ [UNK] ⟩ become ? [SEP] older letters of the russian alphabet include ⟨ [UNK] ⟩ , which merged to ⟨ е ⟩ ( / je / or / ʲ ##e / ) ; ⟨ і ⟩ and ⟨ [UNK] ⟩ , which both merged to ⟨ и ⟩ ( / i / ) ; ⟨ [UNK] ⟩ , which merged to ⟨ ф ⟩ ( / f / ) ; ⟨ [UNK] ⟩ , which merged to ⟨ у ⟩ ( / u / ) ; ⟨ [UNK] ⟩ , which merged to ⟨ ю ⟩ ( / ju / or / ʲ ##u / ) ; and ⟨ [UNK] / ⟨ [UNK] ⟩ ⟩ , which later were graphical ##ly res ##ha ##ped into ⟨ я ⟩ and merged phonetic ##ally to / ja / or / ʲ ##a / . while these older letters have been abandoned at one time or another , they may be used in this and related articles . the yer ##s ⟨ ъ ⟩ and ⟨ ь ⟩ originally indicated the pronunciation of ultra - short or reduced / u / , / i / . [SEP]
INFO:tensorflow:token_to_orig_map: 12:0 13:1 14:2 15:3 16:4 17:5 18:6 19:7 20:7 21:7 22:7 23:8 24:9 25:10 26:11 27:11 28:11 29:12 30:12 31:12 32:12 33:13 34:14 35:14 36:14 37:14 38:14 39:14 40:15 41:15 42:15 43:16 44:17 45:17 46:17 47:17 48:18 49:19 50:20 51:21 52:22 53:22 54:22 55:23 56:23 57:23 58:23 59:23 60:23 61:24 62:24 63:24 64:24 65:25 66:26 67:27 68:28 69:28 70:28 71:29 72:29 73:29 74:29 75:29 76:29 77:30 78:30 79:30 80:30 81:31 82:32 83:33 84:34 85:34 86:34 87:35 88:35 89:35 90:35 91:35 92:35 93:36 94:36 95:36 96:36 97:37 98:38 99:39 100:40 101:40 102:40 103:41 104:41 105:41 106:41 107:42 108:43 109:43 110:43 111:43 112:43 113:43 114:44 115:45 116:45 117:45 118:45 119:45 120:45 121:45 122:45 123:46 124:47 125:48 126:49 127:49 128:50 129:50 130:50 131:51 132:52 133:52 134:52 135:53 136:54 137:55 138:55 139:56 140:57 141:57 142:57 143:58 144:59 145:59 146:59 147:59 148:59 149:60 150:61 151:62 152:63 153:64 154:65 155:66 156:67 157:68 158:69 159:70 160:71 161:71 162:72 163:73 164:74 165:75 166:76 167:77 168:78 169:79 170:80 171:80 172:81 173:82 174:82 175:83 176:83 177:83 178:84 179:85 180:85 181:85 182:86 183:87 184:88 185:89 186:90 187:91 188:91 189:91 190:92 191:93 192:94 193:94 194:94 195:94 196:95 197:95 198:95 199:95
I1203 11:56:12.998524 139660688828288 run_squad.py:438] token_to_orig_map: 12:0 13:1 14:2 15:3 16:4 17:5 18:6 19:7 20:7 21:7 22:7 23:8 24:9 25:10 26:11 27:11 28:11 29:12 30:12 31:12 32:12 33:13 34:14 35:14 36:14 37:14 38:14 39:14 40:15 41:15 42:15 43:16 44:17 45:17 46:17 47:17 48:18 49:19 50:20 51:21 52:22 53:22 54:22 55:23 56:23 57:23 58:23 59:23 60:23 61:24 62:24 63:24 64:24 65:25 66:26 67:27 68:28 69:28 70:28 71:29 72:29 73:29 74:29 75:29 76:29 77:30 78:30 79:30 80:30 81:31 82:32 83:33 84:34 85:34 86:34 87:35 88:35 89:35 90:35 91:35 92:35 93:36 94:36 95:36 96:36 97:37 98:38 99:39 100:40 101:40 102:40 103:41 104:41 105:41 106:41 107:42 108:43 109:43 110:43 111:43 112:43 113:43 114:44 115:45 116:45 117:45 118:45 119:45 120:45 121:45 122:45 123:46 124:47 125:48 126:49 127:49 128:50 129:50 130:50 131:51 132:52 133:52 134:52 135:53 136:54 137:55 138:55 139:56 140:57 141:57 142:57 143:58 144:59 145:59 146:59 147:59 148:59 149:60 150:61 151:62 152:63 153:64 154:65 155:66 156:67 157:68 158:69 159:70 160:71 161:71 162:72 163:73 164:74 165:75 166:76 167:77 168:78 169:79 170:80 171:80 172:81 173:82 174:82 175:83 176:83 177:83 178:84 179:85 180:85 181:85 182:86 183:87 184:88 185:89 186:90 187:91 188:91 189:91 190:92 191:93 192:94 193:94 194:94 195:94 196:95 197:95 198:95 199:95
INFO:tensorflow:token_is_max_context: 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True
I1203 11:56:12.998707 139660688828288 run_squad.py:440] token_is_max_context: 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True
INFO:tensorflow:input_ids: 101 2054 2106 1996 2214 3661 1629 100 1630 2468 1029 102 3080 4144 1997 1996 2845 12440 2421 1629 100 1630 1010 2029 5314 2000 1629 1185 1630 1006 1013 15333 1013 2030 1013 1141 2063 1013 1007 1025 1629 1213 1630 1998 1629 100 1630 1010 2029 2119 5314 2000 1629 1188 1630 1006 1013 1045 1013 1007 1025 1629 100 1630 1010 2029 5314 2000 1629 1199 1630 1006 1013 1042 1013 1007 1025 1629 100 1630 1010 2029 5314 2000 1629 1198 1630 1006 1013 1057 1013 1007 1025 1629 100 1630 1010 2029 5314 2000 1629 1209 1630 1006 1013 18414 1013 2030 1013 1141 2226 1013 1007 1025 1998 1629 100 1013 1629 100 1630 1630 1010 2029 2101 2020 20477 2135 24501 3270 5669 2046 1629 1210 1630 1998 5314 26664 3973 2000 1013 14855 1013 2030 1013 1141 2050 1013 1012 2096 2122 3080 4144 2031 2042 4704 2012 2028 2051 2030 2178 1010 2027 2089 2022 2109 1999 2023 1998 3141 4790 1012 1996 20416 2015 1629 1205 1630 1998 1629 1207 1630 2761 5393 1996 15498 1997 11087 1011 2460 2030 4359 1013 1057 1013 1010 1013 1045 1013 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.092038 139660688828288 run_squad.py:442] input_ids: 101 2054 2106 1996 2214 3661 1629 100 1630 2468 1029 102 3080 4144 1997 1996 2845 12440 2421 1629 100 1630 1010 2029 5314 2000 1629 1185 1630 1006 1013 15333 1013 2030 1013 1141 2063 1013 1007 1025 1629 1213 1630 1998 1629 100 1630 1010 2029 2119 5314 2000 1629 1188 1630 1006 1013 1045 1013 1007 1025 1629 100 1630 1010 2029 5314 2000 1629 1199 1630 1006 1013 1042 1013 1007 1025 1629 100 1630 1010 2029 5314 2000 1629 1198 1630 1006 1013 1057 1013 1007 1025 1629 100 1630 1010 2029 5314 2000 1629 1209 1630 1006 1013 18414 1013 2030 1013 1141 2226 1013 1007 1025 1998 1629 100 1013 1629 100 1630 1630 1010 2029 2101 2020 20477 2135 24501 3270 5669 2046 1629 1210 1630 1998 5314 26664 3973 2000 1013 14855 1013 2030 1013 1141 2050 1013 1012 2096 2122 3080 4144 2031 2042 4704 2012 2028 2051 2030 2178 1010 2027 2089 2022 2109 1999 2023 1998 3141 4790 1012 1996 20416 2015 1629 1205 1630 1998 1629 1207 1630 2761 5393 1996 15498 1997 11087 1011 2460 2030 4359 1013 1057 1013 1010 1013 1045 1013 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.092394 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.092619 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 26
I1203 11:56:13.092739 139660688828288 run_squad.py:451] start_position: 26
INFO:tensorflow:end_position: 28
I1203 11:56:13.092863 139660688828288 run_squad.py:452] end_position: 28
INFO:tensorflow:answer: ⟨ е ⟩
I1203 11:56:13.092951 139660688828288 run_squad.py:454] answer: ⟨ е ⟩
INFO:tensorflow:*** Example ***
I1203 11:56:13.101715 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000010
I1203 11:56:13.102045 139660688828288 run_squad.py:432] unique_id: 1000000010
INFO:tensorflow:example_index: 10
I1203 11:56:13.102162 139660688828288 run_squad.py:433] example_index: 10
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.102251 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what was the problem with the palace ' s chimneys ? [SEP] buckingham palace finally became the principal royal residence in 1837 , on the accession of queen victoria , who was the first monarch to reside there ; her predecessor william iv had died before its completion . while the state rooms were a riot of gil ##t and colour , the nec ##ess ##ities of the new palace were somewhat less luxurious . for one thing , it was reported the chimneys smoked so much that the fires had to be allowed to die down , and consequently the court shivered in icy mag ##ni ##fi ##cence . ventilation was so bad that the interior smelled , and when a decision was taken to install gas lamps , there was a serious worry about the build - up of gas on the lower floors . it was also said that staff were lax and lazy and the palace was dirty . following the queen ' s marriage in 1840 , her husband , prince albert , concerned himself with a reorganisation of the household offices and staff , and with the design faults of the palace . the problems were all rec ##ti ##fied by the close of 1840 . however , the builders were to return within the decade . [SEP]
I1203 11:56:13.102448 139660688828288 run_squad.py:436] tokens: [CLS] what was the problem with the palace ' s chimneys ? [SEP] buckingham palace finally became the principal royal residence in 1837 , on the accession of queen victoria , who was the first monarch to reside there ; her predecessor william iv had died before its completion . while the state rooms were a riot of gil ##t and colour , the nec ##ess ##ities of the new palace were somewhat less luxurious . for one thing , it was reported the chimneys smoked so much that the fires had to be allowed to die down , and consequently the court shivered in icy mag ##ni ##fi ##cence . ventilation was so bad that the interior smelled , and when a decision was taken to install gas lamps , there was a serious worry about the build - up of gas on the lower floors . it was also said that staff were lax and lazy and the palace was dirty . following the queen ' s marriage in 1840 , her husband , prince albert , concerned himself with a reorganisation of the household offices and staff , and with the design faults of the palace . the problems were all rec ##ti ##fied by the close of 1840 . however , the builders were to return within the decade . [SEP]
INFO:tensorflow:token_to_orig_map: 13:0 14:1 15:2 16:3 17:4 18:5 19:6 20:7 21:8 22:9 23:9 24:10 25:11 26:12 27:13 28:14 29:15 30:15 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:23 40:24 41:25 42:26 43:27 44:28 45:29 46:30 47:31 48:32 49:32 50:33 51:34 52:35 53:36 54:37 55:38 56:39 57:40 58:41 59:41 60:42 61:43 62:43 63:44 64:45 65:45 66:45 67:46 68:47 69:48 70:49 71:50 72:51 73:52 74:53 75:53 76:54 77:55 78:56 79:56 80:57 81:58 82:59 83:60 84:61 85:62 86:63 87:64 88:65 89:66 90:67 91:68 92:69 93:70 94:71 95:72 96:73 97:74 98:74 99:75 100:76 101:77 102:78 103:79 104:80 105:81 106:82 107:82 108:82 109:82 110:82 111:83 112:84 113:85 114:86 115:87 116:88 117:89 118:90 119:90 120:91 121:92 122:93 123:94 124:95 125:96 126:97 127:98 128:99 129:100 130:100 131:101 132:102 133:103 134:104 135:105 136:106 137:107 138:108 139:108 140:108 141:109 142:110 143:111 144:112 145:113 146:114 147:114 148:115 149:116 150:117 151:118 152:119 153:120 154:121 155:122 156:123 157:124 158:125 159:126 160:127 161:128 162:129 163:129 164:130 165:131 166:132 167:132 168:132 169:133 170:134 171:135 172:135 173:136 174:137 175:137 176:138 177:139 178:139 179:140 180:141 181:142 182:143 183:144 184:145 185:146 186:147 187:148 188:149 189:150 190:150 191:151 192:152 193:153 194:154 195:155 196:156 197:157 198:158 199:158 200:159 201:160 202:161 203:162 204:163 205:163 206:163 207:164 208:165 209:166 210:167 211:168 212:168 213:169 214:169 215:170 216:171 217:172 218:173 219:174 220:175 221:176 222:177 223:177
I1203 11:56:13.102635 139660688828288 run_squad.py:438] token_to_orig_map: 13:0 14:1 15:2 16:3 17:4 18:5 19:6 20:7 21:8 22:9 23:9 24:10 25:11 26:12 27:13 28:14 29:15 30:15 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:23 40:24 41:25 42:26 43:27 44:28 45:29 46:30 47:31 48:32 49:32 50:33 51:34 52:35 53:36 54:37 55:38 56:39 57:40 58:41 59:41 60:42 61:43 62:43 63:44 64:45 65:45 66:45 67:46 68:47 69:48 70:49 71:50 72:51 73:52 74:53 75:53 76:54 77:55 78:56 79:56 80:57 81:58 82:59 83:60 84:61 85:62 86:63 87:64 88:65 89:66 90:67 91:68 92:69 93:70 94:71 95:72 96:73 97:74 98:74 99:75 100:76 101:77 102:78 103:79 104:80 105:81 106:82 107:82 108:82 109:82 110:82 111:83 112:84 113:85 114:86 115:87 116:88 117:89 118:90 119:90 120:91 121:92 122:93 123:94 124:95 125:96 126:97 127:98 128:99 129:100 130:100 131:101 132:102 133:103 134:104 135:105 136:106 137:107 138:108 139:108 140:108 141:109 142:110 143:111 144:112 145:113 146:114 147:114 148:115 149:116 150:117 151:118 152:119 153:120 154:121 155:122 156:123 157:124 158:125 159:126 160:127 161:128 162:129 163:129 164:130 165:131 166:132 167:132 168:132 169:133 170:134 171:135 172:135 173:136 174:137 175:137 176:138 177:139 178:139 179:140 180:141 181:142 182:143 183:144 184:145 185:146 186:147 187:148 188:149 189:150 190:150 191:151 192:152 193:153 194:154 195:155 196:156 197:157 198:158 199:158 200:159 201:160 202:161 203:162 204:163 205:163 206:163 207:164 208:165 209:166 210:167 211:168 212:168 213:169 214:169 215:170 216:171 217:172 218:173 219:174 220:175 221:176 222:177 223:177
INFO:tensorflow:token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True 205:True 206:True 207:True 208:True 209:True 210:True 211:True 212:True 213:True 214:True 215:True 216:True 217:True 218:True 219:True 220:True 221:True 222:True 223:True
I1203 11:56:13.102858 139660688828288 run_squad.py:440] token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True 205:True 206:True 207:True 208:True 209:True 210:True 211:True 212:True 213:True 214:True 215:True 216:True 217:True 218:True 219:True 220:True 221:True 222:True 223:True
INFO:tensorflow:input_ids: 101 2054 2001 1996 3291 2007 1996 4186 1005 1055 28885 1029 102 17836 4186 2633 2150 1996 4054 2548 5039 1999 9713 1010 2006 1996 16993 1997 3035 3848 1010 2040 2001 1996 2034 11590 2000 13960 2045 1025 2014 8646 2520 4921 2018 2351 2077 2049 6503 1012 2096 1996 2110 4734 2020 1037 11421 1997 13097 2102 1998 6120 1010 1996 26785 7971 6447 1997 1996 2047 4186 2020 5399 2625 20783 1012 2005 2028 2518 1010 2009 2001 2988 1996 28885 20482 2061 2172 2008 1996 8769 2018 2000 2022 3039 2000 3280 2091 1010 1998 8821 1996 2457 13927 1999 13580 23848 3490 8873 29320 1012 19536 2001 2061 2919 2008 1996 4592 9557 1010 1998 2043 1037 3247 2001 2579 2000 16500 3806 14186 1010 2045 2001 1037 3809 4737 2055 1996 3857 1011 2039 1997 3806 2006 1996 2896 8158 1012 2009 2001 2036 2056 2008 3095 2020 27327 1998 13971 1998 1996 4186 2001 6530 1012 2206 1996 3035 1005 1055 3510 1999 8905 1010 2014 3129 1010 3159 4789 1010 4986 2370 2007 1037 24934 1997 1996 4398 4822 1998 3095 1010 1998 2007 1996 2640 19399 1997 1996 4186 1012 1996 3471 2020 2035 28667 3775 10451 2011 1996 2485 1997 8905 1012 2174 1010 1996 16472 2020 2000 2709 2306 1996 5476 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.103345 139660688828288 run_squad.py:442] input_ids: 101 2054 2001 1996 3291 2007 1996 4186 1005 1055 28885 1029 102 17836 4186 2633 2150 1996 4054 2548 5039 1999 9713 1010 2006 1996 16993 1997 3035 3848 1010 2040 2001 1996 2034 11590 2000 13960 2045 1025 2014 8646 2520 4921 2018 2351 2077 2049 6503 1012 2096 1996 2110 4734 2020 1037 11421 1997 13097 2102 1998 6120 1010 1996 26785 7971 6447 1997 1996 2047 4186 2020 5399 2625 20783 1012 2005 2028 2518 1010 2009 2001 2988 1996 28885 20482 2061 2172 2008 1996 8769 2018 2000 2022 3039 2000 3280 2091 1010 1998 8821 1996 2457 13927 1999 13580 23848 3490 8873 29320 1012 19536 2001 2061 2919 2008 1996 4592 9557 1010 1998 2043 1037 3247 2001 2579 2000 16500 3806 14186 1010 2045 2001 1037 3809 4737 2055 1996 3857 1011 2039 1997 3806 2006 1996 2896 8158 1012 2009 2001 2036 2056 2008 3095 2020 27327 1998 13971 1998 1996 4186 2001 6530 1012 2206 1996 3035 1005 1055 3510 1999 8905 1010 2014 3129 1010 3159 4789 1010 4986 2370 2007 1037 24934 1997 1996 4398 4822 1998 3095 1010 1998 2007 1996 2640 19399 1997 1996 4186 1012 1996 3471 2020 2035 28667 3775 10451 2011 1996 2485 1997 8905 1012 2174 1010 1996 16472 2020 2000 2709 2306 1996 5476 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.192540 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.192921 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 83
I1203 11:56:13.193072 139660688828288 run_squad.py:451] start_position: 83
INFO:tensorflow:end_position: 85
I1203 11:56:13.193194 139660688828288 run_squad.py:452] end_position: 85
INFO:tensorflow:answer: the chimneys smoked
I1203 11:56:13.193318 139660688828288 run_squad.py:454] answer: the chimneys smoked
INFO:tensorflow:*** Example ***
I1203 11:56:13.199008 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000011
I1203 11:56:13.199279 139660688828288 run_squad.py:432] unique_id: 1000000011
INFO:tensorflow:example_index: 11
I1203 11:56:13.199392 139660688828288 run_squad.py:433] example_index: 11
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.199477 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] who has noted that binding precedent did not exist when the constitution was written ? [SEP] as federal judge alex ko ##zin ##ski has pointed out , binding precedent as we know it today simply did not exist at the time the constitution was framed . judicial decisions were not consistently , accurately , and faithful ##ly reported on both sides of the atlantic ( reporters often simply re ##wr ##ote or failed to publish decisions which they disliked ) , and the united kingdom lacked a coherent court hierarchy prior to the end of the 19th century . furthermore , english judges in the eighteenth century sub ##scribe ##d to now - obsolete natural law theories of law , by which law was believed to have an existence independent of what individual judges said . judges saw themselves as merely declaring the law which had always theoretically existed , and not as making the law . therefore , a judge could reject another judge ' s opinion as simply an incorrect statement of the law , in the way that scientists regularly reject each other ' s conclusions as incorrect statements of the laws of science . [SEP]
I1203 11:56:13.199648 139660688828288 run_squad.py:436] tokens: [CLS] who has noted that binding precedent did not exist when the constitution was written ? [SEP] as federal judge alex ko ##zin ##ski has pointed out , binding precedent as we know it today simply did not exist at the time the constitution was framed . judicial decisions were not consistently , accurately , and faithful ##ly reported on both sides of the atlantic ( reporters often simply re ##wr ##ote or failed to publish decisions which they disliked ) , and the united kingdom lacked a coherent court hierarchy prior to the end of the 19th century . furthermore , english judges in the eighteenth century sub ##scribe ##d to now - obsolete natural law theories of law , by which law was believed to have an existence independent of what individual judges said . judges saw themselves as merely declaring the law which had always theoretically existed , and not as making the law . therefore , a judge could reject another judge ' s opinion as simply an incorrect statement of the law , in the way that scientists regularly reject each other ' s conclusions as incorrect statements of the laws of science . [SEP]
INFO:tensorflow:token_to_orig_map: 17:0 18:1 19:2 20:3 21:4 22:4 23:4 24:5 25:6 26:7 27:7 28:8 29:9 30:10 31:11 32:12 33:13 34:14 35:15 36:16 37:17 38:18 39:19 40:20 41:21 42:22 43:23 44:24 45:25 46:25 47:26 48:27 49:28 50:29 51:30 52:30 53:31 54:31 55:32 56:33 57:33 58:34 59:35 60:36 61:37 62:38 63:39 64:40 65:41 66:41 67:42 68:43 69:44 70:44 71:44 72:45 73:46 74:47 75:48 76:49 77:50 78:51 79:52 80:52 81:52 82:53 83:54 84:55 85:56 86:57 87:58 88:59 89:60 90:61 91:62 92:63 93:64 94:65 95:66 96:67 97:68 98:69 99:69 100:70 101:70 102:71 103:72 104:73 105:74 106:75 107:76 108:77 109:77 110:77 111:78 112:79 113:79 114:79 115:80 116:81 117:82 118:83 119:84 120:84 121:85 122:86 123:87 124:88 125:89 126:90 127:91 128:92 129:93 130:94 131:95 132:96 133:97 134:98 135:99 136:99 137:100 138:101 139:102 140:103 141:104 142:105 143:106 144:107 145:108 146:109 147:110 148:111 149:112 150:112 151:113 152:114 153:115 154:116 155:117 156:118 157:118 158:119 159:119 160:120 161:121 162:122 163:123 164:124 165:125 166:125 167:125 168:126 169:127 170:128 171:129 172:130 173:131 174:132 175:133 176:134 177:134 178:135 179:136 180:137 181:138 182:139 183:140 184:141 185:142 186:143 187:143 188:143 189:144 190:145 191:146 192:147 193:148 194:149 195:150 196:151 197:152 198:152
I1203 11:56:13.199854 139660688828288 run_squad.py:438] token_to_orig_map: 17:0 18:1 19:2 20:3 21:4 22:4 23:4 24:5 25:6 26:7 27:7 28:8 29:9 30:10 31:11 32:12 33:13 34:14 35:15 36:16 37:17 38:18 39:19 40:20 41:21 42:22 43:23 44:24 45:25 46:25 47:26 48:27 49:28 50:29 51:30 52:30 53:31 54:31 55:32 56:33 57:33 58:34 59:35 60:36 61:37 62:38 63:39 64:40 65:41 66:41 67:42 68:43 69:44 70:44 71:44 72:45 73:46 74:47 75:48 76:49 77:50 78:51 79:52 80:52 81:52 82:53 83:54 84:55 85:56 86:57 87:58 88:59 89:60 90:61 91:62 92:63 93:64 94:65 95:66 96:67 97:68 98:69 99:69 100:70 101:70 102:71 103:72 104:73 105:74 106:75 107:76 108:77 109:77 110:77 111:78 112:79 113:79 114:79 115:80 116:81 117:82 118:83 119:84 120:84 121:85 122:86 123:87 124:88 125:89 126:90 127:91 128:92 129:93 130:94 131:95 132:96 133:97 134:98 135:99 136:99 137:100 138:101 139:102 140:103 141:104 142:105 143:106 144:107 145:108 146:109 147:110 148:111 149:112 150:112 151:113 152:114 153:115 154:116 155:117 156:118 157:118 158:119 159:119 160:120 161:121 162:122 163:123 164:124 165:125 166:125 167:125 168:126 169:127 170:128 171:129 172:130 173:131 174:132 175:133 176:134 177:134 178:135 179:136 180:137 181:138 182:139 183:140 184:141 185:142 186:143 187:143 188:143 189:144 190:145 191:146 192:147 193:148 194:149 195:150 196:151 197:152 198:152
INFO:tensorflow:token_is_max_context: 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True
I1203 11:56:13.200042 139660688828288 run_squad.py:440] token_is_max_context: 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True
INFO:tensorflow:input_ids: 101 2040 2038 3264 2008 8031 20056 2106 2025 4839 2043 1996 4552 2001 2517 1029 102 2004 2976 3648 4074 12849 17168 5488 2038 4197 2041 1010 8031 20056 2004 2057 2113 2009 2651 3432 2106 2025 4839 2012 1996 2051 1996 4552 2001 10366 1012 8268 6567 2020 2025 10862 1010 14125 1010 1998 11633 2135 2988 2006 2119 3903 1997 1996 4448 1006 12060 2411 3432 2128 13088 12184 2030 3478 2000 10172 6567 2029 2027 18966 1007 1010 1998 1996 2142 2983 10858 1037 18920 2457 12571 3188 2000 1996 2203 1997 1996 3708 2301 1012 7297 1010 2394 6794 1999 1996 12965 2301 4942 29234 2094 2000 2085 1011 15832 3019 2375 8106 1997 2375 1010 2011 2029 2375 2001 3373 2000 2031 2019 4598 2981 1997 2054 3265 6794 2056 1012 6794 2387 3209 2004 6414 13752 1996 2375 2029 2018 2467 22634 5839 1010 1998 2025 2004 2437 1996 2375 1012 3568 1010 1037 3648 2071 15454 2178 3648 1005 1055 5448 2004 3432 2019 16542 4861 1997 1996 2375 1010 1999 1996 2126 2008 6529 5570 15454 2169 2060 1005 1055 15306 2004 16542 8635 1997 1996 4277 1997 2671 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.200272 139660688828288 run_squad.py:442] input_ids: 101 2040 2038 3264 2008 8031 20056 2106 2025 4839 2043 1996 4552 2001 2517 1029 102 2004 2976 3648 4074 12849 17168 5488 2038 4197 2041 1010 8031 20056 2004 2057 2113 2009 2651 3432 2106 2025 4839 2012 1996 2051 1996 4552 2001 10366 1012 8268 6567 2020 2025 10862 1010 14125 1010 1998 11633 2135 2988 2006 2119 3903 1997 1996 4448 1006 12060 2411 3432 2128 13088 12184 2030 3478 2000 10172 6567 2029 2027 18966 1007 1010 1998 1996 2142 2983 10858 1037 18920 2457 12571 3188 2000 1996 2203 1997 1996 3708 2301 1012 7297 1010 2394 6794 1999 1996 12965 2301 4942 29234 2094 2000 2085 1011 15832 3019 2375 8106 1997 2375 1010 2011 2029 2375 2001 3373 2000 2031 2019 4598 2981 1997 2054 3265 6794 2056 1012 6794 2387 3209 2004 6414 13752 1996 2375 2029 2018 2467 22634 5839 1010 1998 2025 2004 2437 1996 2375 1012 3568 1010 1037 3648 2071 15454 2178 3648 1005 1055 5448 2004 3432 2019 16542 4861 1997 1996 2375 1010 1999 1996 2126 2008 6529 5570 15454 2169 2060 1005 1055 15306 2004 16542 8635 1997 1996 4277 1997 2671 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.200469 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.201010 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 18
I1203 11:56:13.201176 139660688828288 run_squad.py:451] start_position: 18
INFO:tensorflow:end_position: 23
I1203 11:56:13.201279 139660688828288 run_squad.py:452] end_position: 23
INFO:tensorflow:answer: federal judge alex ko ##zin ##ski
I1203 11:56:13.201383 139660688828288 run_squad.py:454] answer: federal judge alex ko ##zin ##ski
INFO:tensorflow:*** Example ***
I1203 11:56:13.205182 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000012
I1203 11:56:13.205397 139660688828288 run_squad.py:432] unique_id: 1000000012
INFO:tensorflow:example_index: 12
I1203 11:56:13.205506 139660688828288 run_squad.py:433] example_index: 12
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.205592 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what short ##coming was noticeable , from the start , for wesley clark ? [SEP] in september 2003 , retired four - star general wesley clark announced his intention to run in the presidential primary election for the democratic party nomination . his campaign focused on themes of leadership and patriot ##ism ; early campaign ads relied heavily on biography . his late start left him with relatively few detailed policy proposals . this weakness was apparent in his first few debates , although he soon presented a range of position papers , including a major tax - relief plan . nevertheless , the democrats did not flock to support his campaign . [SEP]
I1203 11:56:13.205735 139660688828288 run_squad.py:436] tokens: [CLS] what short ##coming was noticeable , from the start , for wesley clark ? [SEP] in september 2003 , retired four - star general wesley clark announced his intention to run in the presidential primary election for the democratic party nomination . his campaign focused on themes of leadership and patriot ##ism ; early campaign ads relied heavily on biography . his late start left him with relatively few detailed policy proposals . this weakness was apparent in his first few debates , although he soon presented a range of position papers , including a major tax - relief plan . nevertheless , the democrats did not flock to support his campaign . [SEP]
INFO:tensorflow:token_to_orig_map: 16:0 17:1 18:2 19:2 20:3 21:4 22:4 23:4 24:5 25:6 26:7 27:8 28:9 29:10 30:11 31:12 32:13 33:14 34:15 35:16 36:17 37:18 38:19 39:20 40:21 41:22 42:22 43:23 44:24 45:25 46:26 47:27 48:28 49:29 50:30 51:31 52:31 53:31 54:32 55:33 56:34 57:35 58:36 59:37 60:38 61:38 62:39 63:40 64:41 65:42 66:43 67:44 68:45 69:46 70:47 71:48 72:49 73:49 74:50 75:51 76:52 77:53 78:54 79:55 80:56 81:57 82:58 83:58 84:59 85:60 86:61 87:62 88:63 89:64 90:65 91:66 92:67 93:67 94:68 95:69 96:70 97:71 98:71 99:71 100:72 101:72 102:73 103:73 104:74 105:75 106:76 107:77 108:78 109:79 110:80 111:81 112:82 113:82
I1203 11:56:13.205911 139660688828288 run_squad.py:438] token_to_orig_map: 16:0 17:1 18:2 19:2 20:3 21:4 22:4 23:4 24:5 25:6 26:7 27:8 28:9 29:10 30:11 31:12 32:13 33:14 34:15 35:16 36:17 37:18 38:19 39:20 40:21 41:22 42:22 43:23 44:24 45:25 46:26 47:27 48:28 49:29 50:30 51:31 52:31 53:31 54:32 55:33 56:34 57:35 58:36 59:37 60:38 61:38 62:39 63:40 64:41 65:42 66:43 67:44 68:45 69:46 70:47 71:48 72:49 73:49 74:50 75:51 76:52 77:53 78:54 79:55 80:56 81:57 82:58 83:58 84:59 85:60 86:61 87:62 88:63 89:64 90:65 91:66 92:67 93:67 94:68 95:69 96:70 97:71 98:71 99:71 100:72 101:72 102:73 103:73 104:74 105:75 106:76 107:77 108:78 109:79 110:80 111:81 112:82 113:82
INFO:tensorflow:token_is_max_context: 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True
I1203 11:56:13.297038 139660688828288 run_squad.py:440] token_is_max_context: 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True
INFO:tensorflow:input_ids: 101 2054 2460 18935 2001 17725 1010 2013 1996 2707 1010 2005 11482 5215 1029 102 1999 2244 2494 1010 3394 2176 1011 2732 2236 11482 5215 2623 2010 6808 2000 2448 1999 1996 4883 3078 2602 2005 1996 3537 2283 6488 1012 2010 3049 4208 2006 6991 1997 4105 1998 16419 2964 1025 2220 3049 14997 13538 4600 2006 8308 1012 2010 2397 2707 2187 2032 2007 4659 2261 6851 3343 10340 1012 2023 11251 2001 6835 1999 2010 2034 2261 14379 1010 2348 2002 2574 3591 1037 2846 1997 2597 4981 1010 2164 1037 2350 4171 1011 4335 2933 1012 6600 1010 1996 8037 2106 2025 19311 2000 2490 2010 3049 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.298385 139660688828288 run_squad.py:442] input_ids: 101 2054 2460 18935 2001 17725 1010 2013 1996 2707 1010 2005 11482 5215 1029 102 1999 2244 2494 1010 3394 2176 1011 2732 2236 11482 5215 2623 2010 6808 2000 2448 1999 1996 4883 3078 2602 2005 1996 3537 2283 6488 1012 2010 3049 4208 2006 6991 1997 4105 1998 16419 2964 1025 2220 3049 14997 13538 4600 2006 8308 1012 2010 2397 2707 2187 2032 2007 4659 2261 6851 3343 10340 1012 2023 11251 2001 6835 1999 2010 2034 2261 14379 1010 2348 2002 2574 3591 1037 2846 1997 2597 4981 1010 2164 1037 2350 4171 1011 4335 2933 1012 6600 1010 1996 8037 2106 2025 19311 2000 2490 2010 3049 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.299223 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.301889 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 69
I1203 11:56:13.302088 139660688828288 run_squad.py:451] start_position: 69
INFO:tensorflow:end_position: 72
I1203 11:56:13.302200 139660688828288 run_squad.py:452] end_position: 72
INFO:tensorflow:answer: few detailed policy proposals
I1203 11:56:13.302300 139660688828288 run_squad.py:454] answer: few detailed policy proposals
INFO:tensorflow:*** Example ***
I1203 11:56:13.308462 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000013
I1203 11:56:13.308698 139660688828288 run_squad.py:432] unique_id: 1000000013
INFO:tensorflow:example_index: 13
I1203 11:56:13.308819 139660688828288 run_squad.py:433] example_index: 13
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.308938 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] the digest ##ive chambers of the ct ##eno ##ph ##ora and the cn ##ida ##ria serve as what ? [SEP] among the other ph ##yla , the ct ##eno ##ph ##ora and the cn ##ida ##ria , which includes sea an ##emon ##es , coral ##s , and jelly ##fish , are radial ##ly symmetric and have digest ##ive chambers with a single opening , which serves as both the mouth and the an ##us . both have distinct tissues , but they are not organized into organs . there are only two main ge ##rm layers , the ec ##to ##der ##m and end ##oder ##m , with only scattered cells between them . as such , these animals are sometimes called dip ##lo ##bla ##stic . the tiny pl ##aco ##zo ##ans are similar , but they do not have a permanent digest ##ive chamber . [SEP]
I1203 11:56:13.309104 139660688828288 run_squad.py:436] tokens: [CLS] the digest ##ive chambers of the ct ##eno ##ph ##ora and the cn ##ida ##ria serve as what ? [SEP] among the other ph ##yla , the ct ##eno ##ph ##ora and the cn ##ida ##ria , which includes sea an ##emon ##es , coral ##s , and jelly ##fish , are radial ##ly symmetric and have digest ##ive chambers with a single opening , which serves as both the mouth and the an ##us . both have distinct tissues , but they are not organized into organs . there are only two main ge ##rm layers , the ec ##to ##der ##m and end ##oder ##m , with only scattered cells between them . as such , these animals are sometimes called dip ##lo ##bla ##stic . the tiny pl ##aco ##zo ##ans are similar , but they do not have a permanent digest ##ive chamber . [SEP]
INFO:tensorflow:token_to_orig_map: 21:0 22:1 23:2 24:3 25:3 26:3 27:4 28:5 29:5 30:5 31:5 32:6 33:7 34:8 35:8 36:8 37:8 38:9 39:10 40:11 41:12 42:12 43:12 44:12 45:13 46:13 47:13 48:14 49:15 50:15 51:15 52:16 53:17 54:17 55:18 56:19 57:20 58:21 59:21 60:22 61:23 62:24 63:25 64:26 65:26 66:27 67:28 68:29 69:30 70:31 71:32 72:33 73:34 74:35 75:35 76:35 77:36 78:37 79:38 80:39 81:39 82:40 83:41 84:42 85:43 86:44 87:45 88:46 89:46 90:47 91:48 92:49 93:50 94:51 95:52 96:52 97:53 98:53 99:54 100:55 101:55 102:55 103:55 104:56 105:57 106:57 107:57 108:57 109:58 110:59 111:60 112:61 113:62 114:63 115:63 116:64 117:65 118:65 119:66 120:67 121:68 122:69 123:70 124:71 125:71 126:71 127:71 128:71 129:72 130:73 131:74 132:74 133:74 134:74 135:75 136:76 137:76 138:77 139:78 140:79 141:80 142:81 143:82 144:83 145:84 146:84 147:85 148:85
I1203 11:56:13.309274 139660688828288 run_squad.py:438] token_to_orig_map: 21:0 22:1 23:2 24:3 25:3 26:3 27:4 28:5 29:5 30:5 31:5 32:6 33:7 34:8 35:8 36:8 37:8 38:9 39:10 40:11 41:12 42:12 43:12 44:12 45:13 46:13 47:13 48:14 49:15 50:15 51:15 52:16 53:17 54:17 55:18 56:19 57:20 58:21 59:21 60:22 61:23 62:24 63:25 64:26 65:26 66:27 67:28 68:29 69:30 70:31 71:32 72:33 73:34 74:35 75:35 76:35 77:36 78:37 79:38 80:39 81:39 82:40 83:41 84:42 85:43 86:44 87:45 88:46 89:46 90:47 91:48 92:49 93:50 94:51 95:52 96:52 97:53 98:53 99:54 100:55 101:55 102:55 103:55 104:56 105:57 106:57 107:57 108:57 109:58 110:59 111:60 112:61 113:62 114:63 115:63 116:64 117:65 118:65 119:66 120:67 121:68 122:69 123:70 124:71 125:71 126:71 127:71 128:71 129:72 130:73 131:74 132:74 133:74 134:74 135:75 136:76 137:76 138:77 139:78 140:79 141:80 142:81 143:82 144:83 145:84 146:84 147:85 148:85
INFO:tensorflow:token_is_max_context: 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True
I1203 11:56:13.309428 139660688828288 run_squad.py:440] token_is_max_context: 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True
INFO:tensorflow:input_ids: 101 1996 17886 3512 8477 1997 1996 14931 16515 8458 6525 1998 1996 27166 8524 4360 3710 2004 2054 1029 102 2426 1996 2060 6887 23943 1010 1996 14931 16515 8458 6525 1998 1996 27166 8524 4360 1010 2029 2950 2712 2019 26941 2229 1010 11034 2015 1010 1998 20919 7529 1010 2024 15255 2135 19490 1998 2031 17886 3512 8477 2007 1037 2309 3098 1010 2029 4240 2004 2119 1996 2677 1998 1996 2019 2271 1012 2119 2031 5664 14095 1010 2021 2027 2024 2025 4114 2046 11595 1012 2045 2024 2069 2048 2364 16216 10867 9014 1010 1996 14925 3406 4063 2213 1998 2203 27381 2213 1010 2007 2069 7932 4442 2090 2068 1012 2004 2107 1010 2122 4176 2024 2823 2170 16510 4135 28522 10074 1012 1996 4714 20228 22684 6844 6962 2024 2714 1010 2021 2027 2079 2025 2031 1037 4568 17886 3512 4574 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.309638 139660688828288 run_squad.py:442] input_ids: 101 1996 17886 3512 8477 1997 1996 14931 16515 8458 6525 1998 1996 27166 8524 4360 3710 2004 2054 1029 102 2426 1996 2060 6887 23943 1010 1996 14931 16515 8458 6525 1998 1996 27166 8524 4360 1010 2029 2950 2712 2019 26941 2229 1010 11034 2015 1010 1998 20919 7529 1010 2024 15255 2135 19490 1998 2031 17886 3512 8477 2007 1037 2309 3098 1010 2029 4240 2004 2119 1996 2677 1998 1996 2019 2271 1012 2119 2031 5664 14095 1010 2021 2027 2024 2025 4114 2046 11595 1012 2045 2024 2069 2048 2364 16216 10867 9014 1010 1996 14925 3406 4063 2213 1998 2203 27381 2213 1010 2007 2069 7932 4442 2090 2068 1012 2004 2107 1010 2122 4176 2024 2823 2170 16510 4135 28522 10074 1012 1996 4714 20228 22684 6844 6962 2024 2714 1010 2021 2027 2079 2025 2031 1037 4568 17886 3512 4574 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.310145 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.310465 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 69
I1203 11:56:13.310609 139660688828288 run_squad.py:451] start_position: 69
INFO:tensorflow:end_position: 75
I1203 11:56:13.310721 139660688828288 run_squad.py:452] end_position: 75
INFO:tensorflow:answer: both the mouth and the an ##us
I1203 11:56:13.310827 139660688828288 run_squad.py:454] answer: both the mouth and the an ##us
INFO:tensorflow:*** Example ***
I1203 11:56:13.313349 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000014
I1203 11:56:13.313561 139660688828288 run_squad.py:432] unique_id: 1000000014
INFO:tensorflow:example_index: 14
I1203 11:56:13.313696 139660688828288 run_squad.py:433] example_index: 14
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.313800 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] when did buddhism leave zhejiang ? [SEP] in mid - 2015 the government of zhejiang recognised folk religion as " civil religion " beginning the registration of more than twenty thousand folk religious associations . buddhism has an important presence since its arrival in zhejiang 1 , 800 years ago . [SEP]
I1203 11:56:13.313934 139660688828288 run_squad.py:436] tokens: [CLS] when did buddhism leave zhejiang ? [SEP] in mid - 2015 the government of zhejiang recognised folk religion as " civil religion " beginning the registration of more than twenty thousand folk religious associations . buddhism has an important presence since its arrival in zhejiang 1 , 800 years ago . [SEP]
INFO:tensorflow:token_to_orig_map: 8:0 9:1 10:1 11:1 12:2 13:3 14:4 15:5 16:6 17:7 18:8 19:9 20:10 21:10 22:11 23:11 24:12 25:13 26:14 27:15 28:16 29:17 30:18 31:19 32:20 33:21 34:22 35:22 36:23 37:24 38:25 39:26 40:27 41:28 42:29 43:30 44:31 45:32 46:33 47:33 48:33 49:34 50:35 51:35
I1203 11:56:13.314079 139660688828288 run_squad.py:438] token_to_orig_map: 8:0 9:1 10:1 11:1 12:2 13:3 14:4 15:5 16:6 17:7 18:8 19:9 20:10 21:10 22:11 23:11 24:12 25:13 26:14 27:15 28:16 29:17 30:18 31:19 32:20 33:21 34:22 35:22 36:23 37:24 38:25 39:26 40:27 41:28 42:29 43:30 44:31 45:32 46:33 47:33 48:33 49:34 50:35 51:35
INFO:tensorflow:token_is_max_context: 8:True 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True
I1203 11:56:13.314196 139660688828288 run_squad.py:440] token_is_max_context: 8:True 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True
INFO:tensorflow:input_ids: 101 2043 2106 11388 2681 26805 1029 102 1999 3054 1011 2325 1996 2231 1997 26805 7843 5154 4676 2004 1000 2942 4676 1000 2927 1996 8819 1997 2062 2084 3174 4595 5154 3412 8924 1012 11388 2038 2019 2590 3739 2144 2049 5508 1999 26805 1015 1010 5385 2086 3283 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.314404 139660688828288 run_squad.py:442] input_ids: 101 2043 2106 11388 2681 26805 1029 102 1999 3054 1011 2325 1996 2231 1997 26805 7843 5154 4676 2004 1000 2942 4676 1000 2927 1996 8819 1997 2062 2084 3174 4595 5154 3412 8924 1012 11388 2038 2019 2590 3739 2144 2049 5508 1999 26805 1015 1010 5385 2086 3283 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.314603 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.409124 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:impossible example
I1203 11:56:13.409396 139660688828288 run_squad.py:448] impossible example
INFO:tensorflow:*** Example ***
I1203 11:56:13.415258 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000015
I1203 11:56:13.415520 139660688828288 run_squad.py:432] unique_id: 1000000015
INFO:tensorflow:example_index: 15
I1203 11:56:13.415619 139660688828288 run_squad.py:433] example_index: 15
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.415716 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] who embraced his family according to the new testament ? [SEP] the ha ##gio ##graphy of mary and the holy family can be contrasted with other material in the gospels . these references include an incident which can be interpreted as jesus rejecting his family in the new testament : " and his mother and his brothers arrived , and standing outside , they sent in a message asking for him . . . and looking at those who sat in a circle around him , jesus said , ' these are my mother and my brothers . whoever does the will of god is my brother , and sister , and mother ' . " [ 3 : 31 - 35 ] other verses suggest a conflict between jesus and his family , including an attempt to have jesus restrained because " he is out of his mind " , and the famous quote : " a prophet is not without honor except in his own town , among his relatives and in his own home . " a leading biblical scholar commented : " there are clear signs not only that jesus ' s family rejected his message during his public ministry but that he in turn spur ##ned them publicly " . [SEP]
I1203 11:56:13.415878 139660688828288 run_squad.py:436] tokens: [CLS] who embraced his family according to the new testament ? [SEP] the ha ##gio ##graphy of mary and the holy family can be contrasted with other material in the gospels . these references include an incident which can be interpreted as jesus rejecting his family in the new testament : " and his mother and his brothers arrived , and standing outside , they sent in a message asking for him . . . and looking at those who sat in a circle around him , jesus said , ' these are my mother and my brothers . whoever does the will of god is my brother , and sister , and mother ' . " [ 3 : 31 - 35 ] other verses suggest a conflict between jesus and his family , including an attempt to have jesus restrained because " he is out of his mind " , and the famous quote : " a prophet is not without honor except in his own town , among his relatives and in his own home . " a leading biblical scholar commented : " there are clear signs not only that jesus ' s family rejected his message during his public ministry but that he in turn spur ##ned them publicly " . [SEP]
INFO:tensorflow:token_to_orig_map: 12:0 13:1 14:1 15:1 16:2 17:3 18:4 19:5 20:6 21:7 22:8 23:9 24:10 25:11 26:12 27:13 28:14 29:15 30:16 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:26 42:27 43:28 44:29 45:30 46:31 47:32 48:33 49:34 50:34 51:35 52:35 53:36 54:37 55:38 56:39 57:40 58:41 59:41 60:42 61:43 62:44 63:44 64:45 65:46 66:47 67:48 68:49 69:50 70:51 71:52 72:53 73:53 74:53 75:54 76:55 77:56 78:57 79:58 80:59 81:60 82:61 83:62 84:63 85:64 86:64 87:65 88:66 89:66 90:67 91:67 92:68 93:69 94:70 95:71 96:72 97:73 98:73 99:74 100:75 101:76 102:77 103:78 104:79 105:80 106:81 107:82 108:82 109:83 110:84 111:84 112:85 113:86 114:86 115:86 116:86 117:86 118:86 119:86 120:86 121:86 122:86 123:86 124:87 125:88 126:89 127:90 128:91 129:92 130:93 131:94 132:95 133:96 134:96 135:97 136:98 137:99 138:100 139:101 140:102 141:103 142:104 143:105 144:105 145:106 146:107 147:108 148:109 149:110 150:110 151:110 152:111 153:112 154:113 155:114 156:114 157:115 158:115 159:116 160:117 161:118 162:119 163:120 164:121 165:122 166:123 167:124 168:125 169:125 170:126 171:127 172:128 173:129 174:130 175:131 176:132 177:133 178:133 179:133 180:134 181:135 182:136 183:137 184:138 185:138 186:139 187:139 188:140 189:141 190:142 191:143 192:144 193:145 194:146 195:146 196:146 197:147 198:148 199:149 200:150 201:151 202:152 203:153 204:154 205:155 206:156 207:157 208:158 209:159 210:160 211:160 212:161 213:162 214:162 215:162
I1203 11:56:13.416027 139660688828288 run_squad.py:438] token_to_orig_map: 12:0 13:1 14:1 15:1 16:2 17:3 18:4 19:5 20:6 21:7 22:8 23:9 24:10 25:11 26:12 27:13 28:14 29:15 30:16 31:16 32:17 33:18 34:19 35:20 36:21 37:22 38:23 39:24 40:25 41:26 42:27 43:28 44:29 45:30 46:31 47:32 48:33 49:34 50:34 51:35 52:35 53:36 54:37 55:38 56:39 57:40 58:41 59:41 60:42 61:43 62:44 63:44 64:45 65:46 66:47 67:48 68:49 69:50 70:51 71:52 72:53 73:53 74:53 75:54 76:55 77:56 78:57 79:58 80:59 81:60 82:61 83:62 84:63 85:64 86:64 87:65 88:66 89:66 90:67 91:67 92:68 93:69 94:70 95:71 96:72 97:73 98:73 99:74 100:75 101:76 102:77 103:78 104:79 105:80 106:81 107:82 108:82 109:83 110:84 111:84 112:85 113:86 114:86 115:86 116:86 117:86 118:86 119:86 120:86 121:86 122:86 123:86 124:87 125:88 126:89 127:90 128:91 129:92 130:93 131:94 132:95 133:96 134:96 135:97 136:98 137:99 138:100 139:101 140:102 141:103 142:104 143:105 144:105 145:106 146:107 147:108 148:109 149:110 150:110 151:110 152:111 153:112 154:113 155:114 156:114 157:115 158:115 159:116 160:117 161:118 162:119 163:120 164:121 165:122 166:123 167:124 168:125 169:125 170:126 171:127 172:128 173:129 174:130 175:131 176:132 177:133 178:133 179:133 180:134 181:135 182:136 183:137 184:138 185:138 186:139 187:139 188:140 189:141 190:142 191:143 192:144 193:145 194:146 195:146 196:146 197:147 198:148 199:149 200:150 201:151 202:152 203:153 204:154 205:155 206:156 207:157 208:158 209:159 210:160 211:160 212:161 213:162 214:162 215:162
INFO:tensorflow:token_is_max_context: 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True 205:True 206:True 207:True 208:True 209:True 210:True 211:True 212:True 213:True 214:True 215:True
I1203 11:56:13.416152 139660688828288 run_squad.py:440] token_is_max_context: 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True 197:True 198:True 199:True 200:True 201:True 202:True 203:True 204:True 205:True 206:True 207:True 208:True 209:True 210:True 211:True 212:True 213:True 214:True 215:True
INFO:tensorflow:input_ids: 101 2040 14218 2010 2155 2429 2000 1996 2047 9025 1029 102 1996 5292 11411 12565 1997 2984 1998 1996 4151 2155 2064 2022 22085 2007 2060 3430 1999 1996 24131 1012 2122 7604 2421 2019 5043 2029 2064 2022 10009 2004 4441 21936 2010 2155 1999 1996 2047 9025 1024 1000 1998 2010 2388 1998 2010 3428 3369 1010 1998 3061 2648 1010 2027 2741 1999 1037 4471 4851 2005 2032 1012 1012 1012 1998 2559 2012 2216 2040 2938 1999 1037 4418 2105 2032 1010 4441 2056 1010 1005 2122 2024 2026 2388 1998 2026 3428 1012 9444 2515 1996 2097 1997 2643 2003 2026 2567 1010 1998 2905 1010 1998 2388 1005 1012 1000 1031 1017 1024 2861 1011 3486 1033 2060 11086 6592 1037 4736 2090 4441 1998 2010 2155 1010 2164 2019 3535 2000 2031 4441 19868 2138 1000 2002 2003 2041 1997 2010 2568 1000 1010 1998 1996 3297 14686 1024 1000 1037 12168 2003 2025 2302 3932 3272 1999 2010 2219 2237 1010 2426 2010 9064 1998 1999 2010 2219 2188 1012 1000 1037 2877 10213 6288 7034 1024 1000 2045 2024 3154 5751 2025 2069 2008 4441 1005 1055 2155 5837 2010 4471 2076 2010 2270 3757 2021 2008 2002 1999 2735 12996 7228 2068 7271 1000 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.416316 139660688828288 run_squad.py:442] input_ids: 101 2040 14218 2010 2155 2429 2000 1996 2047 9025 1029 102 1996 5292 11411 12565 1997 2984 1998 1996 4151 2155 2064 2022 22085 2007 2060 3430 1999 1996 24131 1012 2122 7604 2421 2019 5043 2029 2064 2022 10009 2004 4441 21936 2010 2155 1999 1996 2047 9025 1024 1000 1998 2010 2388 1998 2010 3428 3369 1010 1998 3061 2648 1010 2027 2741 1999 1037 4471 4851 2005 2032 1012 1012 1012 1998 2559 2012 2216 2040 2938 1999 1037 4418 2105 2032 1010 4441 2056 1010 1005 2122 2024 2026 2388 1998 2026 3428 1012 9444 2515 1996 2097 1997 2643 2003 2026 2567 1010 1998 2905 1010 1998 2388 1005 1012 1000 1031 1017 1024 2861 1011 3486 1033 2060 11086 6592 1037 4736 2090 4441 1998 2010 2155 1010 2164 2019 3535 2000 2031 4441 19868 2138 1000 2002 2003 2041 1997 2010 2568 1000 1010 1998 1996 3297 14686 1024 1000 1037 12168 2003 2025 2302 3932 3272 1999 2010 2219 2237 1010 2426 2010 9064 1998 1999 2010 2219 2188 1012 1000 1037 2877 10213 6288 7034 1024 1000 2045 2024 3154 5751 2025 2069 2008 4441 1005 1055 2155 5837 2010 4471 2076 2010 2270 3757 2021 2008 2002 1999 2735 12996 7228 2068 7271 1000 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.416443 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.416563 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:impossible example
I1203 11:56:13.416619 139660688828288 run_squad.py:448] impossible example
INFO:tensorflow:*** Example ***
I1203 11:56:13.420585 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000016
I1203 11:56:13.420807 139660688828288 run_squad.py:432] unique_id: 1000000016
INFO:tensorflow:example_index: 16
I1203 11:56:13.420875 139660688828288 run_squad.py:433] example_index: 16
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.420927 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] where does a bill go once the president signs it into effect ? [SEP] after the president signs a bill into law ( or congress en ##act ##s it over his veto ) , it is delivered to the office of the federal register ( of ##r ) of the national archives and records administration ( nara ) where it is assigned a law number , and prepared for publication as a slip law . public laws , but not private laws , are also given legal statutory citation by the of ##r . at the end of each session of congress , the slip laws are compiled into bound volumes called the united states statutes at large , and they are known as session laws . the statutes at large present a chronological arrangement of the laws in the exact order that they have been enacted . [SEP]
I1203 11:56:13.421022 139660688828288 run_squad.py:436] tokens: [CLS] where does a bill go once the president signs it into effect ? [SEP] after the president signs a bill into law ( or congress en ##act ##s it over his veto ) , it is delivered to the office of the federal register ( of ##r ) of the national archives and records administration ( nara ) where it is assigned a law number , and prepared for publication as a slip law . public laws , but not private laws , are also given legal statutory citation by the of ##r . at the end of each session of congress , the slip laws are compiled into bound volumes called the united states statutes at large , and they are known as session laws . the statutes at large present a chronological arrangement of the laws in the exact order that they have been enacted . [SEP]
INFO:tensorflow:token_to_orig_map: 15:0 16:1 17:2 18:3 19:4 20:5 21:6 22:7 23:8 24:8 25:9 26:10 27:10 28:10 29:11 30:12 31:13 32:14 33:14 34:14 35:15 36:16 37:17 38:18 39:19 40:20 41:21 42:22 43:23 44:24 45:25 46:25 47:25 48:25 49:26 50:27 51:28 52:29 53:30 54:31 55:32 56:33 57:33 58:33 59:34 60:35 61:36 62:37 63:38 64:39 65:40 66:40 67:41 68:42 69:43 70:44 71:45 72:46 73:47 74:48 75:48 76:49 77:50 78:50 79:51 80:52 81:53 82:54 83:54 84:55 85:56 86:57 87:58 88:59 89:60 90:61 91:62 92:63 93:63 94:63 95:64 96:65 97:66 98:67 99:68 100:69 101:70 102:71 103:71 104:72 105:73 106:74 107:75 108:76 109:77 110:78 111:79 112:80 113:81 114:82 115:83 116:84 117:85 118:86 119:86 120:87 121:88 122:89 123:90 124:91 125:92 126:93 127:93 128:94 129:95 130:96 131:97 132:98 133:99 134:100 135:101 136:102 137:103 138:104 139:105 140:106 141:107 142:108 143:109 144:110 145:111 146:112 147:113 148:113
I1203 11:56:13.512223 139660688828288 run_squad.py:438] token_to_orig_map: 15:0 16:1 17:2 18:3 19:4 20:5 21:6 22:7 23:8 24:8 25:9 26:10 27:10 28:10 29:11 30:12 31:13 32:14 33:14 34:14 35:15 36:16 37:17 38:18 39:19 40:20 41:21 42:22 43:23 44:24 45:25 46:25 47:25 48:25 49:26 50:27 51:28 52:29 53:30 54:31 55:32 56:33 57:33 58:33 59:34 60:35 61:36 62:37 63:38 64:39 65:40 66:40 67:41 68:42 69:43 70:44 71:45 72:46 73:47 74:48 75:48 76:49 77:50 78:50 79:51 80:52 81:53 82:54 83:54 84:55 85:56 86:57 87:58 88:59 89:60 90:61 91:62 92:63 93:63 94:63 95:64 96:65 97:66 98:67 99:68 100:69 101:70 102:71 103:71 104:72 105:73 106:74 107:75 108:76 109:77 110:78 111:79 112:80 113:81 114:82 115:83 116:84 117:85 118:86 119:86 120:87 121:88 122:89 123:90 124:91 125:92 126:93 127:93 128:94 129:95 130:96 131:97 132:98 133:99 134:100 135:101 136:102 137:103 138:104 139:105 140:106 141:107 142:108 143:109 144:110 145:111 146:112 147:113 148:113
INFO:tensorflow:token_is_max_context: 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True
I1203 11:56:13.513422 139660688828288 run_squad.py:440] token_is_max_context: 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True
INFO:tensorflow:input_ids: 101 2073 2515 1037 3021 2175 2320 1996 2343 5751 2009 2046 3466 1029 102 2044 1996 2343 5751 1037 3021 2046 2375 1006 2030 3519 4372 18908 2015 2009 2058 2010 22102 1007 1010 2009 2003 5359 2000 1996 2436 1997 1996 2976 4236 1006 1997 2099 1007 1997 1996 2120 8264 1998 2636 3447 1006 27544 1007 2073 2009 2003 4137 1037 2375 2193 1010 1998 4810 2005 4772 2004 1037 7540 2375 1012 2270 4277 1010 2021 2025 2797 4277 1010 2024 2036 2445 3423 15201 11091 2011 1996 1997 2099 1012 2012 1996 2203 1997 2169 5219 1997 3519 1010 1996 7540 4277 2024 9227 2046 5391 6702 2170 1996 2142 2163 18574 2012 2312 1010 1998 2027 2024 2124 2004 5219 4277 1012 1996 18574 2012 2312 2556 1037 23472 6512 1997 1996 4277 1999 1996 6635 2344 2008 2027 2031 2042 11955 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.514029 139660688828288 run_squad.py:442] input_ids: 101 2073 2515 1037 3021 2175 2320 1996 2343 5751 2009 2046 3466 1029 102 2044 1996 2343 5751 1037 3021 2046 2375 1006 2030 3519 4372 18908 2015 2009 2058 2010 22102 1007 1010 2009 2003 5359 2000 1996 2436 1997 1996 2976 4236 1006 1997 2099 1007 1997 1996 2120 8264 1998 2636 3447 1006 27544 1007 2073 2009 2003 4137 1037 2375 2193 1010 1998 4810 2005 4772 2004 1037 7540 2375 1012 2270 4277 1010 2021 2025 2797 4277 1010 2024 2036 2445 3423 15201 11091 2011 1996 1997 2099 1012 2012 1996 2203 1997 2169 5219 1997 3519 1010 1996 7540 4277 2024 9227 2046 5391 6702 2170 1996 2142 2163 18574 2012 2312 1010 1998 2027 2024 2124 2004 5219 4277 1012 1996 18574 2012 2312 2556 1037 23472 6512 1997 1996 4277 1999 1996 6635 2344 2008 2027 2031 2042 11955 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.514479 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.514999 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 37
I1203 11:56:13.515283 139660688828288 run_squad.py:451] start_position: 37
INFO:tensorflow:end_position: 58
I1203 11:56:13.515416 139660688828288 run_squad.py:452] end_position: 58
INFO:tensorflow:answer: delivered to the office of the federal register ( of ##r ) of the national archives and records administration ( nara )
I1203 11:56:13.515524 139660688828288 run_squad.py:454] answer: delivered to the office of the federal register ( of ##r ) of the national archives and records administration ( nara )
INFO:tensorflow:*** Example ***
I1203 11:56:13.520728 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000017
I1203 11:56:13.520958 139660688828288 run_squad.py:432] unique_id: 1000000017
INFO:tensorflow:example_index: 17
I1203 11:56:13.521080 139660688828288 run_squad.py:433] example_index: 17
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.521167 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what other beginnings of origin ##ation do some of the last names of the greeks share ? [SEP] greek surname ##s were widely in use by the 9th century su ##pp ##lan ##ting the ancient tradition of using the father ’ s name , however greek surname ##s are most commonly patron ##ym ##ics . commonly , greek male surname ##s end in - s , which is the common ending for greek masculine proper nouns in the no ##mina ##tive case . exceptionally , some end in - ou , indicating the gen ##itive case of this proper noun for patron ##ym ##ic reasons . although surname ##s in mainland greece are static today , dynamic and changing patron ##ym ##ic usage survives in middle names where the gen ##itive of father ' s first name is commonly the middle name ( this usage having been passed on to the russians ) . in cyprus , by contrast , surname ##s follow the ancient tradition of being given according to the father ’ s name . finally , in addition to greek - derived surname ##s many have latin , turkish and italian origin . [SEP]
I1203 11:56:13.521336 139660688828288 run_squad.py:436] tokens: [CLS] what other beginnings of origin ##ation do some of the last names of the greeks share ? [SEP] greek surname ##s were widely in use by the 9th century su ##pp ##lan ##ting the ancient tradition of using the father ’ s name , however greek surname ##s are most commonly patron ##ym ##ics . commonly , greek male surname ##s end in - s , which is the common ending for greek masculine proper nouns in the no ##mina ##tive case . exceptionally , some end in - ou , indicating the gen ##itive case of this proper noun for patron ##ym ##ic reasons . although surname ##s in mainland greece are static today , dynamic and changing patron ##ym ##ic usage survives in middle names where the gen ##itive of father ' s first name is commonly the middle name ( this usage having been passed on to the russians ) . in cyprus , by contrast , surname ##s follow the ancient tradition of being given according to the father ’ s name . finally , in addition to greek - derived surname ##s many have latin , turkish and italian origin . [SEP]
INFO:tensorflow:token_to_orig_map: 19:0 20:1 21:1 22:2 23:3 24:4 25:5 26:6 27:7 28:8 29:9 30:10 31:10 32:10 33:10 34:11 35:12 36:13 37:14 38:15 39:16 40:17 41:17 42:17 43:18 44:18 45:19 46:20 47:21 48:21 49:22 50:23 51:24 52:25 53:25 54:25 55:25 56:26 57:26 58:27 59:28 60:29 61:29 62:30 63:31 64:32 65:32 66:32 67:33 68:34 69:35 70:36 71:37 72:38 73:39 74:40 75:41 76:42 77:43 78:44 79:45 80:45 81:45 82:46 83:46 84:47 85:47 86:48 87:49 88:50 89:51 90:51 91:51 92:52 93:53 94:54 95:54 96:55 97:56 98:57 99:58 100:59 101:60 102:61 103:61 104:61 105:62 106:62 107:63 108:64 109:64 110:65 111:66 112:67 113:68 114:69 115:70 116:70 117:71 118:72 119:73 120:74 121:74 122:74 123:75 124:76 125:77 126:78 127:79 128:80 129:81 130:82 131:82 132:83 133:84 134:84 135:84 136:85 137:86 138:87 139:88 140:89 141:90 142:91 143:92 144:92 145:93 146:94 147:95 148:96 149:97 150:98 151:99 152:100 153:100 154:100 155:101 156:102 157:102 158:103 159:104 160:104 161:105 162:105 163:106 164:107 165:108 166:109 167:110 168:111 169:112 170:113 171:114 172:115 173:116 174:116 175:116 176:117 177:117 178:118 179:118 180:119 181:120 182:121 183:122 184:122 185:122 186:123 187:123 188:124 189:125 190:126 191:126 192:127 193:128 194:129 195:130 196:130
I1203 11:56:13.521519 139660688828288 run_squad.py:438] token_to_orig_map: 19:0 20:1 21:1 22:2 23:3 24:4 25:5 26:6 27:7 28:8 29:9 30:10 31:10 32:10 33:10 34:11 35:12 36:13 37:14 38:15 39:16 40:17 41:17 42:17 43:18 44:18 45:19 46:20 47:21 48:21 49:22 50:23 51:24 52:25 53:25 54:25 55:25 56:26 57:26 58:27 59:28 60:29 61:29 62:30 63:31 64:32 65:32 66:32 67:33 68:34 69:35 70:36 71:37 72:38 73:39 74:40 75:41 76:42 77:43 78:44 79:45 80:45 81:45 82:46 83:46 84:47 85:47 86:48 87:49 88:50 89:51 90:51 91:51 92:52 93:53 94:54 95:54 96:55 97:56 98:57 99:58 100:59 101:60 102:61 103:61 104:61 105:62 106:62 107:63 108:64 109:64 110:65 111:66 112:67 113:68 114:69 115:70 116:70 117:71 118:72 119:73 120:74 121:74 122:74 123:75 124:76 125:77 126:78 127:79 128:80 129:81 130:82 131:82 132:83 133:84 134:84 135:84 136:85 137:86 138:87 139:88 140:89 141:90 142:91 143:92 144:92 145:93 146:94 147:95 148:96 149:97 150:98 151:99 152:100 153:100 154:100 155:101 156:102 157:102 158:103 159:104 160:104 161:105 162:105 163:106 164:107 165:108 166:109 167:110 168:111 169:112 170:113 171:114 172:115 173:116 174:116 175:116 176:117 177:117 178:118 179:118 180:119 181:120 182:121 183:122 184:122 185:122 186:123 187:123 188:124 189:125 190:126 191:126 192:127 193:128 194:129 195:130 196:130
INFO:tensorflow:token_is_max_context: 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True
I1203 11:56:13.521690 139660688828288 run_squad.py:440] token_is_max_context: 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True 171:True 172:True 173:True 174:True 175:True 176:True 177:True 178:True 179:True 180:True 181:True 182:True 183:True 184:True 185:True 186:True 187:True 188:True 189:True 190:True 191:True 192:True 193:True 194:True 195:True 196:True
INFO:tensorflow:input_ids: 101 2054 2060 16508 1997 4761 3370 2079 2070 1997 1996 2197 3415 1997 1996 13176 3745 1029 102 3306 11988 2015 2020 4235 1999 2224 2011 1996 6280 2301 10514 9397 5802 3436 1996 3418 4535 1997 2478 1996 2269 1521 1055 2171 1010 2174 3306 11988 2015 2024 2087 4141 9161 24335 6558 1012 4141 1010 3306 3287 11988 2015 2203 1999 1011 1055 1010 2029 2003 1996 2691 4566 2005 3306 14818 5372 19211 1999 1996 2053 22311 6024 2553 1012 17077 1010 2070 2203 1999 1011 15068 1010 8131 1996 8991 13043 2553 1997 2023 5372 15156 2005 9161 24335 2594 4436 1012 2348 11988 2015 1999 8240 5483 2024 10763 2651 1010 8790 1998 5278 9161 24335 2594 8192 13655 1999 2690 3415 2073 1996 8991 13043 1997 2269 1005 1055 2034 2171 2003 4141 1996 2690 2171 1006 2023 8192 2383 2042 2979 2006 2000 1996 12513 1007 1012 1999 9719 1010 2011 5688 1010 11988 2015 3582 1996 3418 4535 1997 2108 2445 2429 2000 1996 2269 1521 1055 2171 1012 2633 1010 1999 2804 2000 3306 1011 5173 11988 2015 2116 2031 3763 1010 5037 1998 3059 4761 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.521919 139660688828288 run_squad.py:442] input_ids: 101 2054 2060 16508 1997 4761 3370 2079 2070 1997 1996 2197 3415 1997 1996 13176 3745 1029 102 3306 11988 2015 2020 4235 1999 2224 2011 1996 6280 2301 10514 9397 5802 3436 1996 3418 4535 1997 2478 1996 2269 1521 1055 2171 1010 2174 3306 11988 2015 2024 2087 4141 9161 24335 6558 1012 4141 1010 3306 3287 11988 2015 2203 1999 1011 1055 1010 2029 2003 1996 2691 4566 2005 3306 14818 5372 19211 1999 1996 2053 22311 6024 2553 1012 17077 1010 2070 2203 1999 1011 15068 1010 8131 1996 8991 13043 2553 1997 2023 5372 15156 2005 9161 24335 2594 4436 1012 2348 11988 2015 1999 8240 5483 2024 10763 2651 1010 8790 1998 5278 9161 24335 2594 8192 13655 1999 2690 3415 2073 1996 8991 13043 1997 2269 1005 1055 2034 2171 2003 4141 1996 2690 2171 1006 2023 8192 2383 2042 2979 2006 2000 1996 12513 1007 1012 1999 9719 1010 2011 5688 1010 11988 2015 3582 1996 3418 4535 1997 2108 2445 2429 2000 1996 2269 1521 1055 2171 1012 2633 1010 1999 2804 2000 3306 1011 5173 11988 2015 2116 2031 3763 1010 5037 1998 3059 4761 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.522124 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.522298 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 188
I1203 11:56:13.522393 139660688828288 run_squad.py:451] start_position: 188
INFO:tensorflow:end_position: 196
I1203 11:56:13.522487 139660688828288 run_squad.py:452] end_position: 196
INFO:tensorflow:answer: many have latin , turkish and italian origin .
I1203 11:56:13.522564 139660688828288 run_squad.py:454] answer: many have latin , turkish and italian origin .
INFO:tensorflow:*** Example ***
I1203 11:56:13.524772 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000018
I1203 11:56:13.524955 139660688828288 run_squad.py:432] unique_id: 1000000018
INFO:tensorflow:example_index: 18
I1203 11:56:13.525058 139660688828288 run_squad.py:433] example_index: 18
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.525146 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what is the function of a " beer engine " ? [SEP] a " beer engine " is a device for pumping beer , originally manually operated and typically used to di ##sp ##ense beer from a cas ##k or container in a pub ' s basement or cellar . [SEP]
I1203 11:56:13.525261 139660688828288 run_squad.py:436] tokens: [CLS] what is the function of a " beer engine " ? [SEP] a " beer engine " is a device for pumping beer , originally manually operated and typically used to di ##sp ##ense beer from a cas ##k or container in a pub ' s basement or cellar . [SEP]
INFO:tensorflow:token_to_orig_map: 13:0 14:1 15:1 16:2 17:2 18:3 19:4 20:5 21:6 22:7 23:8 24:8 25:9 26:10 27:11 28:12 29:13 30:14 31:15 32:16 33:16 34:16 35:17 36:18 37:19 38:20 39:20 40:21 41:22 42:23 43:24 44:25 45:25 46:25 47:26 48:27 49:28 50:28
I1203 11:56:13.525378 139660688828288 run_squad.py:438] token_to_orig_map: 13:0 14:1 15:1 16:2 17:2 18:3 19:4 20:5 21:6 22:7 23:8 24:8 25:9 26:10 27:11 28:12 29:13 30:14 31:15 32:16 33:16 34:16 35:17 36:18 37:19 38:20 39:20 40:21 41:22 42:23 43:24 44:25 45:25 46:25 47:26 48:27 49:28 50:28
INFO:tensorflow:token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True
I1203 11:56:13.525490 139660688828288 run_squad.py:440] token_is_max_context: 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True
INFO:tensorflow:input_ids: 101 2054 2003 1996 3853 1997 1037 1000 5404 3194 1000 1029 102 1037 1000 5404 3194 1000 2003 1037 5080 2005 14107 5404 1010 2761 21118 3498 1998 4050 2109 2000 4487 13102 16700 5404 2013 1037 25222 2243 2030 11661 1999 1037 9047 1005 1055 8102 2030 15423 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.525692 139660688828288 run_squad.py:442] input_ids: 101 2054 2003 1996 3853 1997 1037 1000 5404 3194 1000 1029 102 1037 1000 5404 3194 1000 2003 1037 5080 2005 14107 5404 1010 2761 21118 3498 1998 4050 2109 2000 4487 13102 16700 5404 2013 1037 25222 2243 2030 11661 1999 1037 9047 1005 1055 8102 2030 15423 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.617525 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.618037 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:start_position: 31
I1203 11:56:13.618211 139660688828288 run_squad.py:451] start_position: 31
INFO:tensorflow:end_position: 49
I1203 11:56:13.618308 139660688828288 run_squad.py:452] end_position: 49
INFO:tensorflow:answer: to di ##sp ##ense beer from a cas ##k or container in a pub ' s basement or cellar
I1203 11:56:13.618412 139660688828288 run_squad.py:454] answer: to di ##sp ##ense beer from a cas ##k or container in a pub ' s basement or cellar
INFO:tensorflow:*** Example ***
I1203 11:56:13.622243 139660688828288 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000019
I1203 11:56:13.622463 139660688828288 run_squad.py:432] unique_id: 1000000019
INFO:tensorflow:example_index: 19
I1203 11:56:13.622601 139660688828288 run_squad.py:433] example_index: 19
INFO:tensorflow:doc_span_index: 0
I1203 11:56:13.622711 139660688828288 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] what coats uranium metal in liquid ? [SEP] uranium metal reacts with almost all non - metal elements ( with an exception of the noble gases ) and their compounds , with react ##ivity increasing with temperature . hydro ##ch ##lor ##ic and ni ##tric acids dissolve uranium , but non - ox ##idi ##zing acids other than hydro ##ch ##lor ##ic acid attack the element very slowly . when finely divided , it can react with cold water ; in air , uranium metal becomes coated with a dark layer of uranium oxide . uranium in ore ##s is extracted chemical ##ly and converted into uranium dioxide or other chemical forms usable in industry . [SEP]
I1203 11:56:13.622911 139660688828288 run_squad.py:436] tokens: [CLS] what coats uranium metal in liquid ? [SEP] uranium metal reacts with almost all non - metal elements ( with an exception of the noble gases ) and their compounds , with react ##ivity increasing with temperature . hydro ##ch ##lor ##ic and ni ##tric acids dissolve uranium , but non - ox ##idi ##zing acids other than hydro ##ch ##lor ##ic acid attack the element very slowly . when finely divided , it can react with cold water ; in air , uranium metal becomes coated with a dark layer of uranium oxide . uranium in ore ##s is extracted chemical ##ly and converted into uranium dioxide or other chemical forms usable in industry . [SEP]
INFO:tensorflow:token_to_orig_map: 9:0 10:1 11:2 12:3 13:4 14:5 15:6 16:6 17:6 18:7 19:8 20:8 21:9 22:10 23:11 24:12 25:13 26:14 27:14 28:15 29:16 30:17 31:17 32:18 33:19 34:19 35:20 36:21 37:22 38:22 39:23 40:23 41:23 42:23 43:24 44:25 45:25 46:26 47:27 48:28 49:28 50:29 51:30 52:30 53:30 54:30 55:30 56:31 57:32 58:33 59:34 60:34 61:34 62:34 63:35 64:36 65:37 66:38 67:39 68:40 69:40 70:41 71:42 72:43 73:43 74:44 75:45 76:46 77:47 78:48 79:49 80:49 81:50 82:51 83:51 84:52 85:53 86:54 87:55 88:56 89:57 90:58 91:59 92:60 93:61 94:62 95:62 96:63 97:64 98:65 99:65 100:66 101:67 102:68 103:68 104:69 105:70 106:71 107:72 108:73 109:74 110:75 111:76 112:77 113:78 114:79 115:80 116:80
I1203 11:56:13.623105 139660688828288 run_squad.py:438] token_to_orig_map: 9:0 10:1 11:2 12:3 13:4 14:5 15:6 16:6 17:6 18:7 19:8 20:8 21:9 22:10 23:11 24:12 25:13 26:14 27:14 28:15 29:16 30:17 31:17 32:18 33:19 34:19 35:20 36:21 37:22 38:22 39:23 40:23 41:23 42:23 43:24 44:25 45:25 46:26 47:27 48:28 49:28 50:29 51:30 52:30 53:30 54:30 55:30 56:31 57:32 58:33 59:34 60:34 61:34 62:34 63:35 64:36 65:37 66:38 67:39 68:40 69:40 70:41 71:42 72:43 73:43 74:44 75:45 76:46 77:47 78:48 79:49 80:49 81:50 82:51 83:51 84:52 85:53 86:54 87:55 88:56 89:57 90:58 91:59 92:60 93:61 94:62 95:62 96:63 97:64 98:65 99:65 100:66 101:67 102:68 103:68 104:69 105:70 106:71 107:72 108:73 109:74 110:75 111:76 112:77 113:78 114:79 115:80 116:80
INFO:tensorflow:token_is_max_context: 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True
I1203 11:56:13.623290 139660688828288 run_squad.py:440] token_is_max_context: 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True
INFO:tensorflow:input_ids: 101 2054 15695 14247 3384 1999 6381 1029 102 14247 3384 27325 2007 2471 2035 2512 1011 3384 3787 1006 2007 2019 6453 1997 1996 7015 15865 1007 1998 2037 10099 1010 2007 10509 7730 4852 2007 4860 1012 18479 2818 10626 2594 1998 9152 12412 12737 21969 14247 1010 2021 2512 1011 23060 28173 6774 12737 2060 2084 18479 2818 10626 2594 5648 2886 1996 5783 2200 3254 1012 2043 22126 4055 1010 2009 2064 10509 2007 3147 2300 1025 1999 2250 1010 14247 3384 4150 15026 2007 1037 2601 6741 1997 14247 15772 1012 14247 1999 10848 2015 2003 15901 5072 2135 1998 4991 2046 14247 14384 2030 2060 5072 3596 24013 1999 3068 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.623592 139660688828288 run_squad.py:442] input_ids: 101 2054 15695 14247 3384 1999 6381 1029 102 14247 3384 27325 2007 2471 2035 2512 1011 3384 3787 1006 2007 2019 6453 1997 1996 7015 15865 1007 1998 2037 10099 1010 2007 10509 7730 4852 2007 4860 1012 18479 2818 10626 2594 1998 9152 12412 12737 21969 14247 1010 2021 2512 1011 23060 28173 6774 12737 2060 2084 18479 2818 10626 2594 5648 2886 1996 5783 2200 3254 1012 2043 22126 4055 1010 2009 2064 10509 2007 3147 2300 1025 1999 2250 1010 14247 3384 4150 15026 2007 1037 2601 6741 1997 14247 15772 1012 14247 1999 10848 2015 2003 15901 5072 2135 1998 4991 2046 14247 14384 2030 2060 5072 3596 24013 1999 3068 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.623897 139660688828288 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1203 11:56:13.624171 139660688828288 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:impossible example
I1203 11:56:13.624313 139660688828288 run_squad.py:448] impossible example
INFO:tensorflow:***** Running training *****
I1203 12:04:36.053971 139660688828288 run_squad.py:1203] ***** Running training *****
INFO:tensorflow:  Num orig examples = 130319
I1203 12:04:36.054354 139660688828288 run_squad.py:1204]   Num orig examples = 130319
INFO:tensorflow:  Num split examples = 131944
I1203 12:04:36.054460 139660688828288 run_squad.py:1205]   Num split examples = 131944
INFO:tensorflow:  Batch size = 24
I1203 12:04:36.054545 139660688828288 run_squad.py:1206]   Batch size = 24
INFO:tensorflow:  Num steps = 10859
I1203 12:04:36.054623 139660688828288 run_squad.py:1207]   Num steps = 10859
WARNING:tensorflow:From run_squad.py:691: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

W1203 12:04:36.296595 139660688828288 module_wrapper.py:139] From run_squad.py:691: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

INFO:tensorflow:Querying Tensorflow master (grpc://10.1.118.82:8470) for TPU system metadata.
I1203 12:04:36.436608 139660688828288 tpu_system_metadata.py:78] Querying Tensorflow master (grpc://10.1.118.82:8470) for TPU system metadata.
2019-12-03 12:04:36.438192: W tensorflow/core/distributed_runtime/rpc/grpc_session.cc:370] GrpcSession::ListDevices will initialize the session with an empty graph and other defaults because the session has not yet been created.
INFO:tensorflow:Found TPU system:
I1203 12:04:36.452856 139660688828288 tpu_system_metadata.py:148] Found TPU system:
INFO:tensorflow:*** Num TPU Cores: 8
I1203 12:04:36.453109 139660688828288 tpu_system_metadata.py:149] *** Num TPU Cores: 8
INFO:tensorflow:*** Num TPU Workers: 1
I1203 12:04:36.453222 139660688828288 tpu_system_metadata.py:150] *** Num TPU Workers: 1
INFO:tensorflow:*** Num TPU Cores Per Worker: 8
I1203 12:04:36.453309 139660688828288 tpu_system_metadata.py:152] *** Num TPU Cores Per Worker: 8
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, -1, 8078557097414192577)
I1203 12:04:36.453387 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, -1, 8078557097414192577)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 10463288303569406826)
I1203 12:04:36.454204 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 17179869184, 10463288303569406826)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 17179869184, 15281002826415543107)
I1203 12:04:36.454300 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 17179869184, 15281002826415543107)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 17179869184, 16766211975098988991)
I1203 12:04:36.454389 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 17179869184, 16766211975098988991)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 17179869184, 7739289066451249960)
I1203 12:04:36.454469 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 17179869184, 7739289066451249960)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 17179869184, 7562134758482885320)
I1203 12:04:36.454554 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 17179869184, 7562134758482885320)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 17179869184, 9470549807670972428)
I1203 12:04:36.454631 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 17179869184, 9470549807670972428)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 17179869184, 13869610614254191921)
I1203 12:04:36.454726 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 17179869184, 13869610614254191921)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 17179869184, 1759881672026692344)
I1203 12:04:36.454821 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 17179869184, 1759881672026692344)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 8589934592, 4302427333030094314)
I1203 12:04:36.454900 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 8589934592, 4302427333030094314)
INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 17506310753414732271)
I1203 12:04:36.454980 139660688828288 tpu_system_metadata.py:154] *** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 17179869184, 17506310753414732271)
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W1203 12:04:36.462378 139660688828288 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
W1203 12:04:36.462999 139660688828288 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Calling model_fn.
I1203 12:04:36.475981 139660688828288 estimator.py:1148] Calling model_fn.
WARNING:tensorflow:From run_squad.py:730: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
W1203 12:04:36.500424 139660688828288 deprecation.py:323] From run_squad.py:730: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/data/python/ops/batching.py:276: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
W1203 12:04:36.500720 139660688828288 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/data/python/ops/batching.py:276: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

W1203 12:04:36.574634 139660688828288 module_wrapper.py:139] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

WARNING:tensorflow:From run_squad.py:710: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1203 12:04:36.690506 139660688828288 deprecation.py:323] From run_squad.py:710: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
2019-12-03 12:04:36.767282: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2019-12-03 12:04:36.769076: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2019-12-03 12:04:36.769125: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (7913155a3cb7): /proc/driver/nvidia/version does not exist
INFO:tensorflow:*** Features ***
I1203 12:04:36.788731 139660688828288 run_squad.py:598] *** Features ***
INFO:tensorflow:  name = end_positions, shape = (3,)
I1203 12:04:36.789038 139660688828288 run_squad.py:600]   name = end_positions, shape = (3,)
INFO:tensorflow:  name = input_ids, shape = (3, 384)
I1203 12:04:36.789163 139660688828288 run_squad.py:600]   name = input_ids, shape = (3, 384)
INFO:tensorflow:  name = input_mask, shape = (3, 384)
I1203 12:04:36.789258 139660688828288 run_squad.py:600]   name = input_mask, shape = (3, 384)
INFO:tensorflow:  name = segment_ids, shape = (3, 384)
I1203 12:04:36.789354 139660688828288 run_squad.py:600]   name = segment_ids, shape = (3, 384)
INFO:tensorflow:  name = start_positions, shape = (3,)
I1203 12:04:36.789438 139660688828288 run_squad.py:600]   name = start_positions, shape = (3,)
INFO:tensorflow:  name = unique_ids, shape = (3,)
I1203 12:04:36.789517 139660688828288 run_squad.py:600]   name = unique_ids, shape = (3,)
WARNING:tensorflow:From /content/bert/modeling.py:171: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1203 12:04:36.789761 139660688828288 module_wrapper.py:139] From /content/bert/modeling.py:171: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /content/bert/modeling.py:409: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

W1203 12:04:36.791883 139660688828288 module_wrapper.py:139] From /content/bert/modeling.py:409: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:From /content/bert/modeling.py:490: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.

W1203 12:04:36.823595 139660688828288 module_wrapper.py:139] From /content/bert/modeling.py:490: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.

WARNING:tensorflow:From /content/bert/modeling.py:358: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
W1203 12:04:36.876376 139660688828288 deprecation.py:506] From /content/bert/modeling.py:358: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /content/bert/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
W1203 12:04:36.896015 139660688828288 deprecation.py:323] From /content/bert/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W1203 12:04:36.897552 139660688828288 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From run_squad.py:617: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

W1203 12:04:42.747967 139660688828288 module_wrapper.py:139] From run_squad.py:617: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

INFO:tensorflow:**** Trainable Variables ****
I1203 12:04:43.203573 139660688828288 run_squad.py:634] **** Trainable Variables ****
INFO:tensorflow:  name = bert/embeddings/word_embeddings:0, shape = (30522, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.203895 139660688828288 run_squad.py:640]   name = bert/embeddings/word_embeddings:0, shape = (30522, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/token_type_embeddings:0, shape = (2, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.204073 139660688828288 run_squad.py:640]   name = bert/embeddings/token_type_embeddings:0, shape = (2, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/position_embeddings:0, shape = (512, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.204194 139660688828288 run_squad.py:640]   name = bert/embeddings/position_embeddings:0, shape = (512, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.204306 139660688828288 run_squad.py:640]   name = bert/embeddings/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.204412 139660688828288 run_squad.py:640]   name = bert/embeddings/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.204504 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.204614 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.204707 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.204819 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.204911 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.205014 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.205106 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.205201 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.205292 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.205381 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.205470 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.205565 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.205663 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.205758 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.205882 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.205975 139660688828288 run_squad.py:640]   name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.206070 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.206170 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.206259 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.206354 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.206451 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.206549 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.206642 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.206738 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.206848 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.206940 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.207040 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.207137 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.207227 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.207320 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.207410 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.207498 139660688828288 run_squad.py:640]   name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.207588 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.207683 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.207773 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.207890 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.207984 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.208092 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.208183 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.208279 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.208368 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.208455 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.208546 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.208643 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.208734 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.208850 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.208952 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.209051 139660688828288 run_squad.py:640]   name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.209142 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.209239 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.209328 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.209436 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.209526 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.209622 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.209710 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.209819 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.209911 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.210000 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.210095 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.210206 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.291528 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.291964 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.292140 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.292277 139660688828288 run_squad.py:640]   name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.292421 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.292588 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.292728 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.292895 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.293026 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.293169 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.293305 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.293440 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.293587 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.293718 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.293875 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.294020 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.294152 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.294295 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.294426 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.294569 139660688828288 run_squad.py:640]   name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.294728 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.294919 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.295054 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.295197 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.295328 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.295464 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.295608 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.295747 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.295902 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.296030 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.296156 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.296298 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.296423 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.296573 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.296705 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.296857 139660688828288 run_squad.py:640]   name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.296990 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.297130 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.297264 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.297398 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.297527 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.297679 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.297829 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.297973 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.298100 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.298230 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.298354 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.298492 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.298636 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.298774 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.298938 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.299069 139660688828288 run_squad.py:640]   name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.299203 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.299341 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.299472 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.299624 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.299757 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.299921 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.300048 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.300190 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.300314 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.300444 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.300584 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.300726 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.300882 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.301019 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.301266 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.301437 139660688828288 run_squad.py:640]   name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.301587 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.301734 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.301894 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.302033 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.302168 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.302302 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.302433 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.302582 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.302719 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.302872 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.303014 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.303155 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.303282 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.303421 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.303555 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.303689 139660688828288 run_squad.py:640]   name = bert/encoder/layer_8/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.303840 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.303983 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.304115 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.304250 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.304383 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.304514 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.304656 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.304815 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.304953 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.305084 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.305208 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.305348 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.305473 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.305623 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.305754 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.305907 139660688828288 run_squad.py:640]   name = bert/encoder/layer_9/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.306037 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.306176 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.306307 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.306439 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.306580 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.306720 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.306874 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.307015 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.307145 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.307275 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.307397 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.307535 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.307679 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.307840 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.307974 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.308109 139660688828288 run_squad.py:640]   name = bert/encoder/layer_10/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.308240 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.308375 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.308504 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.308661 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.308813 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.308967 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.309096 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.309237 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.309362 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.309490 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.309633 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.309772 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.309927 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.310062 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.310192 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.310317 139660688828288 run_squad.py:640]   name = bert/encoder/layer_11/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.310443 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.310594 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.310724 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.310887 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.311016 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.311155 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.311280 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.311414 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.311554 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.311685 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.311836 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.311986 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.312117 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.312256 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.312382 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.312511 139660688828288 run_squad.py:640]   name = bert/encoder/layer_12/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.312648 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.312807 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.312945 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.313086 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.313216 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.313350 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.313480 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.313630 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.313761 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.313911 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.314038 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.314180 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.314306 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.314442 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.314579 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.314731 139660688828288 run_squad.py:640]   name = bert/encoder/layer_13/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.314884 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.315021 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.315155 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.315290 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.315419 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.315564 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.315700 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.315863 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.315992 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.316121 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.316244 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.316382 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.316508 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.316657 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.316805 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.316938 139660688828288 run_squad.py:640]   name = bert/encoder/layer_14/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.317068 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.317205 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.317335 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.317469 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.317610 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.317750 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.317903 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.318043 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.318171 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.318299 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.318427 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.318572 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.318706 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.318868 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.319013 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.319141 139660688828288 run_squad.py:640]   name = bert/encoder/layer_15/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.319270 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.319408 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.319535 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.319687 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.319838 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.319982 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.320109 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.320246 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.320377 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.320503 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.320645 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.320800 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.320937 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.321070 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.321199 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.321324 139660688828288 run_squad.py:640]   name = bert/encoder/layer_16/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.321447 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.321599 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.321727 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.321889 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.322018 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.322156 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.322285 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.322419 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.322563 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.322693 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.322846 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.322986 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.323113 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.323251 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.323374 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.323501 139660688828288 run_squad.py:640]   name = bert/encoder/layer_17/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.323641 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.323808 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.323949 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.324087 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.324220 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.324354 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.324498 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.324647 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.324801 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.324937 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.325064 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.325203 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.325330 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.325468 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.325608 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.325741 139660688828288 run_squad.py:640]   name = bert/encoder/layer_18/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.325899 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.326041 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.326173 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.326304 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.326433 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.326581 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.326717 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.326879 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.327009 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.327140 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.327262 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.327398 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.327525 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.327676 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.327826 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.327957 139660688828288 run_squad.py:640]   name = bert/encoder/layer_19/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.328089 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.328225 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.328354 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.328490 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.328631 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.328772 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.328918 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.329070 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.329201 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.329328 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.329456 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.329606 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.329738 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.329897 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.330031 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.330158 139660688828288 run_squad.py:640]   name = bert/encoder/layer_20/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.330284 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.330422 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.330557 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.330703 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.330853 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.330995 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.331123 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.331260 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.331390 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.331515 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.331659 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.331812 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.331950 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.332086 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.332214 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.332341 139660688828288 run_squad.py:640]   name = bert/encoder/layer_21/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.332463 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.332617 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.332747 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.332912 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.333042 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.333179 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.333310 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.333444 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.333588 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.333716 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.333868 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.334008 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.334135 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.334274 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.334400 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.334526 139660688828288 run_squad.py:640]   name = bert/encoder/layer_22/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.334683 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.334848 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.334982 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.335119 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.335252 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.335383 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.335512 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.335661 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.335808 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.335943 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1203 12:04:43.336068 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1203 12:04:43.336209 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.336334 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.336470 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.336611 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.336740 139660688828288 run_squad.py:640]   name = bert/encoder/layer_23/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/pooler/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1203 12:04:43.336892 139660688828288 run_squad.py:640]   name = bert/pooler/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/pooler/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1203 12:04:43.337032 139660688828288 run_squad.py:640]   name = bert/pooler/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = cls/squad/output_weights:0, shape = (2, 1024)
I1203 12:04:43.337167 139660688828288 run_squad.py:640]   name = cls/squad/output_weights:0, shape = (2, 1024)
INFO:tensorflow:  name = cls/squad/output_bias:0, shape = (2,)
I1203 12:04:43.337300 139660688828288 run_squad.py:640]   name = cls/squad/output_bias:0, shape = (2,)
WARNING:tensorflow:From /content/bert/optimization.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.

W1203 12:04:43.356570 139660688828288 module_wrapper.py:139] From /content/bert/optimization.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.

WARNING:tensorflow:From /content/bert/optimization.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W1203 12:04:43.357693 139660688828288 module_wrapper.py:139] From /content/bert/optimization.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1203 12:04:43.687987 139660688828288 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From run_squad.py:627: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

W1203 12:05:03.694965 139660688828288 module_wrapper.py:139] From run_squad.py:627: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

WARNING:tensorflow:From run_squad.py:628: The name tf.train.Scaffold is deprecated. Please use tf.compat.v1.train.Scaffold instead.

W1203 12:05:05.537943 139660688828288 module_wrapper.py:139] From run_squad.py:628: The name tf.train.Scaffold is deprecated. Please use tf.compat.v1.train.Scaffold instead.

INFO:tensorflow:Create CheckpointSaverHook.
I1203 12:05:06.551318 139660688828288 basic_session_run_hooks.py:541] Create CheckpointSaverHook.
INFO:tensorflow:Done calling model_fn.
I1203 12:05:07.195392 139660688828288 estimator.py:1150] Done calling model_fn.
INFO:tensorflow:TPU job name worker
I1203 12:05:11.544954 139660688828288 tpu_estimator.py:506] TPU job name worker
INFO:tensorflow:Graph was finalized.
I1203 12:05:14.376625 139660688828288 monitored_session.py:240] Graph was finalized.
INFO:tensorflow:Running local_init_op.
I1203 12:05:41.918381 139660688828288 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1203 12:05:42.942277 139660688828288 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into gs://bertnlpdemo/bert_output/model.ckpt.
I1203 12:05:57.744137 139660688828288 basic_session_run_hooks.py:606] Saving checkpoints for 0 into gs://bertnlpdemo/bert_output/model.ckpt.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py:751: Variable.load (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Prefer Variable.assign which has equivalent behavior in 2.X.
W1203 12:07:03.671180 139660688828288 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py:751: Variable.load (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Prefer Variable.assign which has equivalent behavior in 2.X.
INFO:tensorflow:Initialized dataset iterators in 1 seconds
I1203 12:07:05.800500 139660688828288 util.py:98] Initialized dataset iterators in 1 seconds
INFO:tensorflow:Installing graceful shutdown hook.
I1203 12:07:05.801046 139660688828288 session_support.py:332] Installing graceful shutdown hook.
2019-12-03 12:07:05.801525: W tensorflow/core/distributed_runtime/rpc/grpc_session.cc:370] GrpcSession::ListDevices will initialize the session with an empty graph and other defaults because the session has not yet been created.
INFO:tensorflow:Creating heartbeat manager for ['/job:worker/replica:0/task:0/device:CPU:0']
I1203 12:07:05.806523 139660688828288 session_support.py:82] Creating heartbeat manager for ['/job:worker/replica:0/task:0/device:CPU:0']
INFO:tensorflow:Configuring worker heartbeat: shutdown_mode: WAIT_FOR_COORDINATOR

I1203 12:07:05.808742 139660688828288 session_support.py:105] Configuring worker heartbeat: shutdown_mode: WAIT_FOR_COORDINATOR

INFO:tensorflow:Init TPU system
I1203 12:07:05.812534 139660688828288 tpu_estimator.py:567] Init TPU system
INFO:tensorflow:Initialized TPU in 7 seconds
I1203 12:07:13.264216 139660688828288 tpu_estimator.py:576] Initialized TPU in 7 seconds
INFO:tensorflow:Starting infeed thread controller.
I1203 12:07:13.265170 139658717681408 tpu_estimator.py:521] Starting infeed thread controller.
INFO:tensorflow:Starting outfeed thread controller.
I1203 12:07:13.265672 139658709288704 tpu_estimator.py:540] Starting outfeed thread controller.
INFO:tensorflow:Enqueue next (1000) batch(es) of data to infeed.
I1203 12:07:14.305896 139660688828288 tpu_estimator.py:600] Enqueue next (1000) batch(es) of data to infeed.
INFO:tensorflow:Dequeue next (1000) batch(es) of data from outfeed.
I1203 12:07:14.306252 139660688828288 tpu_estimator.py:604] Dequeue next (1000) batch(es) of data from outfeed.
INFO:tensorflow:Outfeed finished for iteration (0, 0)
I1203 12:08:25.359033 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (0, 0)
INFO:tensorflow:Outfeed finished for iteration (0, 258)
I1203 12:09:25.450175 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (0, 258)
INFO:tensorflow:Outfeed finished for iteration (0, 516)
I1203 12:10:25.541866 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (0, 516)
INFO:tensorflow:Outfeed finished for iteration (0, 774)
I1203 12:11:25.633392 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (0, 774)
INFO:tensorflow:Saving checkpoints for 1000 into gs://bertnlpdemo/bert_output/model.ckpt.
I1203 12:12:20.143985 139660688828288 basic_session_run_hooks.py:606] Saving checkpoints for 1000 into gs://bertnlpdemo/bert_output/model.ckpt.
INFO:tensorflow:loss = 2.1162045, step = 1000
I1203 12:13:57.150325 139660688828288 basic_session_run_hooks.py:262] loss = 2.1162045, step = 1000
INFO:tensorflow:Enqueue next (1000) batch(es) of data to infeed.
I1203 12:13:57.153505 139660688828288 tpu_estimator.py:600] Enqueue next (1000) batch(es) of data to infeed.
INFO:tensorflow:Dequeue next (1000) batch(es) of data from outfeed.
I1203 12:13:57.153829 139660688828288 tpu_estimator.py:604] Dequeue next (1000) batch(es) of data from outfeed.
INFO:tensorflow:Outfeed finished for iteration (1, 0)
I1203 12:14:08.450512 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (1, 0)
INFO:tensorflow:Outfeed finished for iteration (1, 258)
I1203 12:15:08.559914 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (1, 258)
INFO:tensorflow:Outfeed finished for iteration (1, 516)
I1203 12:16:08.670165 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (1, 516)
INFO:tensorflow:Outfeed finished for iteration (1, 774)
I1203 12:17:08.779463 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (1, 774)
INFO:tensorflow:Saving checkpoints for 2000 into gs://bertnlpdemo/bert_output/model.ckpt.
I1203 12:18:03.079922 139660688828288 basic_session_run_hooks.py:606] Saving checkpoints for 2000 into gs://bertnlpdemo/bert_output/model.ckpt.
INFO:tensorflow:loss = 0.16789994, step = 2000 (327.071 sec)
I1203 12:19:24.221167 139660688828288 basic_session_run_hooks.py:260] loss = 0.16789994, step = 2000 (327.071 sec)
INFO:tensorflow:global_step/sec: 3.05744
I1203 12:19:24.223610 139660688828288 tpu_estimator.py:2307] global_step/sec: 3.05744
INFO:tensorflow:examples/sec: 73.3786
I1203 12:19:24.225670 139660688828288 tpu_estimator.py:2308] examples/sec: 73.3786
INFO:tensorflow:Enqueue next (1000) batch(es) of data to infeed.
I1203 12:19:24.227692 139660688828288 tpu_estimator.py:600] Enqueue next (1000) batch(es) of data to infeed.
INFO:tensorflow:Dequeue next (1000) batch(es) of data from outfeed.
I1203 12:19:24.227924 139660688828288 tpu_estimator.py:604] Dequeue next (1000) batch(es) of data from outfeed.
INFO:tensorflow:Outfeed finished for iteration (2, 0)
I1203 12:19:27.065313 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (2, 0)
INFO:tensorflow:Outfeed finished for iteration (2, 258)
I1203 12:20:27.173299 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (2, 258)
INFO:tensorflow:Outfeed finished for iteration (2, 516)
I1203 12:21:27.285262 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (2, 516)
INFO:tensorflow:Outfeed finished for iteration (2, 774)
I1203 12:22:27.393604 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (2, 774)
INFO:tensorflow:Saving checkpoints for 3000 into gs://bertnlpdemo/bert_output/model.ckpt.
I1203 12:23:21.806639 139660688828288 basic_session_run_hooks.py:606] Saving checkpoints for 3000 into gs://bertnlpdemo/bert_output/model.ckpt.
INFO:tensorflow:loss = 0.23783581, step = 3000 (290.130 sec)
I1203 12:24:14.351115 139660688828288 basic_session_run_hooks.py:260] loss = 0.23783581, step = 3000 (290.130 sec)
INFO:tensorflow:global_step/sec: 3.44674
I1203 12:24:14.352766 139660688828288 tpu_estimator.py:2307] global_step/sec: 3.44674
INFO:tensorflow:examples/sec: 82.7217
I1203 12:24:14.353132 139660688828288 tpu_estimator.py:2308] examples/sec: 82.7217
INFO:tensorflow:Enqueue next (1000) batch(es) of data to infeed.
I1203 12:24:14.354492 139660688828288 tpu_estimator.py:600] Enqueue next (1000) batch(es) of data to infeed.
INFO:tensorflow:Dequeue next (1000) batch(es) of data from outfeed.
I1203 12:24:14.354699 139660688828288 tpu_estimator.py:604] Dequeue next (1000) batch(es) of data from outfeed.
INFO:tensorflow:Outfeed finished for iteration (3, 0)
I1203 12:24:17.176265 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (3, 0)
INFO:tensorflow:Outfeed finished for iteration (3, 258)
I1203 12:25:17.289111 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (3, 258)
INFO:tensorflow:Outfeed finished for iteration (3, 516)
I1203 12:26:17.405220 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (3, 516)
INFO:tensorflow:Outfeed finished for iteration (3, 774)
I1203 12:27:17.521445 139658709288704 tpu_estimator.py:279] Outfeed finished for iteration (3, 774)

Create Testing File

We are creating input_file.json as a blank json file and then writing the data in SQUAD format in the file.

  • touch is used to create a file
  • %%writefile is used to write a file in the colab

You can pass your own questions and context in the below file.





In [0]:



    


!touch input_file.json



    











In [0]:



    


%%writefile input_file.json
{
    "version": "v2.0",
    "data": [
        {
            "title": "your_title",
            "paragraphs": [
                {
                    "qas": [
                        {
                            "question": "Who is current CEO?",
                            "id": "56ddde6b9a695914005b9628",
                            "is_impossible": ""
                        },
                        {
                            "question": "Who founded google?",
                            "id": "56ddde6b9a695914005b9629",
                            "is_impossible": ""
                        },
                        {
                            "question": "when did IPO take place?",
                            "id": "56ddde6b9a695914005b962a",
                            "is_impossible": ""
                        }
                    ],
                    "context": "Google was founded in 1998 by Larry Page and Sergey Brin while they were Ph.D. students at Stanford University in California. Together they own about 14 percent of its shares and control 56 percent of the stockholder voting power through supervoting stock. They incorporated Google as a privately held company on September 4, 1998. An initial public offering (IPO) took place on August 19, 2004, and Google moved to its headquarters in Mountain View, California, nicknamed the Googleplex. In August 2015, Google announced plans to reorganize its various interests as a conglomerate called Alphabet Inc. Google is Alphabet's leading subsidiary and will continue to be the umbrella company for Alphabet's Internet interests. Sundar Pichai was appointed CEO of Google, replacing Larry Page who became the CEO of Alphabet."                
                 }
            ]
        }
    ]
}



    


















    






Overwriting input_file.json

Prediction

Below is the command to perform your own custom prediction, that is you can change the input_file.json by providing your paragraph and questions after then execute the below command.





In [0]:



    


!python run_squad.py \
  --vocab_file=$BUCKET_NAME/uncased_L-24_H-1024_A-16/vocab.txt \
  --bert_config_file=$BUCKET_NAME/uncased_L-24_H-1024_A-16/bert_config.json \
  --init_checkpoint=$OUTPUT_DIR/model.ckpt-10859 \
  --do_train=False \
  --max_query_length=30  \
  --do_predict=True \
  --predict_file=input_file.json \
  --predict_batch_size=8 \
  --n_best_size=3 \
  --max_seq_length=384 \
  --doc_stride=128 \
  --output_dir=output/



    


















    






WARNING:tensorflow:From /content/bert/optimization.py:87: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING:tensorflow:From run_squad.py:1283: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

WARNING:tensorflow:From run_squad.py:1127: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

W1207 12:10:47.304228 140202182711168 module_wrapper.py:139] From run_squad.py:1127: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From run_squad.py:1127: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

W1207 12:10:47.304487 140202182711168 module_wrapper.py:139] From run_squad.py:1127: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /content/bert/modeling.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W1207 12:10:47.304673 140202182711168 module_wrapper.py:139] From /content/bert/modeling.py:93: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From run_squad.py:1133: The name tf.gfile.MakeDirs is deprecated. Please use tf.io.gfile.makedirs instead.

W1207 12:10:48.564931 140202182711168 module_wrapper.py:139] From run_squad.py:1133: The name tf.gfile.MakeDirs is deprecated. Please use tf.io.gfile.makedirs instead.

WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W1207 12:10:48.774623 140202182711168 lazy_loader.py:50] 
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

I1207 12:10:49.286037 140202182711168 utils.py:141] NumExpr defaulting to 2 threads.
WARNING:tensorflow:Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f83094ddbf8>) includes params argument, but params are not passed to Estimator.
W1207 12:10:49.699651 140202182711168 estimator.py:1994] Estimator's model_fn (<function model_fn_builder.<locals>.model_fn at 0x7f83094ddbf8>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:Using config: {'_model_dir': 'output/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f82fb6b7518>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}
I1207 12:10:49.700918 140202182711168 estimator.py:212] Using config: {'_model_dir': 'output/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 1000, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f82fb6b7518>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=1000, num_shards=8, num_cores_per_replica=None, per_host_input_for_training=3, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}
INFO:tensorflow:_TPUContext: eval_on_tpu True
I1207 12:10:49.701192 140202182711168 tpu_context.py:220] _TPUContext: eval_on_tpu True
WARNING:tensorflow:eval_on_tpu ignored because use_tpu is False.
W1207 12:10:49.701765 140202182711168 tpu_context.py:222] eval_on_tpu ignored because use_tpu is False.
WARNING:tensorflow:From run_squad.py:229: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.

W1207 12:10:49.701951 140202182711168 module_wrapper.py:139] From run_squad.py:229: The name tf.gfile.Open is deprecated. Please use tf.io.gfile.GFile instead.

WARNING:tensorflow:From run_squad.py:1065: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

W1207 12:10:49.703150 140202182711168 module_wrapper.py:139] From run_squad.py:1065: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

WARNING:tensorflow:From run_squad.py:431: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

W1207 12:10:49.706889 140202182711168 module_wrapper.py:139] From run_squad.py:431: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

INFO:tensorflow:*** Example ***
I1207 12:10:49.707021 140202182711168 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000000
I1207 12:10:49.707089 140202182711168 run_squad.py:432] unique_id: 1000000000
INFO:tensorflow:example_index: 0
I1207 12:10:49.707143 140202182711168 run_squad.py:433] example_index: 0
INFO:tensorflow:doc_span_index: 0
I1207 12:10:49.707192 140202182711168 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] who is current ceo ? [SEP] google was founded in 1998 by larry page and sergey br ##in while they were ph . d . students at stanford university in california . together they own about 14 percent of its shares and control 56 percent of the stock ##holder voting power through super ##vot ##ing stock . they incorporated google as a privately held company on september 4 , 1998 . an initial public offering ( ip ##o ) took place on august 19 , 2004 , and google moved to its headquarters in mountain view , california , nicknamed the google ##plex . in august 2015 , google announced plans to re ##org ##ani ##ze its various interests as a conglomerate called alphabet inc . google is alphabet ' s leading subsidiary and will continue to be the umbrella company for alphabet ' s internet interests . sun ##dar pic ##hai was appointed ceo of google , replacing larry page who became the ceo of alphabet . [SEP]
I1207 12:10:49.707288 140202182711168 run_squad.py:436] tokens: [CLS] who is current ceo ? [SEP] google was founded in 1998 by larry page and sergey br ##in while they were ph . d . students at stanford university in california . together they own about 14 percent of its shares and control 56 percent of the stock ##holder voting power through super ##vot ##ing stock . they incorporated google as a privately held company on september 4 , 1998 . an initial public offering ( ip ##o ) took place on august 19 , 2004 , and google moved to its headquarters in mountain view , california , nicknamed the google ##plex . in august 2015 , google announced plans to re ##org ##ani ##ze its various interests as a conglomerate called alphabet inc . google is alphabet ' s leading subsidiary and will continue to be the umbrella company for alphabet ' s internet interests . sun ##dar pic ##hai was appointed ceo of google , replacing larry page who became the ceo of alphabet . [SEP]
INFO:tensorflow:token_to_orig_map: 7:0 8:1 9:2 10:3 11:4 12:5 13:6 14:7 15:8 16:9 17:10 18:10 19:11 20:12 21:13 22:14 23:14 24:14 25:14 26:15 27:16 28:17 29:18 30:19 31:20 32:20 33:21 34:22 35:23 36:24 37:25 38:26 39:27 40:28 41:29 42:30 43:31 44:32 45:33 46:34 47:35 48:36 49:36 50:37 51:38 52:39 53:40 54:40 55:40 56:41 57:41 58:42 59:43 60:44 61:45 62:46 63:47 64:48 65:49 66:50 67:51 68:52 69:52 70:53 71:53 72:54 73:55 74:56 75:57 76:58 77:58 78:58 79:58 80:59 81:60 82:61 83:62 84:63 85:63 86:64 87:64 88:65 89:66 90:67 91:68 92:69 93:70 94:71 95:72 96:73 97:73 98:74 99:74 100:75 101:76 102:77 103:77 104:77 105:78 106:79 107:80 108:80 109:81 110:82 111:83 112:84 113:85 114:85 115:85 116:85 117:86 118:87 119:88 120:89 121:90 122:91 123:92 124:93 125:94 126:94 127:95 128:96 129:97 130:97 131:97 132:98 133:99 134:100 135:101 136:102 137:103 138:104 139:105 140:106 141:107 142:108 143:109 144:109 145:109 146:110 147:111 148:111 149:112 150:112 151:113 152:113 153:114 154:115 155:116 156:117 157:118 158:118 159:119 160:120 161:121 162:122 163:123 164:124 165:125 166:126 167:127 168:127
I1207 12:10:49.707389 140202182711168 run_squad.py:438] token_to_orig_map: 7:0 8:1 9:2 10:3 11:4 12:5 13:6 14:7 15:8 16:9 17:10 18:10 19:11 20:12 21:13 22:14 23:14 24:14 25:14 26:15 27:16 28:17 29:18 30:19 31:20 32:20 33:21 34:22 35:23 36:24 37:25 38:26 39:27 40:28 41:29 42:30 43:31 44:32 45:33 46:34 47:35 48:36 49:36 50:37 51:38 52:39 53:40 54:40 55:40 56:41 57:41 58:42 59:43 60:44 61:45 62:46 63:47 64:48 65:49 66:50 67:51 68:52 69:52 70:53 71:53 72:54 73:55 74:56 75:57 76:58 77:58 78:58 79:58 80:59 81:60 82:61 83:62 84:63 85:63 86:64 87:64 88:65 89:66 90:67 91:68 92:69 93:70 94:71 95:72 96:73 97:73 98:74 99:74 100:75 101:76 102:77 103:77 104:77 105:78 106:79 107:80 108:80 109:81 110:82 111:83 112:84 113:85 114:85 115:85 116:85 117:86 118:87 119:88 120:89 121:90 122:91 123:92 124:93 125:94 126:94 127:95 128:96 129:97 130:97 131:97 132:98 133:99 134:100 135:101 136:102 137:103 138:104 139:105 140:106 141:107 142:108 143:109 144:109 145:109 146:110 147:111 148:111 149:112 150:112 151:113 152:113 153:114 154:115 155:116 156:117 157:118 158:118 159:119 160:120 161:121 162:122 163:123 164:124 165:125 166:126 167:127 168:127
INFO:tensorflow:token_is_max_context: 7:True 8:True 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True
I1207 12:10:49.707542 140202182711168 run_squad.py:440] token_is_max_context: 7:True 8:True 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True
INFO:tensorflow:input_ids: 101 2040 2003 2783 5766 1029 102 8224 2001 2631 1999 2687 2011 6554 3931 1998 22703 7987 2378 2096 2027 2020 6887 1012 1040 1012 2493 2012 8422 2118 1999 2662 1012 2362 2027 2219 2055 2403 3867 1997 2049 6661 1998 2491 5179 3867 1997 1996 4518 14528 6830 2373 2083 3565 22994 2075 4518 1012 2027 5100 8224 2004 1037 9139 2218 2194 2006 2244 1018 1010 2687 1012 2019 3988 2270 5378 1006 12997 2080 1007 2165 2173 2006 2257 2539 1010 2432 1010 1998 8224 2333 2000 2049 4075 1999 3137 3193 1010 2662 1010 9919 1996 8224 19386 1012 1999 2257 2325 1010 8224 2623 3488 2000 2128 21759 7088 4371 2049 2536 5426 2004 1037 22453 2170 12440 4297 1012 8224 2003 12440 1005 1055 2877 7506 1998 2097 3613 2000 2022 1996 12977 2194 2005 12440 1005 1055 4274 5426 1012 3103 7662 27263 10932 2001 2805 5766 1997 8224 1010 6419 6554 3931 2040 2150 1996 5766 1997 12440 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.707693 140202182711168 run_squad.py:442] input_ids: 101 2040 2003 2783 5766 1029 102 8224 2001 2631 1999 2687 2011 6554 3931 1998 22703 7987 2378 2096 2027 2020 6887 1012 1040 1012 2493 2012 8422 2118 1999 2662 1012 2362 2027 2219 2055 2403 3867 1997 2049 6661 1998 2491 5179 3867 1997 1996 4518 14528 6830 2373 2083 3565 22994 2075 4518 1012 2027 5100 8224 2004 1037 9139 2218 2194 2006 2244 1018 1010 2687 1012 2019 3988 2270 5378 1006 12997 2080 1007 2165 2173 2006 2257 2539 1010 2432 1010 1998 8224 2333 2000 2049 4075 1999 3137 3193 1010 2662 1010 9919 1996 8224 19386 1012 1999 2257 2325 1010 8224 2623 3488 2000 2128 21759 7088 4371 2049 2536 5426 2004 1037 22453 2170 12440 4297 1012 8224 2003 12440 1005 1055 2877 7506 1998 2097 3613 2000 2022 1996 12977 2194 2005 12440 1005 1055 4274 5426 1012 3103 7662 27263 10932 2001 2805 5766 1997 8224 1010 6419 6554 3931 2040 2150 1996 5766 1997 12440 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.707823 140202182711168 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.707945 140202182711168 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:*** Example ***
I1207 12:10:49.711749 140202182711168 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000001
I1207 12:10:49.711882 140202182711168 run_squad.py:432] unique_id: 1000000001
INFO:tensorflow:example_index: 1
I1207 12:10:49.711943 140202182711168 run_squad.py:433] example_index: 1
INFO:tensorflow:doc_span_index: 0
I1207 12:10:49.711996 140202182711168 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] who founded google ? [SEP] google was founded in 1998 by larry page and sergey br ##in while they were ph . d . students at stanford university in california . together they own about 14 percent of its shares and control 56 percent of the stock ##holder voting power through super ##vot ##ing stock . they incorporated google as a privately held company on september 4 , 1998 . an initial public offering ( ip ##o ) took place on august 19 , 2004 , and google moved to its headquarters in mountain view , california , nicknamed the google ##plex . in august 2015 , google announced plans to re ##org ##ani ##ze its various interests as a conglomerate called alphabet inc . google is alphabet ' s leading subsidiary and will continue to be the umbrella company for alphabet ' s internet interests . sun ##dar pic ##hai was appointed ceo of google , replacing larry page who became the ceo of alphabet . [SEP]
I1207 12:10:49.712097 140202182711168 run_squad.py:436] tokens: [CLS] who founded google ? [SEP] google was founded in 1998 by larry page and sergey br ##in while they were ph . d . students at stanford university in california . together they own about 14 percent of its shares and control 56 percent of the stock ##holder voting power through super ##vot ##ing stock . they incorporated google as a privately held company on september 4 , 1998 . an initial public offering ( ip ##o ) took place on august 19 , 2004 , and google moved to its headquarters in mountain view , california , nicknamed the google ##plex . in august 2015 , google announced plans to re ##org ##ani ##ze its various interests as a conglomerate called alphabet inc . google is alphabet ' s leading subsidiary and will continue to be the umbrella company for alphabet ' s internet interests . sun ##dar pic ##hai was appointed ceo of google , replacing larry page who became the ceo of alphabet . [SEP]
INFO:tensorflow:token_to_orig_map: 6:0 7:1 8:2 9:3 10:4 11:5 12:6 13:7 14:8 15:9 16:10 17:10 18:11 19:12 20:13 21:14 22:14 23:14 24:14 25:15 26:16 27:17 28:18 29:19 30:20 31:20 32:21 33:22 34:23 35:24 36:25 37:26 38:27 39:28 40:29 41:30 42:31 43:32 44:33 45:34 46:35 47:36 48:36 49:37 50:38 51:39 52:40 53:40 54:40 55:41 56:41 57:42 58:43 59:44 60:45 61:46 62:47 63:48 64:49 65:50 66:51 67:52 68:52 69:53 70:53 71:54 72:55 73:56 74:57 75:58 76:58 77:58 78:58 79:59 80:60 81:61 82:62 83:63 84:63 85:64 86:64 87:65 88:66 89:67 90:68 91:69 92:70 93:71 94:72 95:73 96:73 97:74 98:74 99:75 100:76 101:77 102:77 103:77 104:78 105:79 106:80 107:80 108:81 109:82 110:83 111:84 112:85 113:85 114:85 115:85 116:86 117:87 118:88 119:89 120:90 121:91 122:92 123:93 124:94 125:94 126:95 127:96 128:97 129:97 130:97 131:98 132:99 133:100 134:101 135:102 136:103 137:104 138:105 139:106 140:107 141:108 142:109 143:109 144:109 145:110 146:111 147:111 148:112 149:112 150:113 151:113 152:114 153:115 154:116 155:117 156:118 157:118 158:119 159:120 160:121 161:122 162:123 163:124 164:125 165:126 166:127 167:127
I1207 12:10:49.741870 140202182711168 run_squad.py:438] token_to_orig_map: 6:0 7:1 8:2 9:3 10:4 11:5 12:6 13:7 14:8 15:9 16:10 17:10 18:11 19:12 20:13 21:14 22:14 23:14 24:14 25:15 26:16 27:17 28:18 29:19 30:20 31:20 32:21 33:22 34:23 35:24 36:25 37:26 38:27 39:28 40:29 41:30 42:31 43:32 44:33 45:34 46:35 47:36 48:36 49:37 50:38 51:39 52:40 53:40 54:40 55:41 56:41 57:42 58:43 59:44 60:45 61:46 62:47 63:48 64:49 65:50 66:51 67:52 68:52 69:53 70:53 71:54 72:55 73:56 74:57 75:58 76:58 77:58 78:58 79:59 80:60 81:61 82:62 83:63 84:63 85:64 86:64 87:65 88:66 89:67 90:68 91:69 92:70 93:71 94:72 95:73 96:73 97:74 98:74 99:75 100:76 101:77 102:77 103:77 104:78 105:79 106:80 107:80 108:81 109:82 110:83 111:84 112:85 113:85 114:85 115:85 116:86 117:87 118:88 119:89 120:90 121:91 122:92 123:93 124:94 125:94 126:95 127:96 128:97 129:97 130:97 131:98 132:99 133:100 134:101 135:102 136:103 137:104 138:105 139:106 140:107 141:108 142:109 143:109 144:109 145:110 146:111 147:111 148:112 149:112 150:113 151:113 152:114 153:115 154:116 155:117 156:118 157:118 158:119 159:120 160:121 161:122 162:123 163:124 164:125 165:126 166:127 167:127
INFO:tensorflow:token_is_max_context: 6:True 7:True 8:True 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True
I1207 12:10:49.742178 140202182711168 run_squad.py:440] token_is_max_context: 6:True 7:True 8:True 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True
INFO:tensorflow:input_ids: 101 2040 2631 8224 1029 102 8224 2001 2631 1999 2687 2011 6554 3931 1998 22703 7987 2378 2096 2027 2020 6887 1012 1040 1012 2493 2012 8422 2118 1999 2662 1012 2362 2027 2219 2055 2403 3867 1997 2049 6661 1998 2491 5179 3867 1997 1996 4518 14528 6830 2373 2083 3565 22994 2075 4518 1012 2027 5100 8224 2004 1037 9139 2218 2194 2006 2244 1018 1010 2687 1012 2019 3988 2270 5378 1006 12997 2080 1007 2165 2173 2006 2257 2539 1010 2432 1010 1998 8224 2333 2000 2049 4075 1999 3137 3193 1010 2662 1010 9919 1996 8224 19386 1012 1999 2257 2325 1010 8224 2623 3488 2000 2128 21759 7088 4371 2049 2536 5426 2004 1037 22453 2170 12440 4297 1012 8224 2003 12440 1005 1055 2877 7506 1998 2097 3613 2000 2022 1996 12977 2194 2005 12440 1005 1055 4274 5426 1012 3103 7662 27263 10932 2001 2805 5766 1997 8224 1010 6419 6554 3931 2040 2150 1996 5766 1997 12440 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.742563 140202182711168 run_squad.py:442] input_ids: 101 2040 2631 8224 1029 102 8224 2001 2631 1999 2687 2011 6554 3931 1998 22703 7987 2378 2096 2027 2020 6887 1012 1040 1012 2493 2012 8422 2118 1999 2662 1012 2362 2027 2219 2055 2403 3867 1997 2049 6661 1998 2491 5179 3867 1997 1996 4518 14528 6830 2373 2083 3565 22994 2075 4518 1012 2027 5100 8224 2004 1037 9139 2218 2194 2006 2244 1018 1010 2687 1012 2019 3988 2270 5378 1006 12997 2080 1007 2165 2173 2006 2257 2539 1010 2432 1010 1998 8224 2333 2000 2049 4075 1999 3137 3193 1010 2662 1010 9919 1996 8224 19386 1012 1999 2257 2325 1010 8224 2623 3488 2000 2128 21759 7088 4371 2049 2536 5426 2004 1037 22453 2170 12440 4297 1012 8224 2003 12440 1005 1055 2877 7506 1998 2097 3613 2000 2022 1996 12977 2194 2005 12440 1005 1055 4274 5426 1012 3103 7662 27263 10932 2001 2805 5766 1997 8224 1010 6419 6554 3931 2040 2150 1996 5766 1997 12440 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.742852 140202182711168 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.743113 140202182711168 run_squad.py:446] segment_ids: 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:*** Example ***
I1207 12:10:49.747266 140202182711168 run_squad.py:431] *** Example ***
INFO:tensorflow:unique_id: 1000000002
I1207 12:10:49.747435 140202182711168 run_squad.py:432] unique_id: 1000000002
INFO:tensorflow:example_index: 2
I1207 12:10:49.747532 140202182711168 run_squad.py:433] example_index: 2
INFO:tensorflow:doc_span_index: 0
I1207 12:10:49.747608 140202182711168 run_squad.py:434] doc_span_index: 0
INFO:tensorflow:tokens: [CLS] when did ip ##o take place ? [SEP] google was founded in 1998 by larry page and sergey br ##in while they were ph . d . students at stanford university in california . together they own about 14 percent of its shares and control 56 percent of the stock ##holder voting power through super ##vot ##ing stock . they incorporated google as a privately held company on september 4 , 1998 . an initial public offering ( ip ##o ) took place on august 19 , 2004 , and google moved to its headquarters in mountain view , california , nicknamed the google ##plex . in august 2015 , google announced plans to re ##org ##ani ##ze its various interests as a conglomerate called alphabet inc . google is alphabet ' s leading subsidiary and will continue to be the umbrella company for alphabet ' s internet interests . sun ##dar pic ##hai was appointed ceo of google , replacing larry page who became the ceo of alphabet . [SEP]
I1207 12:10:49.747749 140202182711168 run_squad.py:436] tokens: [CLS] when did ip ##o take place ? [SEP] google was founded in 1998 by larry page and sergey br ##in while they were ph . d . students at stanford university in california . together they own about 14 percent of its shares and control 56 percent of the stock ##holder voting power through super ##vot ##ing stock . they incorporated google as a privately held company on september 4 , 1998 . an initial public offering ( ip ##o ) took place on august 19 , 2004 , and google moved to its headquarters in mountain view , california , nicknamed the google ##plex . in august 2015 , google announced plans to re ##org ##ani ##ze its various interests as a conglomerate called alphabet inc . google is alphabet ' s leading subsidiary and will continue to be the umbrella company for alphabet ' s internet interests . sun ##dar pic ##hai was appointed ceo of google , replacing larry page who became the ceo of alphabet . [SEP]
INFO:tensorflow:token_to_orig_map: 9:0 10:1 11:2 12:3 13:4 14:5 15:6 16:7 17:8 18:9 19:10 20:10 21:11 22:12 23:13 24:14 25:14 26:14 27:14 28:15 29:16 30:17 31:18 32:19 33:20 34:20 35:21 36:22 37:23 38:24 39:25 40:26 41:27 42:28 43:29 44:30 45:31 46:32 47:33 48:34 49:35 50:36 51:36 52:37 53:38 54:39 55:40 56:40 57:40 58:41 59:41 60:42 61:43 62:44 63:45 64:46 65:47 66:48 67:49 68:50 69:51 70:52 71:52 72:53 73:53 74:54 75:55 76:56 77:57 78:58 79:58 80:58 81:58 82:59 83:60 84:61 85:62 86:63 87:63 88:64 89:64 90:65 91:66 92:67 93:68 94:69 95:70 96:71 97:72 98:73 99:73 100:74 101:74 102:75 103:76 104:77 105:77 106:77 107:78 108:79 109:80 110:80 111:81 112:82 113:83 114:84 115:85 116:85 117:85 118:85 119:86 120:87 121:88 122:89 123:90 124:91 125:92 126:93 127:94 128:94 129:95 130:96 131:97 132:97 133:97 134:98 135:99 136:100 137:101 138:102 139:103 140:104 141:105 142:106 143:107 144:108 145:109 146:109 147:109 148:110 149:111 150:111 151:112 152:112 153:113 154:113 155:114 156:115 157:116 158:117 159:118 160:118 161:119 162:120 163:121 164:122 165:123 166:124 167:125 168:126 169:127 170:127
I1207 12:10:49.747977 140202182711168 run_squad.py:438] token_to_orig_map: 9:0 10:1 11:2 12:3 13:4 14:5 15:6 16:7 17:8 18:9 19:10 20:10 21:11 22:12 23:13 24:14 25:14 26:14 27:14 28:15 29:16 30:17 31:18 32:19 33:20 34:20 35:21 36:22 37:23 38:24 39:25 40:26 41:27 42:28 43:29 44:30 45:31 46:32 47:33 48:34 49:35 50:36 51:36 52:37 53:38 54:39 55:40 56:40 57:40 58:41 59:41 60:42 61:43 62:44 63:45 64:46 65:47 66:48 67:49 68:50 69:51 70:52 71:52 72:53 73:53 74:54 75:55 76:56 77:57 78:58 79:58 80:58 81:58 82:59 83:60 84:61 85:62 86:63 87:63 88:64 89:64 90:65 91:66 92:67 93:68 94:69 95:70 96:71 97:72 98:73 99:73 100:74 101:74 102:75 103:76 104:77 105:77 106:77 107:78 108:79 109:80 110:80 111:81 112:82 113:83 114:84 115:85 116:85 117:85 118:85 119:86 120:87 121:88 122:89 123:90 124:91 125:92 126:93 127:94 128:94 129:95 130:96 131:97 132:97 133:97 134:98 135:99 136:100 137:101 138:102 139:103 140:104 141:105 142:106 143:107 144:108 145:109 146:109 147:109 148:110 149:111 150:111 151:112 152:112 153:113 154:113 155:114 156:115 157:116 158:117 159:118 160:118 161:119 162:120 163:121 164:122 165:123 166:124 167:125 168:126 169:127 170:127
INFO:tensorflow:token_is_max_context: 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True
I1207 12:10:49.748219 140202182711168 run_squad.py:440] token_is_max_context: 9:True 10:True 11:True 12:True 13:True 14:True 15:True 16:True 17:True 18:True 19:True 20:True 21:True 22:True 23:True 24:True 25:True 26:True 27:True 28:True 29:True 30:True 31:True 32:True 33:True 34:True 35:True 36:True 37:True 38:True 39:True 40:True 41:True 42:True 43:True 44:True 45:True 46:True 47:True 48:True 49:True 50:True 51:True 52:True 53:True 54:True 55:True 56:True 57:True 58:True 59:True 60:True 61:True 62:True 63:True 64:True 65:True 66:True 67:True 68:True 69:True 70:True 71:True 72:True 73:True 74:True 75:True 76:True 77:True 78:True 79:True 80:True 81:True 82:True 83:True 84:True 85:True 86:True 87:True 88:True 89:True 90:True 91:True 92:True 93:True 94:True 95:True 96:True 97:True 98:True 99:True 100:True 101:True 102:True 103:True 104:True 105:True 106:True 107:True 108:True 109:True 110:True 111:True 112:True 113:True 114:True 115:True 116:True 117:True 118:True 119:True 120:True 121:True 122:True 123:True 124:True 125:True 126:True 127:True 128:True 129:True 130:True 131:True 132:True 133:True 134:True 135:True 136:True 137:True 138:True 139:True 140:True 141:True 142:True 143:True 144:True 145:True 146:True 147:True 148:True 149:True 150:True 151:True 152:True 153:True 154:True 155:True 156:True 157:True 158:True 159:True 160:True 161:True 162:True 163:True 164:True 165:True 166:True 167:True 168:True 169:True 170:True
INFO:tensorflow:input_ids: 101 2043 2106 12997 2080 2202 2173 1029 102 8224 2001 2631 1999 2687 2011 6554 3931 1998 22703 7987 2378 2096 2027 2020 6887 1012 1040 1012 2493 2012 8422 2118 1999 2662 1012 2362 2027 2219 2055 2403 3867 1997 2049 6661 1998 2491 5179 3867 1997 1996 4518 14528 6830 2373 2083 3565 22994 2075 4518 1012 2027 5100 8224 2004 1037 9139 2218 2194 2006 2244 1018 1010 2687 1012 2019 3988 2270 5378 1006 12997 2080 1007 2165 2173 2006 2257 2539 1010 2432 1010 1998 8224 2333 2000 2049 4075 1999 3137 3193 1010 2662 1010 9919 1996 8224 19386 1012 1999 2257 2325 1010 8224 2623 3488 2000 2128 21759 7088 4371 2049 2536 5426 2004 1037 22453 2170 12440 4297 1012 8224 2003 12440 1005 1055 2877 7506 1998 2097 3613 2000 2022 1996 12977 2194 2005 12440 1005 1055 4274 5426 1012 3103 7662 27263 10932 2001 2805 5766 1997 8224 1010 6419 6554 3931 2040 2150 1996 5766 1997 12440 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.748446 140202182711168 run_squad.py:442] input_ids: 101 2043 2106 12997 2080 2202 2173 1029 102 8224 2001 2631 1999 2687 2011 6554 3931 1998 22703 7987 2378 2096 2027 2020 6887 1012 1040 1012 2493 2012 8422 2118 1999 2662 1012 2362 2027 2219 2055 2403 3867 1997 2049 6661 1998 2491 5179 3867 1997 1996 4518 14528 6830 2373 2083 3565 22994 2075 4518 1012 2027 5100 8224 2004 1037 9139 2218 2194 2006 2244 1018 1010 2687 1012 2019 3988 2270 5378 1006 12997 2080 1007 2165 2173 2006 2257 2539 1010 2432 1010 1998 8224 2333 2000 2049 4075 1999 3137 3193 1010 2662 1010 9919 1996 8224 19386 1012 1999 2257 2325 1010 8224 2623 3488 2000 2128 21759 7088 4371 2049 2536 5426 2004 1037 22453 2170 12440 4297 1012 8224 2003 12440 1005 1055 2877 7506 1998 2097 3613 2000 2022 1996 12977 2194 2005 12440 1005 1055 4274 5426 1012 3103 7662 27263 10932 2001 2805 5766 1997 8224 1010 6419 6554 3931 2040 2150 1996 5766 1997 12440 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.748804 140202182711168 run_squad.py:444] input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
I1207 12:10:49.749049 140202182711168 run_squad.py:446] segment_ids: 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
INFO:tensorflow:***** Running predictions *****
I1207 12:10:49.749744 140202182711168 run_squad.py:1240] ***** Running predictions *****
INFO:tensorflow:  Num orig examples = 3
I1207 12:10:49.749995 140202182711168 run_squad.py:1241]   Num orig examples = 3
INFO:tensorflow:  Num split examples = 3
I1207 12:10:49.750131 140202182711168 run_squad.py:1242]   Num split examples = 3
INFO:tensorflow:  Batch size = 8
I1207 12:10:49.750242 140202182711168 run_squad.py:1243]   Batch size = 8
WARNING:tensorflow:From run_squad.py:691: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

W1207 12:10:49.750467 140202182711168 module_wrapper.py:139] From run_squad.py:691: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

INFO:tensorflow:Could not find trained model in model_dir: output/, running initialization to predict.
I1207 12:10:49.750977 140202182711168 estimator.py:615] Could not find trained model in model_dir: output/, running initialization to predict.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W1207 12:10:49.756826 140202182711168 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From run_squad.py:730: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
W1207 12:10:49.779744 140202182711168 deprecation.py:323] From run_squad.py:730: map_and_batch (from tensorflow.contrib.data.python.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.map_and_batch(...)`.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/data/python/ops/batching.py:276: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
W1207 12:10:49.780065 140202182711168 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/data/python/ops/batching.py:276: map_and_batch (from tensorflow.python.data.experimental.ops.batching) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map(map_func, num_parallel_calls)` followed by `tf.data.Dataset.batch(batch_size, drop_remainder)`. Static tf.data optimizations will take care of using the fused implementation.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

W1207 12:10:49.842756 140202182711168 module_wrapper.py:139] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

WARNING:tensorflow:From run_squad.py:710: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1207 12:10:49.958963 140202182711168 deprecation.py:323] From run_squad.py:710: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:tensorflow:Calling model_fn.
I1207 12:10:49.976630 140202182711168 estimator.py:1148] Calling model_fn.
INFO:tensorflow:Running infer on CPU
I1207 12:10:49.976942 140202182711168 tpu_estimator.py:3124] Running infer on CPU
INFO:tensorflow:*** Features ***
I1207 12:10:49.977321 140202182711168 run_squad.py:598] *** Features ***
INFO:tensorflow:  name = input_ids, shape = (?, 384)
I1207 12:10:49.977520 140202182711168 run_squad.py:600]   name = input_ids, shape = (?, 384)
INFO:tensorflow:  name = input_mask, shape = (?, 384)
I1207 12:10:49.977654 140202182711168 run_squad.py:600]   name = input_mask, shape = (?, 384)
INFO:tensorflow:  name = segment_ids, shape = (?, 384)
I1207 12:10:49.977768 140202182711168 run_squad.py:600]   name = segment_ids, shape = (?, 384)
INFO:tensorflow:  name = unique_ids, shape = (?,)
I1207 12:10:49.977869 140202182711168 run_squad.py:600]   name = unique_ids, shape = (?,)
WARNING:tensorflow:From /content/bert/modeling.py:171: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1207 12:10:49.980969 140202182711168 module_wrapper.py:139] From /content/bert/modeling.py:171: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /content/bert/modeling.py:409: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

W1207 12:10:49.982532 140202182711168 module_wrapper.py:139] From /content/bert/modeling.py:409: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:From /content/bert/modeling.py:490: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.

W1207 12:10:50.012444 140202182711168 module_wrapper.py:139] From /content/bert/modeling.py:490: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.

WARNING:tensorflow:From /content/bert/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
W1207 12:10:50.071944 140202182711168 deprecation.py:323] From /content/bert/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W1207 12:10:50.073563 140202182711168 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From run_squad.py:617: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

W1207 12:10:54.048188 140202182711168 module_wrapper.py:139] From run_squad.py:617: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

WARNING:tensorflow:From run_squad.py:632: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

W1207 12:10:54.430091 140202182711168 module_wrapper.py:139] From run_squad.py:632: The name tf.train.init_from_checkpoint is deprecated. Please use tf.compat.v1.train.init_from_checkpoint instead.

INFO:tensorflow:**** Trainable Variables ****
I1207 12:10:56.067378 140202182711168 run_squad.py:634] **** Trainable Variables ****
INFO:tensorflow:  name = bert/embeddings/word_embeddings:0, shape = (30522, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.067712 140202182711168 run_squad.py:640]   name = bert/embeddings/word_embeddings:0, shape = (30522, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/token_type_embeddings:0, shape = (2, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.067846 140202182711168 run_squad.py:640]   name = bert/embeddings/token_type_embeddings:0, shape = (2, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/position_embeddings:0, shape = (512, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.067930 140202182711168 run_squad.py:640]   name = bert/embeddings/position_embeddings:0, shape = (512, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068006 140202182711168 run_squad.py:640]   name = bert/embeddings/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/embeddings/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068076 140202182711168 run_squad.py:640]   name = bert/embeddings/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.068141 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068213 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.068280 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068351 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.068430 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068504 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.068577 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068646 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068711 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.068774 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.068837 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.068906 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.068970 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069038 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069101 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069164 140202182711168 run_squad.py:640]   name = bert/encoder/layer_0/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.069229 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069298 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.069361 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069442 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.069509 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069584 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.069651 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069719 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069783 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.069845 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.069907 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.069975 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.070037 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070105 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070168 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070230 140202182711168 run_squad.py:640]   name = bert/encoder/layer_1/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.070293 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070362 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.070453 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070567 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.070636 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070706 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.070770 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070838 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070901 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.070964 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.071026 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.071096 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.071159 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071225 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071290 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071364 140202182711168 run_squad.py:640]   name = bert/encoder/layer_2/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.071441 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071513 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.071582 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071650 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.071712 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071780 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.071843 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071910 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.071972 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.072035 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.072097 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.072164 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.072226 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.112028 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.112329 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.112537 140202182711168 run_squad.py:640]   name = bert/encoder/layer_3/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.112665 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.112802 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.112918 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.113048 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.113167 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.113310 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.113468 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.113618 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.113744 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.113876 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.114002 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.114139 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.114294 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.114451 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.114586 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.114717 140202182711168 run_squad.py:640]   name = bert/encoder/layer_4/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.114842 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.114979 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.115108 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.115254 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.115391 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.115553 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.115687 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.115828 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.115959 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.116089 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.116219 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.116379 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.116536 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.116677 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.116807 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.116940 140202182711168 run_squad.py:640]   name = bert/encoder/layer_5/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.117066 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.117437 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.117597 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.117747 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.117882 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.118025 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.118170 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.118323 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.118474 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.118607 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.118737 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.118879 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.119007 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.119145 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.119399 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.119639 140202182711168 run_squad.py:640]   name = bert/encoder/layer_6/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.119784 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.119931 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.120065 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.120227 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.120367 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.120539 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.120680 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.120828 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.120969 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.121127 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.121359 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.121545 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.121688 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.121842 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.121979 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.122112 140202182711168 run_squad.py:640]   name = bert/encoder/layer_7/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.122256 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.122402 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.122561 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.122704 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.122837 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.122983 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.123114 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.123265 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.123403 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.123556 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.123690 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.123840 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.123970 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.124111 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.124256 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_8/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.124389 140202182711168 run_squad.py:640]   name = bert/encoder/layer_8/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.124540 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.124680 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.124813 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.124957 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.125087 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.125240 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.125376 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.125538 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.125676 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.125811 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.125967 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.126109 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.126251 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.126426 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.126590 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_9/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.126770 140202182711168 run_squad.py:640]   name = bert/encoder/layer_9/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.126910 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.127054 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.127194 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.127336 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.127491 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.127632 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.127765 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.127905 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.128037 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.128166 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.128306 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.128463 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.128603 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.128741 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.128872 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_10/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.128998 140202182711168 run_squad.py:640]   name = bert/encoder/layer_10/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.129125 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.129278 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.129409 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.129566 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.129697 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.129837 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.129969 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.130110 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.130250 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.130394 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.130542 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.130684 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.130819 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.130957 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.131089 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_11/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.131226 140202182711168 run_squad.py:640]   name = bert/encoder/layer_11/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.131369 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.131532 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.131664 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.131805 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.131940 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.132080 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.132221 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.132361 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.132512 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.132644 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.132773 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.132912 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.133042 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.133179 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.133324 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_12/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.133470 140202182711168 run_squad.py:640]   name = bert/encoder/layer_12/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.133604 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.133745 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.133872 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.134013 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.134142 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.134311 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.134468 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.134610 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.134743 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.134875 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.135001 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.135142 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.135282 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.135437 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.135572 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_13/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.135700 140202182711168 run_squad.py:640]   name = bert/encoder/layer_13/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.135828 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.135967 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.136093 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.136241 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.136380 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.136539 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.136670 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.136807 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.136937 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.137066 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.137203 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.137346 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.137498 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.137640 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.137769 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_14/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.137896 140202182711168 run_squad.py:640]   name = bert/encoder/layer_14/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.138023 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.138160 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.138302 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.138461 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.138594 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.138734 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.138863 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.138997 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.139127 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.139263 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.139392 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.139553 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.139682 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.139824 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.139955 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_15/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.140079 140202182711168 run_squad.py:640]   name = bert/encoder/layer_15/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.140215 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.140356 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.140505 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.140645 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.140773 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.140914 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.141041 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.141178 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.141331 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.141481 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.141615 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.141756 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.141886 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.142026 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.142154 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_16/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.142294 140202182711168 run_squad.py:640]   name = bert/encoder/layer_16/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.142440 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.142585 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.142715 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.142854 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.142983 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.143122 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.143261 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.143400 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.143551 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.143679 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.143809 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.143947 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.144075 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.144221 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.144351 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_17/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.144500 140202182711168 run_squad.py:640]   name = bert/encoder/layer_17/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.144628 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.144765 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.144894 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.145030 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.145161 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.145310 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.145457 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.145603 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.145731 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.145860 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.145988 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.146124 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.146267 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.146435 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.146569 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_18/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.146700 140202182711168 run_squad.py:640]   name = bert/encoder/layer_18/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.146825 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.146964 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.147096 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.147243 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.147373 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.147532 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.147664 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.147804 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.147933 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.148062 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.148196 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.148334 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.148485 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.148632 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.148762 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_19/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.148892 140202182711168 run_squad.py:640]   name = bert/encoder/layer_19/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.149019 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.149158 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.149296 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.149447 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.149583 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.149719 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.149849 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.149984 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.150111 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.150249 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.150386 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.150542 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.150677 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.150813 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.150936 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_20/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.151063 140202182711168 run_squad.py:640]   name = bert/encoder/layer_20/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.151196 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.151350 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.151496 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.151637 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.151770 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.151905 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.152032 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.152168 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.152307 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.152451 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.152580 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.152718 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.152851 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.152986 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.153116 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_21/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.153255 140202182711168 run_squad.py:640]   name = bert/encoder/layer_21/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.153383 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.153542 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.153669 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.153805 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.153935 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.154067 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.154205 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.154366 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.154511 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.154643 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.154769 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.154906 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.155035 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.155169 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.155311 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_22/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.155454 140202182711168 run_squad.py:640]   name = bert/encoder/layer_22/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.155586 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/query/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.155726 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/query/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.155853 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/key/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.155990 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/key/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.156120 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/value/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.156266 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/self/value/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.156426 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.156570 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.156703 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.156833 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/attention/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
I1207 12:10:56.156959 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/intermediate/dense/kernel:0, shape = (1024, 4096), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
I1207 12:10:56.157096 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/intermediate/dense/bias:0, shape = (4096,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.157234 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/output/dense/kernel:0, shape = (4096, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.157373 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/output/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.157522 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/output/LayerNorm/beta:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/encoder/layer_23/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.157650 140202182711168 run_squad.py:640]   name = bert/encoder/layer_23/output/LayerNorm/gamma:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/pooler/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.157777 140202182711168 run_squad.py:640]   name = bert/pooler/dense/kernel:0, shape = (1024, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = bert/pooler/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
I1207 12:10:56.157916 140202182711168 run_squad.py:640]   name = bert/pooler/dense/bias:0, shape = (1024,), *INIT_FROM_CKPT*
INFO:tensorflow:  name = cls/squad/output_weights:0, shape = (2, 1024), *INIT_FROM_CKPT*
I1207 12:10:56.158046 140202182711168 run_squad.py:640]   name = cls/squad/output_weights:0, shape = (2, 1024), *INIT_FROM_CKPT*
INFO:tensorflow:  name = cls/squad/output_bias:0, shape = (2,), *INIT_FROM_CKPT*
I1207 12:10:56.158196 140202182711168 run_squad.py:640]   name = cls/squad/output_bias:0, shape = (2,), *INIT_FROM_CKPT*
INFO:tensorflow:Done calling model_fn.
I1207 12:10:56.159069 140202182711168 estimator.py:1150] Done calling model_fn.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1207 12:10:56.473066 140202182711168 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
INFO:tensorflow:Graph was finalized.
I1207 12:10:57.068443 140202182711168 monitored_session.py:240] Graph was finalized.
2019-12-07 12:10:57.086768: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz
2019-12-07 12:10:57.089612: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1eeaa00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2019-12-07 12:10:57.089684: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2019-12-07 12:10:57.097910: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2019-12-07 12:10:57.099879: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2019-12-07 12:10:57.099935: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (d8c06fee4ca6): /proc/driver/nvidia/version does not exist
INFO:tensorflow:Running local_init_op.
I1207 12:12:33.675824 140202182711168 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1207 12:12:33.851502 140202182711168 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Processing example: 0
I1207 12:12:51.340045 140202182711168 run_squad.py:1259] Processing example: 0
INFO:tensorflow:prediction_loop marked as finished
I1207 12:12:51.460263 140202182711168 error_handling.py:101] prediction_loop marked as finished
INFO:tensorflow:prediction_loop marked as finished
I1207 12:12:51.460633 140202182711168 error_handling.py:101] prediction_loop marked as finished
INFO:tensorflow:Writing predictions to: output/predictions.json
I1207 12:12:51.460788 140202182711168 run_squad.py:745] Writing predictions to: output/predictions.json
INFO:tensorflow:Writing nbest to: output/nbest_predictions.json
I1207 12:12:51.460851 140202182711168 run_squad.py:746] Writing nbest to: output/nbest_predictions.json

Content source: psyllost/02819

Similar notebooks:

notebook.community | gallery | about