Flair Training Sequence Labeling Models

Version: 0.3, February 2020

Download: This and various other Jupyter notebooks are available from my GitHub repo.

For the Flair tutorial 7 license and copyright restrictions, see the website below. For all the parts that I added, consider the license to be Creative Commons Attribution-ShareAlike 4.0 International License (CA BY-SA 4.0).

Based on the Flair Tutorial 7 Training a Model.

This tutorial is using the CoNLL-03 Named Entity Recognition data set. See this website for more details and to download an independent copy of the data set.

Training a Sequence Labeling Model

We will need the following modules:



In [6]:

    
from flair.data import Corpus
from flair.datasets import WNUT_17
from flair.embeddings import TokenEmbeddings, WordEmbeddings, StackedEmbeddings
from typing import List

If you want to use the CoNLL-03 corpus, you need to download it and unpack it in your Flair data and model folder. This folder should be in your home-directory and it is named .flair. Once you have downloaded the corpus, unpack it into a folder .flair/datasets/conll_03. If you do not want to use the CoNLL-03 corpus, but rather the free W-NUT 17 corpus, you can use the Flair command: WNUT_17()

If you decide to download the CoNLL-03 corpus, adapt the following code. We load the W-NUT17 corpus and down-sample it to 10% of its size:



In [7]:

    
corpus: Corpus = WNUT_17().downsample(0.1)
print(corpus)









    



2020-02-03 08:33:26,978 Reading data from C:\Users\damir\.flair\datasets\wnut_17
2020-02-03 08:33:26,986 Train: C:\Users\damir\.flair\datasets\wnut_17\wnut17train.conll
2020-02-03 08:33:26,987 Dev: C:\Users\damir\.flair\datasets\wnut_17\emerging.dev.conll
2020-02-03 08:33:26,987 Test: C:\Users\damir\.flair\datasets\wnut_17\emerging.test.annotated
Corpus: 339 train + 101 dev + 129 test sentences

Declare the tag type to be predicted:



In [8]:

    
tag_type = 'ner'

Create the tag-dictionary for the tag-type:



In [10]:

    
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)
print(tag_dictionary)









    



Dictionary with 28 tags: <unk>, O, S-person, B-location, E-location, S-corporation, B-product, E-product, S-location, B-person, E-person, I-product, I-location, S-group, S-product, B-creative-work, E-creative-work, B-group, E-group, S-creative-work, I-creative-work, B-corporation, I-corporation, E-corporation, I-person, I-group, <START>, <STOP>

Load the embeddings:



In [12]:

    
embedding_types: List[TokenEmbeddings] = [

    WordEmbeddings('glove'),

    # comment in this line to use character embeddings
    # CharacterEmbeddings(),

    # comment in these lines to use flair embeddings
    # FlairEmbeddings('news-forward'),
    # FlairEmbeddings('news-backward'),
]
embeddings: StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)

Load and initialize the sequence tagger:



In [13]:

    
from flair.models import SequenceTagger

tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag_dictionary,
                                        tag_type=tag_type,
                                        use_crf=True)

Load and initialize the trainer:



In [14]:

    
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

If you have a GPU (otherwise maybe tweak the batch size, etc.), run the training with 150 epochs:



In [15]:

    
trainer.train('resources/taggers/example-ner',
              learning_rate=0.1,
              mini_batch_size=32,
              max_epochs=150)









    



2020-02-03 08:51:41,233 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:41,238 Model: "SequenceTagger(
  (embeddings): StackedEmbeddings(
    (list_embedding_0): WordEmbeddings('glove')
  )
  (word_dropout): WordDropout(p=0.05)
  (locked_dropout): LockedDropout(p=0.5)
  (embedding2nn): Linear(in_features=100, out_features=100, bias=True)
  (rnn): LSTM(100, 256, batch_first=True, bidirectional=True)
  (linear): Linear(in_features=512, out_features=28, bias=True)
  (beta): 1.0
  (weights): None
  (weight_tensor) None
)"
2020-02-03 08:51:41,239 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:41,239 Corpus: "Corpus: 339 train + 101 dev + 129 test sentences"
2020-02-03 08:51:41,239 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:41,239 Parameters:
2020-02-03 08:51:41,239  - learning_rate: "0.1"
2020-02-03 08:51:41,239  - mini_batch_size: "32"
2020-02-03 08:51:41,239  - patience: "3"
2020-02-03 08:51:41,239  - anneal_factor: "0.5"
2020-02-03 08:51:41,239  - max_epochs: "150"
2020-02-03 08:51:41,239  - shuffle: "True"
2020-02-03 08:51:41,239  - train_with_dev: "False"
2020-02-03 08:51:41,239  - batch_growth_annealing: "False"
2020-02-03 08:51:41,239 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:41,239 Model training base path: "resources\taggers\example-ner"
2020-02-03 08:51:41,239 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:41,239 Device: cpu
2020-02-03 08:51:41,255 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:41,255 Embeddings storage mode: cpu
2020-02-03 08:51:41,255 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:41,726 epoch 1 - iter 1/11 - loss 62.42728424 - samples/sec: 67.88
2020-02-03 08:51:42,272 epoch 1 - iter 2/11 - loss 54.72027969 - samples/sec: 66.44
2020-02-03 08:51:42,827 epoch 1 - iter 3/11 - loss 46.89226659 - samples/sec: 63.41
2020-02-03 08:51:43,429 epoch 1 - iter 4/11 - loss 38.98322487 - samples/sec: 59.10
2020-02-03 08:51:43,912 epoch 1 - iter 5/11 - loss 32.80314579 - samples/sec: 73.87
2020-02-03 08:51:44,436 epoch 1 - iter 6/11 - loss 29.20696052 - samples/sec: 68.74
2020-02-03 08:51:45,019 epoch 1 - iter 7/11 - loss 25.94517367 - samples/sec: 60.11
2020-02-03 08:51:45,647 epoch 1 - iter 8/11 - loss 24.24179721 - samples/sec: 55.63
2020-02-03 08:51:46,160 epoch 1 - iter 9/11 - loss 22.36033890 - samples/sec: 69.15
2020-02-03 08:51:46,665 epoch 1 - iter 10/11 - loss 20.74594336 - samples/sec: 70.66
2020-02-03 08:51:47,108 epoch 1 - iter 11/11 - loss 19.64247400 - samples/sec: 81.53
2020-02-03 08:51:47,158 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:47,158 EPOCH 1 done: loss 19.6425 - lr 0.1000
2020-02-03 08:51:47,662 DEV : loss 7.944559097290039 - score 0.0
2020-02-03 08:51:47,672 BAD EPOCHS (no improvement): 0
2020-02-03 08:51:50,914 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:51,357 epoch 2 - iter 1/11 - loss 6.77688932 - samples/sec: 72.20
2020-02-03 08:51:51,851 epoch 2 - iter 2/11 - loss 7.16069794 - samples/sec: 72.24
2020-02-03 08:51:52,314 epoch 2 - iter 3/11 - loss 7.90957387 - samples/sec: 77.53
2020-02-03 08:51:52,797 epoch 2 - iter 4/11 - loss 7.38920498 - samples/sec: 75.70
2020-02-03 08:51:53,349 epoch 2 - iter 5/11 - loss 7.81130352 - samples/sec: 63.81
2020-02-03 08:51:53,853 epoch 2 - iter 6/11 - loss 7.51350935 - samples/sec: 71.78
2020-02-03 08:51:54,377 epoch 2 - iter 7/11 - loss 7.77998481 - samples/sec: 67.58
2020-02-03 08:51:54,929 epoch 2 - iter 8/11 - loss 7.86519498 - samples/sec: 63.83
2020-02-03 08:51:55,442 epoch 2 - iter 9/11 - loss 7.64515585 - samples/sec: 69.36
2020-02-03 08:51:55,958 epoch 2 - iter 10/11 - loss 7.39100981 - samples/sec: 69.12
2020-02-03 08:51:56,341 epoch 2 - iter 11/11 - loss 7.32374092 - samples/sec: 96.19
2020-02-03 08:51:56,399 ----------------------------------------------------------------------------------------------------
2020-02-03 08:51:56,401 EPOCH 2 done: loss 7.3237 - lr 0.1000
2020-02-03 08:51:56,854 DEV : loss 7.51180362701416 - score 0.0
2020-02-03 08:51:56,854 BAD EPOCHS (no improvement): 1
2020-02-03 08:52:00,036 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:00,439 epoch 3 - iter 1/11 - loss 6.61197853 - samples/sec: 81.03
2020-02-03 08:52:00,981 epoch 3 - iter 2/11 - loss 6.84571886 - samples/sec: 66.47
2020-02-03 08:52:01,477 epoch 3 - iter 3/11 - loss 7.75518131 - samples/sec: 72.13
2020-02-03 08:52:02,020 epoch 3 - iter 4/11 - loss 7.97363853 - samples/sec: 66.43
2020-02-03 08:52:02,515 epoch 3 - iter 5/11 - loss 8.02803211 - samples/sec: 72.21
2020-02-03 08:52:03,003 epoch 3 - iter 6/11 - loss 8.02778800 - samples/sec: 73.05
2020-02-03 08:52:03,491 epoch 3 - iter 7/11 - loss 7.90520736 - samples/sec: 74.38
2020-02-03 08:52:04,004 epoch 3 - iter 8/11 - loss 7.75512600 - samples/sec: 70.62
2020-02-03 08:52:04,528 epoch 3 - iter 9/11 - loss 7.19794289 - samples/sec: 67.61
2020-02-03 08:52:05,072 epoch 3 - iter 10/11 - loss 6.85181365 - samples/sec: 64.84
2020-02-03 08:52:05,515 epoch 3 - iter 11/11 - loss 6.64778536 - samples/sec: 83.08
2020-02-03 08:52:05,576 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:05,576 EPOCH 3 done: loss 6.6478 - lr 0.1000
2020-02-03 08:52:05,968 DEV : loss 7.254205226898193 - score 0.0
2020-02-03 08:52:05,979 BAD EPOCHS (no improvement): 2
2020-02-03 08:52:09,353 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:09,824 epoch 4 - iter 1/11 - loss 6.32489300 - samples/sec: 67.91
2020-02-03 08:52:10,341 epoch 4 - iter 2/11 - loss 6.15437698 - samples/sec: 68.99
2020-02-03 08:52:10,804 epoch 4 - iter 3/11 - loss 6.20432377 - samples/sec: 79.39
2020-02-03 08:52:11,356 epoch 4 - iter 4/11 - loss 6.55288947 - samples/sec: 63.77
2020-02-03 08:52:11,893 epoch 4 - iter 5/11 - loss 6.27991924 - samples/sec: 65.80
2020-02-03 08:52:12,407 epoch 4 - iter 6/11 - loss 6.51886368 - samples/sec: 70.21
2020-02-03 08:52:12,933 epoch 4 - iter 7/11 - loss 6.52522652 - samples/sec: 67.23
2020-02-03 08:52:13,554 epoch 4 - iter 8/11 - loss 6.36817271 - samples/sec: 56.44
2020-02-03 08:52:14,120 epoch 4 - iter 9/11 - loss 6.43955263 - samples/sec: 63.35
2020-02-03 08:52:14,624 epoch 4 - iter 10/11 - loss 6.51294065 - samples/sec: 71.80
2020-02-03 08:52:15,047 epoch 4 - iter 11/11 - loss 6.27435368 - samples/sec: 85.83
2020-02-03 08:52:15,097 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:15,105 EPOCH 4 done: loss 6.2744 - lr 0.1000
2020-02-03 08:52:15,620 DEV : loss 6.809615135192871 - score 0.0
2020-02-03 08:52:15,622 BAD EPOCHS (no improvement): 3
2020-02-03 08:52:19,094 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:19,590 epoch 5 - iter 1/11 - loss 9.48925972 - samples/sec: 64.80
2020-02-03 08:52:20,101 epoch 5 - iter 2/11 - loss 6.84309292 - samples/sec: 70.92
2020-02-03 08:52:20,568 epoch 5 - iter 3/11 - loss 5.89818891 - samples/sec: 76.93
2020-02-03 08:52:21,172 epoch 5 - iter 4/11 - loss 6.30742204 - samples/sec: 57.76
2020-02-03 08:52:21,666 epoch 5 - iter 5/11 - loss 6.14482307 - samples/sec: 72.08
2020-02-03 08:52:22,191 epoch 5 - iter 6/11 - loss 6.12260834 - samples/sec: 67.54
2020-02-03 08:52:22,705 epoch 5 - iter 7/11 - loss 6.17421722 - samples/sec: 69.05
2020-02-03 08:52:23,215 epoch 5 - iter 8/11 - loss 6.24382114 - samples/sec: 71.09
2020-02-03 08:52:23,821 epoch 5 - iter 9/11 - loss 6.27144310 - samples/sec: 57.76
2020-02-03 08:52:24,335 epoch 5 - iter 10/11 - loss 6.19822526 - samples/sec: 69.00
2020-02-03 08:52:24,708 epoch 5 - iter 11/11 - loss 6.25345958 - samples/sec: 99.31
2020-02-03 08:52:24,758 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:24,758 EPOCH 5 done: loss 6.2535 - lr 0.1000
2020-02-03 08:52:25,231 DEV : loss 6.468904495239258 - score 0.0
Epoch     5: reducing learning rate of group 0 to 5.0000e-02.
2020-02-03 08:52:25,241 BAD EPOCHS (no improvement): 4
2020-02-03 08:52:28,626 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:29,044 epoch 6 - iter 1/11 - loss 6.45317459 - samples/sec: 76.45
2020-02-03 08:52:29,616 epoch 6 - iter 2/11 - loss 6.89186525 - samples/sec: 61.35
2020-02-03 08:52:30,171 epoch 6 - iter 3/11 - loss 6.45178795 - samples/sec: 64.70
2020-02-03 08:52:30,797 epoch 6 - iter 4/11 - loss 7.05859566 - samples/sec: 55.81
2020-02-03 08:52:31,301 epoch 6 - iter 5/11 - loss 6.84184303 - samples/sec: 70.49
2020-02-03 08:52:31,868 epoch 6 - iter 6/11 - loss 6.57329567 - samples/sec: 62.16
2020-02-03 08:52:32,402 epoch 6 - iter 7/11 - loss 6.12817686 - samples/sec: 69.10
2020-02-03 08:52:32,896 epoch 6 - iter 8/11 - loss 6.38105181 - samples/sec: 72.25
2020-02-03 08:52:33,367 epoch 6 - iter 9/11 - loss 5.99511070 - samples/sec: 76.02
2020-02-03 08:52:33,931 epoch 6 - iter 10/11 - loss 5.92111208 - samples/sec: 62.31
2020-02-03 08:52:34,314 epoch 6 - iter 11/11 - loss 6.03110576 - samples/sec: 96.19
2020-02-03 08:52:34,372 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:34,374 EPOCH 6 done: loss 6.0311 - lr 0.0500
2020-02-03 08:52:34,818 DEV : loss 6.3388872146606445 - score 0.0
2020-02-03 08:52:34,818 BAD EPOCHS (no improvement): 1
2020-02-03 08:52:38,469 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:38,977 epoch 7 - iter 1/11 - loss 5.51349020 - samples/sec: 65.00
2020-02-03 08:52:39,431 epoch 7 - iter 2/11 - loss 5.28359199 - samples/sec: 79.45
2020-02-03 08:52:39,883 epoch 7 - iter 3/11 - loss 5.15292946 - samples/sec: 79.50
2020-02-03 08:52:40,376 epoch 7 - iter 4/11 - loss 5.68514609 - samples/sec: 73.64
2020-02-03 08:52:40,908 epoch 7 - iter 5/11 - loss 5.41621857 - samples/sec: 67.55
2020-02-03 08:52:41,432 epoch 7 - iter 6/11 - loss 5.20418525 - samples/sec: 67.88
2020-02-03 08:52:42,019 epoch 7 - iter 7/11 - loss 5.51338945 - samples/sec: 58.80
2020-02-03 08:52:42,543 epoch 7 - iter 8/11 - loss 5.36086476 - samples/sec: 68.72
2020-02-03 08:52:43,053 epoch 7 - iter 9/11 - loss 5.24261994 - samples/sec: 69.74
2020-02-03 08:52:43,596 epoch 7 - iter 10/11 - loss 5.51374149 - samples/sec: 65.14
2020-02-03 08:52:43,991 epoch 7 - iter 11/11 - loss 5.82901140 - samples/sec: 93.51
2020-02-03 08:52:44,041 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:44,049 EPOCH 7 done: loss 5.8290 - lr 0.0500
2020-02-03 08:52:44,562 DEV : loss 6.178075790405273 - score 0.0
2020-02-03 08:52:44,564 BAD EPOCHS (no improvement): 2
2020-02-03 08:52:48,272 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:48,734 epoch 8 - iter 1/11 - loss 4.37068939 - samples/sec: 69.40
2020-02-03 08:52:49,248 epoch 8 - iter 2/11 - loss 5.95447063 - samples/sec: 70.35
2020-02-03 08:52:49,793 epoch 8 - iter 3/11 - loss 5.80714639 - samples/sec: 64.66
2020-02-03 08:52:50,317 epoch 8 - iter 4/11 - loss 5.92247880 - samples/sec: 68.73
2020-02-03 08:52:50,820 epoch 8 - iter 5/11 - loss 5.96706476 - samples/sec: 70.65
2020-02-03 08:52:51,323 epoch 8 - iter 6/11 - loss 6.02020502 - samples/sec: 70.66
2020-02-03 08:52:51,867 epoch 8 - iter 7/11 - loss 5.91198594 - samples/sec: 65.87
2020-02-03 08:52:52,442 epoch 8 - iter 8/11 - loss 5.96371043 - samples/sec: 62.22
2020-02-03 08:52:52,966 epoch 8 - iter 9/11 - loss 5.60586299 - samples/sec: 69.10
2020-02-03 08:52:53,476 epoch 8 - iter 10/11 - loss 5.84172730 - samples/sec: 69.48
2020-02-03 08:52:53,860 epoch 8 - iter 11/11 - loss 5.59901782 - samples/sec: 99.21
2020-02-03 08:52:53,910 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:53,918 EPOCH 8 done: loss 5.5990 - lr 0.0500
2020-02-03 08:52:54,373 DEV : loss 6.267953395843506 - score 0.0
2020-02-03 08:52:54,383 BAD EPOCHS (no improvement): 3
2020-02-03 08:52:57,964 ----------------------------------------------------------------------------------------------------
2020-02-03 08:52:58,377 epoch 9 - iter 1/11 - loss 6.02514744 - samples/sec: 79.04
2020-02-03 08:52:58,927 epoch 9 - iter 2/11 - loss 7.51161814 - samples/sec: 63.76
2020-02-03 08:52:59,423 epoch 9 - iter 3/11 - loss 6.86398633 - samples/sec: 72.17
2020-02-03 08:52:59,998 epoch 9 - iter 4/11 - loss 5.95767391 - samples/sec: 60.99
2020-02-03 08:53:00,510 epoch 9 - iter 5/11 - loss 6.39319487 - samples/sec: 69.37
2020-02-03 08:53:01,007 epoch 9 - iter 6/11 - loss 6.07235495 - samples/sec: 72.01
2020-02-03 08:53:01,478 epoch 9 - iter 7/11 - loss 5.60639916 - samples/sec: 77.57
2020-02-03 08:53:02,054 epoch 9 - iter 8/11 - loss 5.34703508 - samples/sec: 61.05
2020-02-03 08:53:02,538 epoch 9 - iter 9/11 - loss 5.55232898 - samples/sec: 73.81
2020-02-03 08:53:03,086 epoch 9 - iter 10/11 - loss 5.69044588 - samples/sec: 64.02
2020-02-03 08:53:03,442 epoch 9 - iter 11/11 - loss 5.60601163 - samples/sec: 106.68
2020-02-03 08:53:03,492 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:03,492 EPOCH 9 done: loss 5.6060 - lr 0.0500
2020-02-03 08:53:03,895 DEV : loss 6.270799160003662 - score 0.0
Epoch     9: reducing learning rate of group 0 to 2.5000e-02.
2020-02-03 08:53:03,903 BAD EPOCHS (no improvement): 4
2020-02-03 08:53:07,595 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:08,068 epoch 10 - iter 1/11 - loss 6.16870689 - samples/sec: 67.54
2020-02-03 08:53:08,600 epoch 10 - iter 2/11 - loss 5.60666275 - samples/sec: 67.08
2020-02-03 08:53:09,084 epoch 10 - iter 3/11 - loss 6.00124963 - samples/sec: 75.22
2020-02-03 08:53:09,628 epoch 10 - iter 4/11 - loss 5.84133685 - samples/sec: 65.90
2020-02-03 08:53:10,162 epoch 10 - iter 5/11 - loss 6.25164566 - samples/sec: 66.20
2020-02-03 08:53:10,645 epoch 10 - iter 6/11 - loss 6.25318631 - samples/sec: 75.34
2020-02-03 08:53:11,179 epoch 10 - iter 7/11 - loss 6.23052658 - samples/sec: 67.29
2020-02-03 08:53:11,783 epoch 10 - iter 8/11 - loss 6.04517120 - samples/sec: 58.63
2020-02-03 08:53:12,297 epoch 10 - iter 9/11 - loss 5.73714267 - samples/sec: 70.58
2020-02-03 08:53:12,780 epoch 10 - iter 10/11 - loss 5.62469511 - samples/sec: 75.28
2020-02-03 08:53:13,153 epoch 10 - iter 11/11 - loss 5.48566645 - samples/sec: 102.38
2020-02-03 08:53:13,210 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:13,211 EPOCH 10 done: loss 5.4857 - lr 0.0250
2020-02-03 08:53:13,633 DEV : loss 6.120639801025391 - score 0.022
2020-02-03 08:53:13,641 BAD EPOCHS (no improvement): 0
2020-02-03 08:53:17,254 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:17,657 epoch 11 - iter 1/11 - loss 6.15016270 - samples/sec: 79.42
2020-02-03 08:53:18,231 epoch 11 - iter 2/11 - loss 5.34794950 - samples/sec: 61.11
2020-02-03 08:53:18,780 epoch 11 - iter 3/11 - loss 5.05054188 - samples/sec: 64.22
2020-02-03 08:53:19,334 epoch 11 - iter 4/11 - loss 5.44121337 - samples/sec: 64.79
2020-02-03 08:53:19,858 epoch 11 - iter 5/11 - loss 5.57905445 - samples/sec: 67.93
2020-02-03 08:53:20,404 epoch 11 - iter 6/11 - loss 5.22748224 - samples/sec: 64.82
2020-02-03 08:53:20,956 epoch 11 - iter 7/11 - loss 5.20546116 - samples/sec: 65.14
2020-02-03 08:53:21,491 epoch 11 - iter 8/11 - loss 5.13080138 - samples/sec: 66.22
2020-02-03 08:53:22,015 epoch 11 - iter 9/11 - loss 5.14500512 - samples/sec: 69.07
2020-02-03 08:53:22,569 epoch 11 - iter 10/11 - loss 5.23125000 - samples/sec: 64.83
2020-02-03 08:53:22,962 epoch 11 - iter 11/11 - loss 5.67733981 - samples/sec: 95.69
2020-02-03 08:53:23,022 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:23,022 EPOCH 11 done: loss 5.6773 - lr 0.0250
2020-02-03 08:53:23,473 DEV : loss 5.976884841918945 - score 0.022
2020-02-03 08:53:23,483 BAD EPOCHS (no improvement): 1
2020-02-03 08:53:27,031 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:27,445 epoch 12 - iter 1/11 - loss 6.23016548 - samples/sec: 77.43
2020-02-03 08:53:28,020 epoch 12 - iter 2/11 - loss 6.08264279 - samples/sec: 62.19
2020-02-03 08:53:28,559 epoch 12 - iter 3/11 - loss 5.78351816 - samples/sec: 66.86
2020-02-03 08:53:29,165 epoch 12 - iter 4/11 - loss 6.31968439 - samples/sec: 57.76
2020-02-03 08:53:29,668 epoch 12 - iter 5/11 - loss 6.21189957 - samples/sec: 72.28
2020-02-03 08:53:30,253 epoch 12 - iter 6/11 - loss 6.27824593 - samples/sec: 61.12
2020-02-03 08:53:30,766 epoch 12 - iter 7/11 - loss 5.73613719 - samples/sec: 70.64
2020-02-03 08:53:31,330 epoch 12 - iter 8/11 - loss 5.64280975 - samples/sec: 63.56
2020-02-03 08:53:31,854 epoch 12 - iter 9/11 - loss 5.51896980 - samples/sec: 69.06
2020-02-03 08:53:32,488 epoch 12 - iter 10/11 - loss 5.49403973 - samples/sec: 55.79
2020-02-03 08:53:32,921 epoch 12 - iter 11/11 - loss 5.50901599 - samples/sec: 83.50
2020-02-03 08:53:32,982 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:32,982 EPOCH 12 done: loss 5.5090 - lr 0.0250
2020-02-03 08:53:33,462 DEV : loss 6.062685012817383 - score 0.022
2020-02-03 08:53:33,472 BAD EPOCHS (no improvement): 2
2020-02-03 08:53:37,005 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:37,468 epoch 13 - iter 1/11 - loss 6.70333958 - samples/sec: 68.97
2020-02-03 08:53:38,043 epoch 13 - iter 2/11 - loss 5.39774990 - samples/sec: 62.25
2020-02-03 08:53:38,633 epoch 13 - iter 3/11 - loss 6.51228460 - samples/sec: 60.37
2020-02-03 08:53:39,210 epoch 13 - iter 4/11 - loss 6.00198817 - samples/sec: 62.19
2020-02-03 08:53:39,814 epoch 13 - iter 5/11 - loss 6.10420647 - samples/sec: 58.87
2020-02-03 08:53:40,369 epoch 13 - iter 6/11 - loss 5.89874355 - samples/sec: 64.78
2020-02-03 08:53:40,953 epoch 13 - iter 7/11 - loss 5.55606876 - samples/sec: 61.10
2020-02-03 08:53:41,535 epoch 13 - iter 8/11 - loss 5.56633574 - samples/sec: 61.30
2020-02-03 08:53:42,162 epoch 13 - iter 9/11 - loss 5.34884342 - samples/sec: 56.52
2020-02-03 08:53:42,765 epoch 13 - iter 10/11 - loss 5.40605054 - samples/sec: 59.00
2020-02-03 08:53:43,188 epoch 13 - iter 11/11 - loss 5.37144817 - samples/sec: 88.08
2020-02-03 08:53:43,249 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:43,250 EPOCH 13 done: loss 5.3714 - lr 0.0250
2020-02-03 08:53:43,719 DEV : loss 6.083741188049316 - score 0.022
2020-02-03 08:53:43,721 BAD EPOCHS (no improvement): 3
2020-02-03 08:53:47,131 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:47,562 epoch 14 - iter 1/11 - loss 7.29788399 - samples/sec: 74.26
2020-02-03 08:53:48,118 epoch 14 - iter 2/11 - loss 7.48885298 - samples/sec: 63.55
2020-02-03 08:53:48,647 epoch 14 - iter 3/11 - loss 6.94577789 - samples/sec: 68.85
2020-02-03 08:53:49,252 epoch 14 - iter 4/11 - loss 6.61860895 - samples/sec: 57.90
2020-02-03 08:53:49,798 epoch 14 - iter 5/11 - loss 5.77425079 - samples/sec: 65.89
2020-02-03 08:53:50,443 epoch 14 - iter 6/11 - loss 5.54892874 - samples/sec: 54.76
2020-02-03 08:53:50,965 epoch 14 - iter 7/11 - loss 5.35292462 - samples/sec: 69.37
2020-02-03 08:53:51,490 epoch 14 - iter 8/11 - loss 5.46914625 - samples/sec: 67.64
2020-02-03 08:53:52,044 epoch 14 - iter 9/11 - loss 5.63940710 - samples/sec: 64.85
2020-02-03 08:53:52,547 epoch 14 - iter 10/11 - loss 5.60669398 - samples/sec: 72.21
2020-02-03 08:53:52,940 epoch 14 - iter 11/11 - loss 5.41720241 - samples/sec: 96.37
2020-02-03 08:53:52,998 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:53,000 EPOCH 14 done: loss 5.4172 - lr 0.0250
2020-02-03 08:53:53,468 DEV : loss 5.947115898132324 - score 0.022
Epoch    14: reducing learning rate of group 0 to 1.2500e-02.
2020-02-03 08:53:53,474 BAD EPOCHS (no improvement): 4
2020-02-03 08:53:56,891 ----------------------------------------------------------------------------------------------------
2020-02-03 08:53:57,342 epoch 15 - iter 1/11 - loss 3.54983950 - samples/sec: 70.88
2020-02-03 08:53:57,886 epoch 15 - iter 2/11 - loss 3.43985093 - samples/sec: 66.43
2020-02-03 08:53:58,461 epoch 15 - iter 3/11 - loss 4.17819047 - samples/sec: 62.55
2020-02-03 08:53:59,074 epoch 15 - iter 4/11 - loss 4.66914624 - samples/sec: 57.79
2020-02-03 08:53:59,679 epoch 15 - iter 5/11 - loss 5.29557948 - samples/sec: 58.83
2020-02-03 08:54:00,253 epoch 15 - iter 6/11 - loss 5.12174968 - samples/sec: 62.25
2020-02-03 08:54:00,838 epoch 15 - iter 7/11 - loss 4.98247729 - samples/sec: 61.04
2020-02-03 08:54:01,402 epoch 15 - iter 8/11 - loss 5.13258186 - samples/sec: 63.52
2020-02-03 08:54:02,027 epoch 15 - iter 9/11 - loss 5.14211350 - samples/sec: 56.51
2020-02-03 08:54:02,601 epoch 15 - iter 10/11 - loss 5.00760038 - samples/sec: 61.09
2020-02-03 08:54:03,022 epoch 15 - iter 11/11 - loss 5.57299343 - samples/sec: 88.75
2020-02-03 08:54:03,084 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:03,084 EPOCH 15 done: loss 5.5730 - lr 0.0125
2020-02-03 08:54:03,544 DEV : loss 5.911517143249512 - score 0.022
2020-02-03 08:54:03,552 BAD EPOCHS (no improvement): 1
2020-02-03 08:54:07,834 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:08,335 epoch 16 - iter 1/11 - loss 5.91561794 - samples/sec: 63.86
2020-02-03 08:54:08,980 epoch 16 - iter 2/11 - loss 5.31361628 - samples/sec: 55.14
2020-02-03 08:54:09,766 epoch 16 - iter 3/11 - loss 5.46685123 - samples/sec: 44.58
2020-02-03 08:54:10,431 epoch 16 - iter 4/11 - loss 5.77804708 - samples/sec: 52.93
2020-02-03 08:54:11,198 epoch 16 - iter 5/11 - loss 5.29452586 - samples/sec: 45.98
2020-02-03 08:54:12,136 epoch 16 - iter 6/11 - loss 5.63010228 - samples/sec: 38.10
2020-02-03 08:54:12,832 epoch 16 - iter 7/11 - loss 5.44208264 - samples/sec: 51.17
2020-02-03 08:54:13,748 epoch 16 - iter 8/11 - loss 5.35705689 - samples/sec: 37.43
2020-02-03 08:54:14,535 epoch 16 - iter 9/11 - loss 5.38117793 - samples/sec: 44.49
2020-02-03 08:54:15,594 epoch 16 - iter 10/11 - loss 5.39818857 - samples/sec: 32.32
2020-02-03 08:54:16,007 epoch 16 - iter 11/11 - loss 5.15628420 - samples/sec: 93.33
2020-02-03 08:54:16,078 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:16,078 EPOCH 16 done: loss 5.1563 - lr 0.0125
2020-02-03 08:54:16,544 DEV : loss 5.973292350769043 - score 0.022
2020-02-03 08:54:16,549 BAD EPOCHS (no improvement): 2
2020-02-03 08:54:20,385 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:20,806 epoch 17 - iter 1/11 - loss 7.08767509 - samples/sec: 76.00
2020-02-03 08:54:21,401 epoch 17 - iter 2/11 - loss 5.78037119 - samples/sec: 59.85
2020-02-03 08:54:21,937 epoch 17 - iter 3/11 - loss 6.14361334 - samples/sec: 67.57
2020-02-03 08:54:22,480 epoch 17 - iter 4/11 - loss 5.49493563 - samples/sec: 66.28
2020-02-03 08:54:23,003 epoch 17 - iter 5/11 - loss 5.34629297 - samples/sec: 69.14
2020-02-03 08:54:23,534 epoch 17 - iter 6/11 - loss 5.63093328 - samples/sec: 68.05
2020-02-03 08:54:24,117 epoch 17 - iter 7/11 - loss 5.45088155 - samples/sec: 61.16
2020-02-03 08:54:24,621 epoch 17 - iter 8/11 - loss 5.19171610 - samples/sec: 72.17
2020-02-03 08:54:25,256 epoch 17 - iter 9/11 - loss 5.54780761 - samples/sec: 55.75
2020-02-03 08:54:25,809 epoch 17 - iter 10/11 - loss 5.31668959 - samples/sec: 64.58
2020-02-03 08:54:26,232 epoch 17 - iter 11/11 - loss 5.28244647 - samples/sec: 88.28
2020-02-03 08:54:26,293 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:26,293 EPOCH 17 done: loss 5.2824 - lr 0.0125
2020-02-03 08:54:26,754 DEV : loss 6.028558254241943 - score 0.022
2020-02-03 08:54:26,756 BAD EPOCHS (no improvement): 3
2020-02-03 08:54:30,291 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:30,693 epoch 18 - iter 1/11 - loss 3.43650746 - samples/sec: 79.48
2020-02-03 08:54:31,187 epoch 18 - iter 2/11 - loss 4.29816115 - samples/sec: 73.42
2020-02-03 08:54:31,731 epoch 18 - iter 3/11 - loss 4.98313848 - samples/sec: 66.18
2020-02-03 08:54:32,306 epoch 18 - iter 4/11 - loss 5.03285366 - samples/sec: 62.25
2020-02-03 08:54:32,850 epoch 18 - iter 5/11 - loss 4.83098359 - samples/sec: 66.20
2020-02-03 08:54:33,491 epoch 18 - iter 6/11 - loss 4.75197017 - samples/sec: 55.14
2020-02-03 08:54:34,087 epoch 18 - iter 7/11 - loss 4.79946072 - samples/sec: 60.27
2020-02-03 08:54:34,658 epoch 18 - iter 8/11 - loss 5.14870754 - samples/sec: 62.57
2020-02-03 08:54:35,204 epoch 18 - iter 9/11 - loss 5.28498218 - samples/sec: 65.93
2020-02-03 08:54:35,758 epoch 18 - iter 10/11 - loss 5.23333752 - samples/sec: 64.84
2020-02-03 08:54:36,151 epoch 18 - iter 11/11 - loss 5.40952394 - samples/sec: 96.32
2020-02-03 08:54:36,211 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:36,211 EPOCH 18 done: loss 5.4095 - lr 0.0125
2020-02-03 08:54:36,643 DEV : loss 5.888984680175781 - score 0.022
Epoch    18: reducing learning rate of group 0 to 6.2500e-03.
2020-02-03 08:54:36,653 BAD EPOCHS (no improvement): 4
2020-02-03 08:54:40,189 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:40,660 epoch 19 - iter 1/11 - loss 3.88627005 - samples/sec: 67.91
2020-02-03 08:54:41,206 epoch 19 - iter 2/11 - loss 3.94635010 - samples/sec: 65.94
2020-02-03 08:54:41,719 epoch 19 - iter 3/11 - loss 4.72114690 - samples/sec: 70.67
2020-02-03 08:54:42,273 epoch 19 - iter 4/11 - loss 4.75362277 - samples/sec: 64.82
2020-02-03 08:54:42,756 epoch 19 - iter 5/11 - loss 4.67478886 - samples/sec: 75.34
2020-02-03 08:54:43,290 epoch 19 - iter 6/11 - loss 4.56568972 - samples/sec: 67.64
2020-02-03 08:54:43,871 epoch 19 - iter 7/11 - loss 4.59808275 - samples/sec: 62.23
2020-02-03 08:54:44,413 epoch 19 - iter 8/11 - loss 4.96171397 - samples/sec: 66.52
2020-02-03 08:54:44,968 epoch 19 - iter 9/11 - loss 5.19528808 - samples/sec: 64.87
2020-02-03 08:54:45,603 epoch 19 - iter 10/11 - loss 5.27724352 - samples/sec: 55.74
2020-02-03 08:54:46,035 epoch 19 - iter 11/11 - loss 5.42269698 - samples/sec: 86.15
2020-02-03 08:54:46,097 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:46,097 EPOCH 19 done: loss 5.4227 - lr 0.0063
2020-02-03 08:54:46,580 DEV : loss 5.88798713684082 - score 0.022
2020-02-03 08:54:46,590 BAD EPOCHS (no improvement): 1
2020-02-03 08:54:50,215 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:50,600 epoch 20 - iter 1/11 - loss 5.97278214 - samples/sec: 83.10
2020-02-03 08:54:51,103 epoch 20 - iter 2/11 - loss 6.05469990 - samples/sec: 72.26
2020-02-03 08:54:51,695 epoch 20 - iter 3/11 - loss 6.27930371 - samples/sec: 60.25
2020-02-03 08:54:52,220 epoch 20 - iter 4/11 - loss 5.75106561 - samples/sec: 69.13
2020-02-03 08:54:52,804 epoch 20 - iter 5/11 - loss 5.65186529 - samples/sec: 62.30
2020-02-03 08:54:53,341 epoch 20 - iter 6/11 - loss 5.48180874 - samples/sec: 67.22
2020-02-03 08:54:53,947 epoch 20 - iter 7/11 - loss 5.35837466 - samples/sec: 58.32
2020-02-03 08:54:54,562 epoch 20 - iter 8/11 - loss 5.68852252 - samples/sec: 57.93
2020-02-03 08:54:55,198 epoch 20 - iter 9/11 - loss 5.67969380 - samples/sec: 55.82
2020-02-03 08:54:55,754 epoch 20 - iter 10/11 - loss 5.43555019 - samples/sec: 64.79
2020-02-03 08:54:56,167 epoch 20 - iter 11/11 - loss 5.13037621 - samples/sec: 90.75
2020-02-03 08:54:56,228 ----------------------------------------------------------------------------------------------------
2020-02-03 08:54:56,236 EPOCH 20 done: loss 5.1304 - lr 0.0063
2020-02-03 08:54:56,711 DEV : loss 5.933487892150879 - score 0.022
2020-02-03 08:54:56,721 BAD EPOCHS (no improvement): 2
2020-02-03 08:55:00,176 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:00,529 epoch 21 - iter 1/11 - loss 8.66816902 - samples/sec: 90.74
2020-02-03 08:55:01,073 epoch 21 - iter 2/11 - loss 6.35994816 - samples/sec: 64.77
2020-02-03 08:55:01,597 epoch 21 - iter 3/11 - loss 5.39528982 - samples/sec: 69.11
2020-02-03 08:55:02,211 epoch 21 - iter 4/11 - loss 5.36364514 - samples/sec: 57.82
2020-02-03 08:55:02,742 epoch 21 - iter 5/11 - loss 5.64832320 - samples/sec: 67.94
2020-02-03 08:55:03,288 epoch 21 - iter 6/11 - loss 5.37178671 - samples/sec: 65.92
2020-02-03 08:55:03,860 epoch 21 - iter 7/11 - loss 5.30107856 - samples/sec: 63.02
2020-02-03 08:55:04,431 epoch 21 - iter 8/11 - loss 5.32317480 - samples/sec: 62.60
2020-02-03 08:55:05,058 epoch 21 - iter 9/11 - loss 5.55084724 - samples/sec: 55.68
2020-02-03 08:55:05,602 epoch 21 - iter 10/11 - loss 5.30182970 - samples/sec: 67.30
2020-02-03 08:55:06,008 epoch 21 - iter 11/11 - loss 5.23851497 - samples/sec: 92.52
2020-02-03 08:55:06,065 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:06,065 EPOCH 21 done: loss 5.2385 - lr 0.0063
2020-02-03 08:55:06,589 DEV : loss 5.925188064575195 - score 0.022
2020-02-03 08:55:06,599 BAD EPOCHS (no improvement): 3
2020-02-03 08:55:10,129 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:10,525 epoch 22 - iter 1/11 - loss 6.70335102 - samples/sec: 80.81
2020-02-03 08:55:11,149 epoch 22 - iter 2/11 - loss 6.14632082 - samples/sec: 56.78
2020-02-03 08:55:11,653 epoch 22 - iter 3/11 - loss 5.83356539 - samples/sec: 70.58
2020-02-03 08:55:12,204 epoch 22 - iter 4/11 - loss 5.23634189 - samples/sec: 66.25
2020-02-03 08:55:12,758 epoch 22 - iter 5/11 - loss 5.06797929 - samples/sec: 63.84
2020-02-03 08:55:13,361 epoch 22 - iter 6/11 - loss 5.23916161 - samples/sec: 59.27
2020-02-03 08:55:13,934 epoch 22 - iter 7/11 - loss 5.56800205 - samples/sec: 62.25
2020-02-03 08:55:14,531 epoch 22 - iter 8/11 - loss 5.61531976 - samples/sec: 60.84
2020-02-03 08:55:15,055 epoch 22 - iter 9/11 - loss 5.30011924 - samples/sec: 69.06
2020-02-03 08:55:15,619 epoch 22 - iter 10/11 - loss 5.19746385 - samples/sec: 63.50
2020-02-03 08:55:16,082 epoch 22 - iter 11/11 - loss 5.31781942 - samples/sec: 79.42
2020-02-03 08:55:16,143 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:16,151 EPOCH 22 done: loss 5.3178 - lr 0.0063
2020-02-03 08:55:16,644 DEV : loss 5.893540382385254 - score 0.022
Epoch    22: reducing learning rate of group 0 to 3.1250e-03.
2020-02-03 08:55:16,646 BAD EPOCHS (no improvement): 4
2020-02-03 08:55:20,272 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:20,705 epoch 23 - iter 1/11 - loss 2.44687843 - samples/sec: 73.86
2020-02-03 08:55:21,268 epoch 23 - iter 2/11 - loss 3.29671192 - samples/sec: 63.76
2020-02-03 08:55:21,813 epoch 23 - iter 3/11 - loss 4.50855859 - samples/sec: 66.25
2020-02-03 08:55:22,447 epoch 23 - iter 4/11 - loss 4.43704712 - samples/sec: 56.59
2020-02-03 08:55:23,021 epoch 23 - iter 5/11 - loss 4.67560625 - samples/sec: 61.15
2020-02-03 08:55:23,592 epoch 23 - iter 6/11 - loss 4.81096721 - samples/sec: 62.68
2020-02-03 08:55:24,125 epoch 23 - iter 7/11 - loss 4.98460940 - samples/sec: 68.27
2020-02-03 08:55:24,710 epoch 23 - iter 8/11 - loss 5.08164245 - samples/sec: 61.99
2020-02-03 08:55:25,224 epoch 23 - iter 9/11 - loss 4.97728830 - samples/sec: 70.57
2020-02-03 08:55:25,767 epoch 23 - iter 10/11 - loss 5.25138955 - samples/sec: 66.20
2020-02-03 08:55:26,210 epoch 23 - iter 11/11 - loss 5.33368171 - samples/sec: 85.47
2020-02-03 08:55:26,270 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:26,270 EPOCH 23 done: loss 5.3337 - lr 0.0031
2020-02-03 08:55:26,733 DEV : loss 5.914398670196533 - score 0.022
2020-02-03 08:55:26,743 BAD EPOCHS (no improvement): 1
2020-02-03 08:55:30,308 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:30,732 epoch 24 - iter 1/11 - loss 3.21351695 - samples/sec: 75.60
2020-02-03 08:55:31,296 epoch 24 - iter 2/11 - loss 3.56172764 - samples/sec: 62.01
2020-02-03 08:55:31,911 epoch 24 - iter 3/11 - loss 4.16734831 - samples/sec: 57.66
2020-02-03 08:55:32,547 epoch 24 - iter 4/11 - loss 5.46593624 - samples/sec: 56.42
2020-02-03 08:55:33,214 epoch 24 - iter 5/11 - loss 5.46387181 - samples/sec: 53.69
2020-02-03 08:55:33,765 epoch 24 - iter 6/11 - loss 5.85624397 - samples/sec: 67.64
2020-02-03 08:55:34,380 epoch 24 - iter 7/11 - loss 5.77485422 - samples/sec: 57.74
2020-02-03 08:55:34,935 epoch 24 - iter 8/11 - loss 5.67491111 - samples/sec: 64.75
2020-02-03 08:55:35,489 epoch 24 - iter 9/11 - loss 5.63281894 - samples/sec: 65.90
2020-02-03 08:55:36,173 epoch 24 - iter 10/11 - loss 5.39995458 - samples/sec: 52.17
2020-02-03 08:55:36,538 epoch 24 - iter 11/11 - loss 5.25282949 - samples/sec: 105.04
2020-02-03 08:55:36,608 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:36,608 EPOCH 24 done: loss 5.2528 - lr 0.0031
2020-02-03 08:55:37,092 DEV : loss 5.902481555938721 - score 0.022
2020-02-03 08:55:37,102 BAD EPOCHS (no improvement): 2
2020-02-03 08:55:40,665 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:41,201 epoch 25 - iter 1/11 - loss 2.21456623 - samples/sec: 59.88
2020-02-03 08:55:41,745 epoch 25 - iter 2/11 - loss 4.73606730 - samples/sec: 66.21
2020-02-03 08:55:42,319 epoch 25 - iter 3/11 - loss 5.33834791 - samples/sec: 62.33
2020-02-03 08:55:42,893 epoch 25 - iter 4/11 - loss 5.33970213 - samples/sec: 62.32
2020-02-03 08:55:43,480 epoch 25 - iter 5/11 - loss 5.04884048 - samples/sec: 60.73
2020-02-03 08:55:44,008 epoch 25 - iter 6/11 - loss 4.85471201 - samples/sec: 69.37
2020-02-03 08:55:44,509 epoch 25 - iter 7/11 - loss 5.14230796 - samples/sec: 72.61
2020-02-03 08:55:45,056 epoch 25 - iter 8/11 - loss 4.80295989 - samples/sec: 64.77
2020-02-03 08:55:45,701 epoch 25 - iter 9/11 - loss 5.15898236 - samples/sec: 54.74
2020-02-03 08:55:46,275 epoch 25 - iter 10/11 - loss 5.10619662 - samples/sec: 63.49
2020-02-03 08:55:46,718 epoch 25 - iter 11/11 - loss 5.35262327 - samples/sec: 83.63
2020-02-03 08:55:46,779 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:46,779 EPOCH 25 done: loss 5.3526 - lr 0.0031
2020-02-03 08:55:47,311 DEV : loss 5.887154579162598 - score 0.022
2020-02-03 08:55:47,321 BAD EPOCHS (no improvement): 3
2020-02-03 08:55:50,716 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:51,177 epoch 26 - iter 1/11 - loss 3.29046369 - samples/sec: 69.37
2020-02-03 08:55:51,713 epoch 26 - iter 2/11 - loss 3.32256746 - samples/sec: 66.20
2020-02-03 08:55:52,286 epoch 26 - iter 3/11 - loss 4.30396016 - samples/sec: 61.28
2020-02-03 08:55:52,802 epoch 26 - iter 4/11 - loss 3.92066073 - samples/sec: 70.53
2020-02-03 08:55:53,326 epoch 26 - iter 5/11 - loss 4.58878336 - samples/sec: 69.01
2020-02-03 08:55:53,877 epoch 26 - iter 6/11 - loss 4.97014976 - samples/sec: 65.21
2020-02-03 08:55:54,421 epoch 26 - iter 7/11 - loss 5.26605722 - samples/sec: 67.67
2020-02-03 08:55:54,955 epoch 26 - iter 8/11 - loss 5.20368594 - samples/sec: 67.57
2020-02-03 08:55:55,609 epoch 26 - iter 9/11 - loss 5.33130773 - samples/sec: 53.85
2020-02-03 08:55:56,244 epoch 26 - iter 10/11 - loss 5.22921419 - samples/sec: 55.74
2020-02-03 08:55:56,679 epoch 26 - iter 11/11 - loss 5.30758329 - samples/sec: 88.26
2020-02-03 08:55:56,739 ----------------------------------------------------------------------------------------------------
2020-02-03 08:55:56,739 EPOCH 26 done: loss 5.3076 - lr 0.0031
2020-02-03 08:55:57,261 DEV : loss 5.901633262634277 - score 0.022
Epoch    26: reducing learning rate of group 0 to 1.5625e-03.
2020-02-03 08:55:57,271 BAD EPOCHS (no improvement): 4
2020-02-03 08:56:01,061 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:01,472 epoch 27 - iter 1/11 - loss 5.70121336 - samples/sec: 77.87
2020-02-03 08:56:02,018 epoch 27 - iter 2/11 - loss 5.59804487 - samples/sec: 65.93
2020-02-03 08:56:02,562 epoch 27 - iter 3/11 - loss 4.65278721 - samples/sec: 67.62
2020-02-03 08:56:03,032 epoch 27 - iter 4/11 - loss 5.20230412 - samples/sec: 77.96
2020-02-03 08:56:03,643 epoch 27 - iter 5/11 - loss 5.43994265 - samples/sec: 58.37
2020-02-03 08:56:04,179 epoch 27 - iter 6/11 - loss 5.63724208 - samples/sec: 67.56
2020-02-03 08:56:04,682 epoch 27 - iter 7/11 - loss 5.18828089 - samples/sec: 72.31
2020-02-03 08:56:05,246 epoch 27 - iter 8/11 - loss 5.18370619 - samples/sec: 63.57
2020-02-03 08:56:05,797 epoch 27 - iter 9/11 - loss 5.28764282 - samples/sec: 64.91
2020-02-03 08:56:06,333 epoch 27 - iter 10/11 - loss 5.19689391 - samples/sec: 67.63
2020-02-03 08:56:06,816 epoch 27 - iter 11/11 - loss 5.20945967 - samples/sec: 75.66
2020-02-03 08:56:06,877 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:06,877 EPOCH 27 done: loss 5.2095 - lr 0.0016
2020-02-03 08:56:07,348 DEV : loss 5.903687000274658 - score 0.022
2020-02-03 08:56:07,358 BAD EPOCHS (no improvement): 1
2020-02-03 08:56:10,682 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:11,214 epoch 28 - iter 1/11 - loss 6.64661407 - samples/sec: 60.10
2020-02-03 08:56:11,740 epoch 28 - iter 2/11 - loss 5.76468563 - samples/sec: 67.63
2020-02-03 08:56:12,271 epoch 28 - iter 3/11 - loss 5.29088704 - samples/sec: 67.89
2020-02-03 08:56:12,827 epoch 28 - iter 4/11 - loss 5.22902787 - samples/sec: 63.52
2020-02-03 08:56:13,361 epoch 28 - iter 5/11 - loss 4.83799639 - samples/sec: 67.67
2020-02-03 08:56:13,933 epoch 28 - iter 6/11 - loss 5.40926957 - samples/sec: 63.83
2020-02-03 08:56:14,557 epoch 28 - iter 7/11 - loss 5.73988649 - samples/sec: 57.54
2020-02-03 08:56:15,131 epoch 28 - iter 8/11 - loss 5.55149192 - samples/sec: 62.29
2020-02-03 08:56:15,706 epoch 28 - iter 9/11 - loss 5.20358014 - samples/sec: 62.25
2020-02-03 08:56:16,321 epoch 28 - iter 10/11 - loss 5.24708793 - samples/sec: 57.73
2020-02-03 08:56:16,711 epoch 28 - iter 11/11 - loss 5.22185779 - samples/sec: 96.84
2020-02-03 08:56:16,774 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:16,774 EPOCH 28 done: loss 5.2219 - lr 0.0016
2020-02-03 08:56:17,258 DEV : loss 5.895233154296875 - score 0.022
2020-02-03 08:56:17,268 BAD EPOCHS (no improvement): 2
2020-02-03 08:56:20,779 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:21,293 epoch 29 - iter 1/11 - loss 6.11373234 - samples/sec: 62.27
2020-02-03 08:56:21,844 epoch 29 - iter 2/11 - loss 4.90121126 - samples/sec: 66.23
2020-02-03 08:56:22,381 epoch 29 - iter 3/11 - loss 5.06075382 - samples/sec: 67.50
2020-02-03 08:56:23,116 epoch 29 - iter 4/11 - loss 5.04807246 - samples/sec: 48.10
2020-02-03 08:56:23,655 epoch 29 - iter 5/11 - loss 4.62523270 - samples/sec: 66.97
2020-02-03 08:56:24,221 epoch 29 - iter 6/11 - loss 4.79883862 - samples/sec: 62.70
2020-02-03 08:56:24,783 epoch 29 - iter 7/11 - loss 5.06419972 - samples/sec: 64.87
2020-02-03 08:56:25,389 epoch 29 - iter 8/11 - loss 5.44006538 - samples/sec: 58.65
2020-02-03 08:56:25,913 epoch 29 - iter 9/11 - loss 5.52780030 - samples/sec: 69.02
2020-02-03 08:56:26,447 epoch 29 - iter 10/11 - loss 5.36485059 - samples/sec: 68.74
2020-02-03 08:56:26,868 epoch 29 - iter 11/11 - loss 5.18434910 - samples/sec: 88.75
2020-02-03 08:56:26,920 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:26,928 EPOCH 29 done: loss 5.1843 - lr 0.0016
2020-02-03 08:56:27,404 DEV : loss 5.896324157714844 - score 0.022
2020-02-03 08:56:27,412 BAD EPOCHS (no improvement): 3
2020-02-03 08:56:30,931 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:31,385 epoch 30 - iter 1/11 - loss 3.89359999 - samples/sec: 71.80
2020-02-03 08:56:32,010 epoch 30 - iter 2/11 - loss 3.92497885 - samples/sec: 56.69
2020-02-03 08:56:32,574 epoch 30 - iter 3/11 - loss 4.26560346 - samples/sec: 63.55
2020-02-03 08:56:33,107 epoch 30 - iter 4/11 - loss 4.58838278 - samples/sec: 67.59
2020-02-03 08:56:33,670 epoch 30 - iter 5/11 - loss 4.64010777 - samples/sec: 63.41
2020-02-03 08:56:34,221 epoch 30 - iter 6/11 - loss 4.79377313 - samples/sec: 65.34
2020-02-03 08:56:34,777 epoch 30 - iter 7/11 - loss 4.91277623 - samples/sec: 64.62
2020-02-03 08:56:35,290 epoch 30 - iter 8/11 - loss 4.97782412 - samples/sec: 70.65
2020-02-03 08:56:35,832 epoch 30 - iter 9/11 - loss 5.16275313 - samples/sec: 66.43
2020-02-03 08:56:36,448 epoch 30 - iter 10/11 - loss 5.33106954 - samples/sec: 57.57
2020-02-03 08:56:36,811 epoch 30 - iter 11/11 - loss 5.06421267 - samples/sec: 105.95
2020-02-03 08:56:36,871 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:36,871 EPOCH 30 done: loss 5.0642 - lr 0.0016
2020-02-03 08:56:37,366 DEV : loss 5.892666816711426 - score 0.022
Epoch    30: reducing learning rate of group 0 to 7.8125e-04.
2020-02-03 08:56:37,376 BAD EPOCHS (no improvement): 4
2020-02-03 08:56:41,044 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:41,538 epoch 31 - iter 1/11 - loss 4.58575249 - samples/sec: 65.04
2020-02-03 08:56:42,062 epoch 31 - iter 2/11 - loss 5.01195002 - samples/sec: 68.66
2020-02-03 08:56:42,737 epoch 31 - iter 3/11 - loss 5.75716035 - samples/sec: 52.06
2020-02-03 08:56:43,279 epoch 31 - iter 4/11 - loss 5.42470753 - samples/sec: 66.49
2020-02-03 08:56:43,835 epoch 31 - iter 5/11 - loss 5.27966433 - samples/sec: 64.81
2020-02-03 08:56:44,366 epoch 31 - iter 6/11 - loss 5.61391409 - samples/sec: 67.63
2020-02-03 08:56:44,988 epoch 31 - iter 7/11 - loss 5.71890088 - samples/sec: 56.92
2020-02-03 08:56:45,504 epoch 31 - iter 8/11 - loss 5.58644885 - samples/sec: 69.06
2020-02-03 08:56:46,068 epoch 31 - iter 9/11 - loss 5.36483786 - samples/sec: 63.57
2020-02-03 08:56:46,762 epoch 31 - iter 10/11 - loss 5.22437634 - samples/sec: 50.52
2020-02-03 08:56:47,198 epoch 31 - iter 11/11 - loss 5.31842145 - samples/sec: 87.61
2020-02-03 08:56:47,258 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:47,266 EPOCH 31 done: loss 5.3184 - lr 0.0008
2020-02-03 08:56:47,772 DEV : loss 5.886855602264404 - score 0.022
2020-02-03 08:56:47,782 BAD EPOCHS (no improvement): 1
2020-02-03 08:56:51,649 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:52,081 epoch 32 - iter 1/11 - loss 3.52422738 - samples/sec: 73.99
2020-02-03 08:56:52,605 epoch 32 - iter 2/11 - loss 3.99462521 - samples/sec: 67.57
2020-02-03 08:56:53,189 epoch 32 - iter 3/11 - loss 5.19067550 - samples/sec: 61.09
2020-02-03 08:56:53,779 epoch 32 - iter 4/11 - loss 4.97302812 - samples/sec: 60.45
2020-02-03 08:56:54,285 epoch 32 - iter 5/11 - loss 4.82146564 - samples/sec: 71.59
2020-02-03 08:56:54,819 epoch 32 - iter 6/11 - loss 4.63359841 - samples/sec: 67.59
2020-02-03 08:56:55,401 epoch 32 - iter 7/11 - loss 4.70820332 - samples/sec: 61.30
2020-02-03 08:56:55,975 epoch 32 - iter 8/11 - loss 4.80831957 - samples/sec: 62.34
2020-02-03 08:56:56,601 epoch 32 - iter 9/11 - loss 4.64154948 - samples/sec: 56.76
2020-02-03 08:56:57,216 epoch 32 - iter 10/11 - loss 5.14540305 - samples/sec: 57.72
2020-02-03 08:56:57,699 epoch 32 - iter 11/11 - loss 5.26009820 - samples/sec: 77.51
2020-02-03 08:56:57,760 ----------------------------------------------------------------------------------------------------
2020-02-03 08:56:57,760 EPOCH 32 done: loss 5.2601 - lr 0.0008
2020-02-03 08:56:58,273 DEV : loss 5.882058620452881 - score 0.022
2020-02-03 08:56:58,283 BAD EPOCHS (no improvement): 2
2020-02-03 08:57:01,748 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:02,192 epoch 33 - iter 1/11 - loss 3.98337197 - samples/sec: 71.99
2020-02-03 08:57:02,787 epoch 33 - iter 2/11 - loss 4.11289513 - samples/sec: 59.91
2020-02-03 08:57:03,391 epoch 33 - iter 3/11 - loss 3.41462986 - samples/sec: 58.86
2020-02-03 08:57:03,972 epoch 33 - iter 4/11 - loss 4.33481520 - samples/sec: 62.38
2020-02-03 08:57:04,514 epoch 33 - iter 5/11 - loss 4.66553741 - samples/sec: 66.48
2020-02-03 08:57:05,038 epoch 33 - iter 6/11 - loss 5.17382554 - samples/sec: 67.79
2020-02-03 08:57:05,544 epoch 33 - iter 7/11 - loss 5.35703976 - samples/sec: 72.23
2020-02-03 08:57:06,108 epoch 33 - iter 8/11 - loss 5.34807023 - samples/sec: 64.56
2020-02-03 08:57:06,702 epoch 33 - iter 9/11 - loss 5.53157963 - samples/sec: 59.93
2020-02-03 08:57:07,297 epoch 33 - iter 10/11 - loss 5.36929166 - samples/sec: 59.88
2020-02-03 08:57:07,770 epoch 33 - iter 11/11 - loss 5.22659668 - samples/sec: 77.50
2020-02-03 08:57:07,831 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:07,831 EPOCH 33 done: loss 5.2266 - lr 0.0008
2020-02-03 08:57:08,342 DEV : loss 5.8817667961120605 - score 0.022
2020-02-03 08:57:08,344 BAD EPOCHS (no improvement): 3
2020-02-03 08:57:12,022 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:12,590 epoch 34 - iter 1/11 - loss 5.23984432 - samples/sec: 56.43
2020-02-03 08:57:13,186 epoch 34 - iter 2/11 - loss 5.85560083 - samples/sec: 58.80
2020-02-03 08:57:13,763 epoch 34 - iter 3/11 - loss 5.51792097 - samples/sec: 61.95
2020-02-03 08:57:14,371 epoch 34 - iter 4/11 - loss 5.55522108 - samples/sec: 58.19
2020-02-03 08:57:14,997 epoch 34 - iter 5/11 - loss 5.47720470 - samples/sec: 56.73
2020-02-03 08:57:15,531 epoch 34 - iter 6/11 - loss 5.60804884 - samples/sec: 67.58
2020-02-03 08:57:16,045 epoch 34 - iter 7/11 - loss 5.40140608 - samples/sec: 70.62
2020-02-03 08:57:16,600 epoch 34 - iter 8/11 - loss 5.22709388 - samples/sec: 65.83
2020-02-03 08:57:17,072 epoch 34 - iter 9/11 - loss 5.14414554 - samples/sec: 77.71
2020-02-03 08:57:17,676 epoch 34 - iter 10/11 - loss 5.20493813 - samples/sec: 58.82
2020-02-03 08:57:18,091 epoch 34 - iter 11/11 - loss 5.19641907 - samples/sec: 90.78
2020-02-03 08:57:18,152 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:18,152 EPOCH 34 done: loss 5.1964 - lr 0.0008
2020-02-03 08:57:18,660 DEV : loss 5.882002830505371 - score 0.022
Epoch    34: reducing learning rate of group 0 to 3.9063e-04.
2020-02-03 08:57:18,671 BAD EPOCHS (no improvement): 4
2020-02-03 08:57:22,196 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:22,610 epoch 35 - iter 1/11 - loss 3.58408976 - samples/sec: 77.34
2020-02-03 08:57:23,154 epoch 35 - iter 2/11 - loss 5.65412474 - samples/sec: 67.28
2020-02-03 08:57:23,810 epoch 35 - iter 3/11 - loss 6.28582811 - samples/sec: 53.69
2020-02-03 08:57:24,392 epoch 35 - iter 4/11 - loss 5.79221201 - samples/sec: 61.59
2020-02-03 08:57:24,975 epoch 35 - iter 5/11 - loss 5.52104855 - samples/sec: 62.45
2020-02-03 08:57:25,551 epoch 35 - iter 6/11 - loss 5.60898232 - samples/sec: 62.24
2020-02-03 08:57:26,105 epoch 35 - iter 7/11 - loss 5.50105143 - samples/sec: 64.86
2020-02-03 08:57:26,679 epoch 35 - iter 8/11 - loss 5.73515338 - samples/sec: 63.29
2020-02-03 08:57:27,334 epoch 35 - iter 9/11 - loss 5.57323922 - samples/sec: 53.83
2020-02-03 08:57:27,969 epoch 35 - iter 10/11 - loss 5.40601563 - samples/sec: 56.66
2020-02-03 08:57:28,421 epoch 35 - iter 11/11 - loss 5.18469550 - samples/sec: 83.98
2020-02-03 08:57:28,494 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:28,494 EPOCH 35 done: loss 5.1847 - lr 0.0004
2020-02-03 08:57:29,046 DEV : loss 5.883790016174316 - score 0.022
2020-02-03 08:57:29,054 BAD EPOCHS (no improvement): 1
2020-02-03 08:57:32,841 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:33,325 epoch 36 - iter 1/11 - loss 7.34654999 - samples/sec: 66.11
2020-02-03 08:57:33,987 epoch 36 - iter 2/11 - loss 6.89498997 - samples/sec: 54.08
2020-02-03 08:57:34,621 epoch 36 - iter 3/11 - loss 5.62297106 - samples/sec: 56.84
2020-02-03 08:57:35,175 epoch 36 - iter 4/11 - loss 5.73646176 - samples/sec: 65.15
2020-02-03 08:57:35,731 epoch 36 - iter 5/11 - loss 6.21581640 - samples/sec: 64.83
2020-02-03 08:57:36,295 epoch 36 - iter 6/11 - loss 5.67982535 - samples/sec: 63.54
2020-02-03 08:57:36,839 epoch 36 - iter 7/11 - loss 5.31354751 - samples/sec: 66.15
2020-02-03 08:57:37,372 epoch 36 - iter 8/11 - loss 5.25262913 - samples/sec: 67.60
2020-02-03 08:57:37,926 epoch 36 - iter 9/11 - loss 5.35170219 - samples/sec: 64.88
2020-02-03 08:57:38,550 epoch 36 - iter 10/11 - loss 5.43617222 - samples/sec: 55.78
2020-02-03 08:57:38,928 epoch 36 - iter 11/11 - loss 5.11979839 - samples/sec: 101.00
2020-02-03 08:57:38,991 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:38,991 EPOCH 36 done: loss 5.1198 - lr 0.0004
2020-02-03 08:57:39,434 DEV : loss 5.884423732757568 - score 0.022
2020-02-03 08:57:39,444 BAD EPOCHS (no improvement): 2
2020-02-03 08:57:43,063 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:43,456 epoch 37 - iter 1/11 - loss 3.39949036 - samples/sec: 81.39
2020-02-03 08:57:44,058 epoch 37 - iter 2/11 - loss 3.89472556 - samples/sec: 59.11
2020-02-03 08:57:44,684 epoch 37 - iter 3/11 - loss 4.72021675 - samples/sec: 56.74
2020-02-03 08:57:45,259 epoch 37 - iter 4/11 - loss 4.12646103 - samples/sec: 63.25
2020-02-03 08:57:45,800 epoch 37 - iter 5/11 - loss 4.22146978 - samples/sec: 66.49
2020-02-03 08:57:46,408 epoch 37 - iter 6/11 - loss 4.49991512 - samples/sec: 58.75
2020-02-03 08:57:46,949 epoch 37 - iter 7/11 - loss 4.81867266 - samples/sec: 67.62
2020-02-03 08:57:47,485 epoch 37 - iter 8/11 - loss 5.04012769 - samples/sec: 67.62
2020-02-03 08:57:48,171 epoch 37 - iter 9/11 - loss 5.05430259 - samples/sec: 51.20
2020-02-03 08:57:48,799 epoch 37 - iter 10/11 - loss 5.32401576 - samples/sec: 57.42
2020-02-03 08:57:49,275 epoch 37 - iter 11/11 - loss 5.25648893 - samples/sec: 79.11
2020-02-03 08:57:49,337 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:49,337 EPOCH 37 done: loss 5.2565 - lr 0.0004
2020-02-03 08:57:49,791 DEV : loss 5.882631301879883 - score 0.022
2020-02-03 08:57:49,801 BAD EPOCHS (no improvement): 3
2020-02-03 08:57:53,455 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:53,987 epoch 38 - iter 1/11 - loss 4.10286808 - samples/sec: 60.07
2020-02-03 08:57:54,631 epoch 38 - iter 2/11 - loss 4.19697595 - samples/sec: 54.70
2020-02-03 08:57:55,215 epoch 38 - iter 3/11 - loss 3.92642196 - samples/sec: 62.24
2020-02-03 08:57:55,820 epoch 38 - iter 4/11 - loss 4.44362926 - samples/sec: 58.79
2020-02-03 08:57:56,465 epoch 38 - iter 5/11 - loss 4.53568478 - samples/sec: 55.70
2020-02-03 08:57:57,047 epoch 38 - iter 6/11 - loss 4.85243917 - samples/sec: 61.31
2020-02-03 08:57:57,652 epoch 38 - iter 7/11 - loss 5.12265675 - samples/sec: 59.04
2020-02-03 08:57:58,279 epoch 38 - iter 8/11 - loss 5.07453036 - samples/sec: 56.71
2020-02-03 08:57:58,884 epoch 38 - iter 9/11 - loss 5.10518773 - samples/sec: 59.91
2020-02-03 08:57:59,466 epoch 38 - iter 10/11 - loss 5.28046565 - samples/sec: 61.37
2020-02-03 08:57:59,847 epoch 38 - iter 11/11 - loss 5.24793577 - samples/sec: 99.72
2020-02-03 08:57:59,909 ----------------------------------------------------------------------------------------------------
2020-02-03 08:57:59,917 EPOCH 38 done: loss 5.2479 - lr 0.0004
2020-02-03 08:58:00,383 DEV : loss 5.879620552062988 - score 0.022
Epoch    38: reducing learning rate of group 0 to 1.9531e-04.
2020-02-03 08:58:00,391 BAD EPOCHS (no improvement): 4
2020-02-03 08:58:03,999 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:04,444 epoch 39 - iter 1/11 - loss 4.46234560 - samples/sec: 72.07
2020-02-03 08:58:05,026 epoch 39 - iter 2/11 - loss 4.89645529 - samples/sec: 62.24
2020-02-03 08:58:05,664 epoch 39 - iter 3/11 - loss 4.81580019 - samples/sec: 55.63
2020-02-03 08:58:06,198 epoch 39 - iter 4/11 - loss 5.05525029 - samples/sec: 69.04
2020-02-03 08:58:06,813 epoch 39 - iter 5/11 - loss 4.84441748 - samples/sec: 58.59
2020-02-03 08:58:07,447 epoch 39 - iter 6/11 - loss 4.98805404 - samples/sec: 55.77
2020-02-03 08:58:08,092 epoch 39 - iter 7/11 - loss 4.83012571 - samples/sec: 54.76
2020-02-03 08:58:08,646 epoch 39 - iter 8/11 - loss 5.22021863 - samples/sec: 64.84
2020-02-03 08:58:09,207 epoch 39 - iter 9/11 - loss 5.15839076 - samples/sec: 63.84
2020-02-03 08:58:09,771 epoch 39 - iter 10/11 - loss 5.32761638 - samples/sec: 63.58
2020-02-03 08:58:10,225 epoch 39 - iter 11/11 - loss 5.10062339 - samples/sec: 81.33
2020-02-03 08:58:10,296 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:10,296 EPOCH 39 done: loss 5.1006 - lr 0.0002
2020-02-03 08:58:10,767 DEV : loss 5.880942344665527 - score 0.022
2020-02-03 08:58:10,777 BAD EPOCHS (no improvement): 1
2020-02-03 08:58:14,418 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:14,962 epoch 40 - iter 1/11 - loss 4.34396458 - samples/sec: 58.79
2020-02-03 08:58:15,536 epoch 40 - iter 2/11 - loss 5.62661099 - samples/sec: 62.57
2020-02-03 08:58:16,124 epoch 40 - iter 3/11 - loss 4.56118226 - samples/sec: 60.90
2020-02-03 08:58:16,716 epoch 40 - iter 4/11 - loss 4.69568509 - samples/sec: 61.34
2020-02-03 08:58:17,374 epoch 40 - iter 5/11 - loss 4.75908351 - samples/sec: 53.76
2020-02-03 08:58:17,976 epoch 40 - iter 6/11 - loss 4.63431720 - samples/sec: 60.20
2020-02-03 08:58:18,552 epoch 40 - iter 7/11 - loss 5.02879575 - samples/sec: 62.26
2020-02-03 08:58:19,055 epoch 40 - iter 8/11 - loss 5.00541815 - samples/sec: 74.06
2020-02-03 08:58:19,637 epoch 40 - iter 9/11 - loss 5.03749930 - samples/sec: 61.08
2020-02-03 08:58:20,219 epoch 40 - iter 10/11 - loss 4.97096617 - samples/sec: 61.33
2020-02-03 08:58:20,624 epoch 40 - iter 11/11 - loss 5.25848824 - samples/sec: 92.93
2020-02-03 08:58:20,693 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:20,695 EPOCH 40 done: loss 5.2585 - lr 0.0002
2020-02-03 08:58:21,188 DEV : loss 5.880982875823975 - score 0.022
2020-02-03 08:58:21,199 BAD EPOCHS (no improvement): 2
2020-02-03 08:58:24,930 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:25,380 epoch 41 - iter 1/11 - loss 4.50473261 - samples/sec: 71.36
2020-02-03 08:58:25,974 epoch 41 - iter 2/11 - loss 3.87273896 - samples/sec: 59.95
2020-02-03 08:58:26,699 epoch 41 - iter 3/11 - loss 4.82082597 - samples/sec: 48.92
2020-02-03 08:58:27,367 epoch 41 - iter 4/11 - loss 5.68294698 - samples/sec: 52.85
2020-02-03 08:58:27,953 epoch 41 - iter 5/11 - loss 5.06655664 - samples/sec: 60.95
2020-02-03 08:58:28,545 epoch 41 - iter 6/11 - loss 5.32939577 - samples/sec: 61.31
2020-02-03 08:58:29,169 epoch 41 - iter 7/11 - loss 5.16050720 - samples/sec: 56.96
2020-02-03 08:58:29,754 epoch 41 - iter 8/11 - loss 5.36520886 - samples/sec: 61.00
2020-02-03 08:58:30,400 epoch 41 - iter 9/11 - loss 5.30810801 - samples/sec: 55.66
2020-02-03 08:58:31,045 epoch 41 - iter 10/11 - loss 5.24546852 - samples/sec: 54.73
2020-02-03 08:58:31,509 epoch 41 - iter 11/11 - loss 5.16899443 - samples/sec: 79.23
2020-02-03 08:58:31,578 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:31,580 EPOCH 41 done: loss 5.1690 - lr 0.0002
2020-02-03 08:58:32,115 DEV : loss 5.881915092468262 - score 0.022
2020-02-03 08:58:32,125 BAD EPOCHS (no improvement): 3
2020-02-03 08:58:35,769 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:36,195 epoch 42 - iter 1/11 - loss 5.13844013 - samples/sec: 75.14
2020-02-03 08:58:36,799 epoch 42 - iter 2/11 - loss 4.58374739 - samples/sec: 58.84
2020-02-03 08:58:37,333 epoch 42 - iter 3/11 - loss 4.94820595 - samples/sec: 69.02
2020-02-03 08:58:38,009 epoch 42 - iter 4/11 - loss 5.06684828 - samples/sec: 52.75
2020-02-03 08:58:38,603 epoch 42 - iter 5/11 - loss 4.46963367 - samples/sec: 61.05
2020-02-03 08:58:39,185 epoch 42 - iter 6/11 - loss 4.51425139 - samples/sec: 62.35
2020-02-03 08:58:39,760 epoch 42 - iter 7/11 - loss 4.63971560 - samples/sec: 63.47
2020-02-03 08:58:40,393 epoch 42 - iter 8/11 - loss 4.88308132 - samples/sec: 56.64
2020-02-03 08:58:41,000 epoch 42 - iter 9/11 - loss 5.01398595 - samples/sec: 58.78
2020-02-03 08:58:41,684 epoch 42 - iter 10/11 - loss 5.07032166 - samples/sec: 53.05
2020-02-03 08:58:42,148 epoch 42 - iter 11/11 - loss 5.35146193 - samples/sec: 81.37
2020-02-03 08:58:42,208 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:42,210 EPOCH 42 done: loss 5.3515 - lr 0.0002
2020-02-03 08:58:42,734 DEV : loss 5.882656574249268 - score 0.022
Epoch    42: reducing learning rate of group 0 to 9.7656e-05.
2020-02-03 08:58:42,744 BAD EPOCHS (no improvement): 4
2020-02-03 08:58:46,468 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:46,468 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:46,468 learning rate too small - quitting training!
2020-02-03 08:58:46,468 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:49,938 ----------------------------------------------------------------------------------------------------
2020-02-03 08:58:49,938 Testing using best model ...
2020-02-03 08:58:49,940 loading file resources\taggers\example-ner\best-model.pt
2020-02-03 08:58:51,547 1.0	0.01	0.0198
2020-02-03 08:58:51,547 
MICRO_AVG: acc 0.01 - f1-score 0.0198
MACRO_AVG: acc 0.0044 - f1-score 0.00855
corporation tp: 0 - fp: 0 - fn: 7 - tn: 0 - precision: 0.0000 - recall: 0.0000 - accuracy: 0.0000 - f1-score: 0.0000
creative-work tp: 0 - fp: 0 - fn: 18 - tn: 0 - precision: 0.0000 - recall: 0.0000 - accuracy: 0.0000 - f1-score: 0.0000
group      tp: 0 - fp: 0 - fn: 17 - tn: 0 - precision: 0.0000 - recall: 0.0000 - accuracy: 0.0000 - f1-score: 0.0000
location   tp: 0 - fp: 0 - fn: 9 - tn: 0 - precision: 0.0000 - recall: 0.0000 - accuracy: 0.0000 - f1-score: 0.0000
person     tp: 1 - fp: 0 - fn: 37 - tn: 1 - precision: 1.0000 - recall: 0.0263 - accuracy: 0.0263 - f1-score: 0.0513
product    tp: 0 - fp: 0 - fn: 11 - tn: 0 - precision: 0.0000 - recall: 0.0000 - accuracy: 0.0000 - f1-score: 0.0000
2020-02-03 08:58:51,547 ----------------------------------------------------------------------------------------------------






    Out[15]:





{'test_score': 0.0198,
 'dev_score_history': [0.0,
  0.0,
  0.0,
  0.0,
  0.0,
  0.0,
  0.0,
  0.0,
  0.0,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022,
  0.022],
 'train_loss_history': [19.64247400110418,
  7.3237409158186475,
  6.647785360162908,
  6.274353677576238,
  6.253459583629262,
  6.0311057567596436,
  5.829011396928267,
  5.599017815156416,
  5.606011629104614,
  5.485666448419744,
  5.677339813926003,
  5.509015993638472,
  5.371448170055043,
  5.417202407663519,
  5.572993430224332,
  5.156284202228893,
  5.282446471127597,
  5.409523942253807,
  5.422696980563077,
  5.130376208912242,
  5.2385149652307685,
  5.317819421941584,
  5.333681713451039,
  5.252829486673528,
  5.352623267607256,
  5.307583288712935,
  5.209459673274647,
  5.221857786178589,
  5.184349103407427,
  5.064212669025768,
  5.3184214505282315,
  5.260098197243431,
  5.226596680554477,
  5.196419065648859,
  5.184695503928444,
  5.119798389348117,
  5.256488930095326,
  5.247935771942139,
  5.100623390891335,
  5.258488243276423,
  5.168994426727295,
  5.351461930708452],
 'dev_loss_history': [tensor(7.9446),
  tensor(7.5118),
  tensor(7.2542),
  tensor(6.8096),
  tensor(6.4689),
  tensor(6.3389),
  tensor(6.1781),
  tensor(6.2680),
  tensor(6.2708),
  tensor(6.1206),
  tensor(5.9769),
  tensor(6.0627),
  tensor(6.0837),
  tensor(5.9471),
  tensor(5.9115),
  tensor(5.9733),
  tensor(6.0286),
  tensor(5.8890),
  tensor(5.8880),
  tensor(5.9335),
  tensor(5.9252),
  tensor(5.8935),
  tensor(5.9144),
  tensor(5.9025),
  tensor(5.8872),
  tensor(5.9016),
  tensor(5.9037),
  tensor(5.8952),
  tensor(5.8963),
  tensor(5.8927),
  tensor(5.8869),
  tensor(5.8821),
  tensor(5.8818),
  tensor(5.8820),
  tensor(5.8838),
  tensor(5.8844),
  tensor(5.8826),
  tensor(5.8796),
  tensor(5.8809),
  tensor(5.8810),
  tensor(5.8819),
  tensor(5.8827)]}

Plot the training curves and results:



In [17]:

    
from flair.visual.training_curves import Plotter
plotter = Plotter()
plotter.plot_training_curves('resources/taggers/example-ner/loss.tsv')
plotter.plot_weights('resources/taggers/example-ner/weights.txt')









    



2020-02-03 09:00:03,963 ----------------------------------------------------------------------------------------------------
2020-02-03 09:00:03,964 WARNING: No LOSS found for test split in this data.
2020-02-03 09:00:03,964 Are you sure you want to plot LOSS and not another value?
2020-02-03 09:00:03,965 ----------------------------------------------------------------------------------------------------
2020-02-03 09:00:03,987 ----------------------------------------------------------------------------------------------------
2020-02-03 09:00:03,988 WARNING: No F1 found for test split in this data.
2020-02-03 09:00:03,989 Are you sure you want to plot F1 and not another value?
2020-02-03 09:00:03,990 ----------------------------------------------------------------------------------------------------
Loss and F1 plots are saved in resources\taggers\example-ner\training.png






    












    



Weights plots are saved in resources\taggers\example-ner\weights.png

Use the model via the predict method:



In [25]:

    
from flair.data import Sentence
model = SequenceTagger.load('resources/taggers/example-ner/final-model.pt')
sentence = Sentence('John lives in the Empire State Building .')
model.predict(sentence)
print(sentence.to_tagged_string())









    



2020-02-03 09:08:37,099 loading file resources/taggers/example-ner/final-model.pt
John lives in the Empire State Building .