notebook.community

Edit and run 





In [1]:



    


import warnings
warnings.filterwarnings("ignore") # suppress h5py deprecation warning

import os
import backtrader as bt
import numpy as np

from btgym import BTgymEnv, BTgymDataset
from btgym.strategy.observers import Reward, Position, NormPnL
from btgym.algorithms import Launcher, Unreal, AacStackedRL2Policy
from btgym.research.strategy_gen_4 import DevStrat_4_11



    


















    






WARNING: Logging before flag parsing goes to stderr.
W0501 12:37:55.281946 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:19: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

W0501 12:37:55.282888 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:19: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

W0501 12:37:55.283457 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:22: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

Stacked LSTM Agent usage example.

Based on NAV_A3C+D from "LEARNING TO NAVIGATE IN COMPLEX ENVIRONMENTS" paper by Mirowski at al.;

Modifications to original paper arhcitecture:

  • splitted Policy/Value outputs: Policy is taken off first LSTM layer, Value - off the second;
  • LSTM state initialisation: first RNN layer context (policy) is initialised on every episode start, while second (Value) is reset either on begining of every Trial (future work) or or every N-constant episodes (60 for this example), motivated by RL^2 approach by Duan et al., "FAST REINFORCEMENT LEARNING VIA SLOW REINFORCEMENT LEARNING";
  • inner/external observation state state split: external (market) is encoded via conolution layers and fed to first LSTM layer, inner (broker) state is fed into second LSTM layer, can optionally be encoded via separate convolution block (doesnt seem to improve much though);
  • optional Value Replay losss (Unreal feature) improves sample efficiency, but is computationally expensive;

Other details:

  • All convolution and LSTM layers are layer-normalized, see "Layer Normalisation" paper by Jimmy Ba at al.;

  • Upd 2.02.18: linear layers are Noisy-Net ones, see: [Noisy Networks for Exploration] (https://arxiv.org/abs/1706.10295) paper by Fortunato at al.; policy output is centered using layer normalisation; added linearly decayed state scaling;

  • A3C option time_flat is ON by default, improves training stability, reduces computation costs, see Base_AAC class Note for details;

Diagram: https://kismuz.github.io/btgym/_images/a3c_stacked_lstm_agent.png

NOTE: Currently it takes ~20M env.steps to fit 6-month 1min bars data set. Much faster on smaller ones.





In [2]:



    


# Set backtesting engine parameters:

MyCerebro = bt.Cerebro()

# Define strategy and broker account parameters:
MyCerebro.addstrategy(
    DevStrat_4_11,
    start_cash=2000,  # initial broker cash
    commission=0.0001,  # commisssion to imitate spread
    leverage=10.0,
    order_size=2000,  # fixed stake, mind leverage
    drawdown_call=10, # max % to loose, in percent of initial cash
    target_call=10,  # max % to win, same
    skip_frame=10,
    gamma=0.99,
    reward_scale=7, # gardient`s nitrox, touch with care!
    state_ext_scale = np.linspace(3e3, 1e3, num=5)
)
# Visualisations for reward, position and PnL dynamics:
MyCerebro.addobserver(Reward)
MyCerebro.addobserver(Position)
MyCerebro.addobserver(NormPnL)

# Data: uncomment to get up to six month of 1 minute bars:
data_m1_6_month = [
    './data/DAT_ASCII_EURUSD_M1_201701.csv',
    './data/DAT_ASCII_EURUSD_M1_201702.csv',
    './data/DAT_ASCII_EURUSD_M1_201703.csv',
    './data/DAT_ASCII_EURUSD_M1_201704.csv',
    './data/DAT_ASCII_EURUSD_M1_201705.csv',
    './data/DAT_ASCII_EURUSD_M1_201706.csv',
]

# Uncomment single choice:
MyDataset = BTgymDataset(
    #filename=data_m1_6_month,
    filename='./data/test_sine_1min_period256_delta0002.csv',  # simple sine 
    start_weekdays={0, 1, 2, 3, 4, 5, 6},
    episode_duration={'days': 1, 'hours': 23, 'minutes': 40}, # note: 2day-long episode
    start_00=False,
    time_gap={'hours': 10},
)

env_config = dict(
    class_ref=BTgymEnv, 
    kwargs=dict(
        dataset=MyDataset,
        engine=MyCerebro,
        render_modes=['episode', 'human', 'internal', ], #'external'],
        render_state_as_image=True,
        render_ylabel='OHL_diff. / Internals',
        render_size_episode=(12,8),
        render_size_human=(9, 4),
        render_size_state=(11, 3),
        render_dpi=75,
        port=5000,
        data_port=4999,
        connect_timeout=90,
        verbose=0,
    )
)

cluster_config = dict(
    host='127.0.0.1',
    port=12230,
    num_workers=4,  # set according CPU's available or so
    num_ps=1,
    num_envs=1,
    log_dir=os.path.expanduser('~/tmp/test_4_11'),  # current checkpoints and summaries are here
    initial_ckpt_dir=os.path.expanduser('~/tmp/pre_trained_model/test_4_11'),  # load pre-trained model, if chekpoint found  
)

policy_config = dict(
    class_ref=AacStackedRL2Policy,
    kwargs={
        'lstm_layers': (256, 256),
        'lstm_2_init_period': 60,
    }
)

trainer_config = dict(
    class_ref=Unreal,
    kwargs=dict(
        opt_learn_rate=[1e-4, 1e-4], # random log-uniform 
        opt_end_learn_rate=1e-5,
        opt_decay_steps=50*10**6,
        model_gamma=0.99,
        model_gae_lambda=1.0,
        model_beta=0.05, # entropy reg
        rollout_length=20,
        time_flat=True, 
        use_value_replay=False, 
        model_summary_freq=10,
        episode_summary_freq=1,
        env_render_freq=2,
    )
)



    


















    






BTgymDataset class is DEPRECATED, use btgym.datafeed.derivative.BTgymDataset2 instead.

















In [3]:



    


launcher = Launcher(
    cluster_config=cluster_config,
    env_config=env_config,
    trainer_config=trainer_config,
    policy_config=policy_config,
    test_mode=False,
    max_env_steps=100*10**6,
    save_secs=300,  # save checkpoint every N seconds (default is 600)
    root_random_seed=0,
    verbose=0
)

# Train it:
launcher.run()



    


















    






</home/muzikin/tmp/test_4_11> already exists. Override[y/n]? y
[2019-05-01 09:38:23.514035] NOTICE: LauncherShell: files in: /home/muzikin/tmp/test_4_11 purged.










    






W0501 12:38:24.120093 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:191: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.

W0501 12:38:24.126140 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:201: The name tf.train.Server is deprecated. Please use tf.distribute.Server instead.
W0501 12:38:24.126061 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:191: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.


W0501 12:38:24.129139 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:205: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0501 12:38:24.129642 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:212: The name tf.train.Server is deprecated. Please use tf.distribute.Server instead.

W0501 12:38:24.135105 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:216: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0501 12:38:26.394120 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:221: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.











    






[2019-05-01 09:38:26.406598] NOTICE: UNREAL_0: learn_rate: 0.000100, entropy_beta: 0.050000










    






W0501 12:38:26.408271 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:428: The name tf.train.replica_device_setter is deprecated. Please use tf.compat.v1.train.replica_device_setter instead.

W0501 12:38:26.410790 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:860: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0501 12:38:26.415243 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/utils.py:52: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0501 12:38:26.479786 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:58: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
W0501 12:38:26.620964 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/policy/stacked_lstm.py:189: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
W0501 12:38:27.009598 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/layers.py:34: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
W0501 12:38:27.031301 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:140: The name tf.nn.rnn_cell.DropoutWrapper is deprecated. Please use tf.compat.v1.nn.rnn_cell.DropoutWrapper instead.

W0501 12:38:27.035575 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:144: MultiRNNCell.__init__ (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.StackedRNNCells, and will be replaced by that in Tensorflow 2.0.
W0501 12:38:27.068239 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:159: static_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell, unroll=True)`, which is equivalent to this API
W0501 12:38:27.583704 139801407391552 deprecation.py:506] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:28.853516 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:827: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:29.136342 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:191: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.

W0501 12:38:29.142140 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:212: The name tf.train.Server is deprecated. Please use tf.distribute.Server instead.

W0501 12:38:29.142348 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:191: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
W0501 12:38:29.149414 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:216: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.


W0501 12:38:29.150807 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:191: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.
W0501 12:38:29.153527 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:212: The name tf.train.Server is deprecated. Please use tf.distribute.Server instead.


W0501 12:38:29.159186 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:212: The name tf.train.Server is deprecated. Please use tf.distribute.Server instead.

W0501 12:38:29.157516 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:216: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
W0501 12:38:29.161225 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:216: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.












    






********************************************************************************************
**  Press `Ctrl-C` or jupyter:[Kernel]->[Interrupt] to stop training and close launcher.  **
********************************************************************************************











    






W0501 12:38:30.301744 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:221: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

W0501 12:38:30.305003 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:221: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.











    






[2019-05-01 09:38:30.307060] NOTICE: UNREAL_1: learn_rate: 0.000100, entropy_beta: 0.050000










    






W0501 12:38:30.308536 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:428: The name tf.train.replica_device_setter is deprecated. Please use tf.compat.v1.train.replica_device_setter instead.











    






[2019-05-01 09:38:30.310888] NOTICE: UNREAL_2: learn_rate: 0.000100, entropy_beta: 0.050000










    






W0501 12:38:30.312513 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:428: The name tf.train.replica_device_setter is deprecated. Please use tf.compat.v1.train.replica_device_setter instead.
W0501 12:38:30.312589 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:860: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0501 12:38:30.315204 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:860: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0501 12:38:30.315888 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:221: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.
W0501 12:38:30.317957 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/utils.py:52: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.













    






[2019-05-01 09:38:30.321333] NOTICE: UNREAL_3: learn_rate: 0.000100, entropy_beta: 0.050000










    






W0501 12:38:30.323216 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:428: The name tf.train.replica_device_setter is deprecated. Please use tf.compat.v1.train.replica_device_setter instead.
W0501 12:38:30.333499 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/utils.py:52: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.


W0501 12:38:30.338224 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:860: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0501 12:38:30.341470 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/utils.py:52: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0501 12:38:30.414032 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:58: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.W0501 12:38:30.414503 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:58: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.

W0501 12:38:30.431789 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:58: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
W0501 12:38:30.532077 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/policy/stacked_lstm.py:189: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
W0501 12:38:30.566405 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/policy/stacked_lstm.py:189: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
W0501 12:38:30.589377 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/policy/stacked_lstm.py:189: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
W0501 12:38:30.806571 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:463: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W0501 12:38:30.817833 139801407391552 deprecation.py:323] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/learning_rate_schedule.py:417: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
W0501 12:38:30.859029 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:34: The name tf.losses.mean_squared_error is deprecated. Please use tf.compat.v1.losses.mean_squared_error instead.

W0501 12:38:30.923352 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:43: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W0501 12:38:30.945428 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:636: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W0501 12:38:31.039243 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/layers.py:34: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
W0501 12:38:31.079536 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:140: The name tf.nn.rnn_cell.DropoutWrapper is deprecated. Please use tf.compat.v1.nn.rnn_cell.DropoutWrapper instead.

W0501 12:38:31.083827 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:144: MultiRNNCell.__init__ (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.StackedRNNCells, and will be replaced by that in Tensorflow 2.0.
W0501 12:38:31.117558 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/layers.py:34: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
W0501 12:38:31.128330 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:159: static_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell, unroll=True)`, which is equivalent to this API
W0501 12:38:31.141059 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/layers.py:34: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
W0501 12:38:31.158703 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:140: The name tf.nn.rnn_cell.DropoutWrapper is deprecated. Please use tf.compat.v1.nn.rnn_cell.DropoutWrapper instead.

W0501 12:38:31.170001 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:144: MultiRNNCell.__init__ (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.StackedRNNCells, and will be replaced by that in Tensorflow 2.0.
W0501 12:38:31.203523 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:140: The name tf.nn.rnn_cell.DropoutWrapper is deprecated. Please use tf.compat.v1.nn.rnn_cell.DropoutWrapper instead.

W0501 12:38:31.211045 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:144: MultiRNNCell.__init__ (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.StackedRNNCells, and will be replaced by that in Tensorflow 2.0.
W0501 12:38:31.234858 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:159: static_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell, unroll=True)`, which is equivalent to this API
W0501 12:38:31.284226 139801407391552 deprecation.py:323] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/networks.py:159: static_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell, unroll=True)`, which is equivalent to this API
W0501 12:38:31.757507 139801407391552 deprecation.py:506] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:31.885086 139801407391552 deprecation.py:506] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:31.947027 139801407391552 deprecation.py:506] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:33.218857 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:827: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:33.296944 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:827: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:33.465772 139801407391552 deprecation.py:506] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:827: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0501 12:38:34.013138 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:704: The name tf.summary.merge is deprecated. Please use tf.compat.v1.summary.merge instead.

W0501 12:38:34.018737 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:729: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

W0501 12:38:34.044258 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:313: The name tf.initializers.variables is deprecated. Please use tf.compat.v1.initializers.variables instead.











    






[2019-05-01 09:38:34.843608] NOTICE: Worker_0: initializing all parameters...










    






W0501 12:38:35.161447 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:463: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W0501 12:38:35.174301 139801407391552 deprecation.py:323] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/learning_rate_schedule.py:417: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
W0501 12:38:35.218007 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:34: The name tf.losses.mean_squared_error is deprecated. Please use tf.compat.v1.losses.mean_squared_error instead.

W0501 12:38:35.244060 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:463: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W0501 12:38:35.256249 139801407391552 deprecation.py:323] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/learning_rate_schedule.py:417: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
W0501 12:38:35.264313 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:43: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W0501 12:38:35.277047 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:636: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W0501 12:38:35.307018 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:34: The name tf.losses.mean_squared_error is deprecated. Please use tf.compat.v1.losses.mean_squared_error instead.

W0501 12:38:35.350171 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:43: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W0501 12:38:35.361384 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:636: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W0501 12:38:35.498770 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:463: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W0501 12:38:35.508708 139801407391552 deprecation.py:323] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/keras/optimizer_v2/learning_rate_schedule.py:417: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
W0501 12:38:35.559430 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:34: The name tf.losses.mean_squared_error is deprecated. Please use tf.compat.v1.losses.mean_squared_error instead.

W0501 12:38:35.602700 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/nn/losses.py:43: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

I0501 12:38:35.605363 139801407391552 session_manager.py:500] Running local_init_op.
W0501 12:38:35.618116 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:636: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

I0501 12:38:35.811869 139801407391552 session_manager.py:502] Done running local_init_op.










    






[2019-05-01 09:38:35.813111] NOTICE: Worker_0: no saved model parameters found in:
/home/muzikin/tmp/pre_trained_model/test_4_11
[2019-05-01 09:38:35.927841] NOTICE: Worker_0: no saved model parameters found in:
/home/muzikin/tmp/test_4_11/current_train_checkpoint
[2019-05-01 09:38:35.928963] NOTICE: Worker_0: training from scratch...










    






W0501 12:38:35.930952 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:367: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.











    






[2019-05-01 09:38:37.420617] NOTICE: BTgymDataServer_0: Initial global_time set to: 2017-01-01 03:00:00 / stamp: 1483228800.0
[2019-05-01 09:38:37.572177] NOTICE: Worker_0: started training at step: 0










    






W0501 12:38:38.438426 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:704: The name tf.summary.merge is deprecated. Please use tf.compat.v1.summary.merge instead.

W0501 12:38:38.448212 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:729: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

W0501 12:38:38.470903 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:313: The name tf.initializers.variables is deprecated. Please use tf.compat.v1.initializers.variables instead.

W0501 12:38:38.486378 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:704: The name tf.summary.merge is deprecated. Please use tf.compat.v1.summary.merge instead.

W0501 12:38:38.497119 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:729: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

W0501 12:38:38.521061 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:313: The name tf.initializers.variables is deprecated. Please use tf.compat.v1.initializers.variables instead.

W0501 12:38:38.806043 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:704: The name tf.summary.merge is deprecated. Please use tf.compat.v1.summary.merge instead.

W0501 12:38:38.812782 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/aac.py:729: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

W0501 12:38:38.836258 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:313: The name tf.initializers.variables is deprecated. Please use tf.compat.v1.initializers.variables instead.











    






[2019-05-01 09:38:39.408251] NOTICE: Worker_2: initializing all parameters...
[2019-05-01 09:38:39.470794] NOTICE: Worker_1: initializing all parameters...
[2019-05-01 09:38:39.857457] NOTICE: Worker_3: initializing all parameters...










    






I0501 12:38:40.195459 139801407391552 session_manager.py:500] Running local_init_op.
I0501 12:38:40.255829 139801407391552 session_manager.py:500] Running local_init_op.
I0501 12:38:40.448325 139801407391552 session_manager.py:502] Done running local_init_op.










    






[2019-05-01 09:38:40.449941] NOTICE: Worker_2: no saved model parameters found in:
/home/muzikin/tmp/pre_trained_model/test_4_11










    






I0501 12:38:40.544522 139801407391552 session_manager.py:502] Done running local_init_op.










    






[2019-05-01 09:38:40.545895] NOTICE: Worker_1: no saved model parameters found in:
/home/muzikin/tmp/pre_trained_model/test_4_11
[2019-05-01 09:38:40.605498] NOTICE: Worker_2: no saved model parameters found in:
/home/muzikin/tmp/test_4_11/current_train_checkpoint
[2019-05-01 09:38:40.606864] NOTICE: Worker_2: training from scratch...










    






W0501 12:38:40.609505 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:367: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

I0501 12:38:40.731080 139801407391552 session_manager.py:500] Running local_init_op.










    






[2019-05-01 09:38:40.737775] NOTICE: Worker_1: no saved model parameters found in:
/home/muzikin/tmp/test_4_11/current_train_checkpoint
[2019-05-01 09:38:40.745865] NOTICE: Worker_1: training from scratch...










    






W0501 12:38:40.749837 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:367: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

I0501 12:38:40.983467 139801407391552 session_manager.py:502] Done running local_init_op.










    






[2019-05-01 09:38:40.985029] NOTICE: Worker_3: no saved model parameters found in:
/home/muzikin/tmp/pre_trained_model/test_4_11
[2019-05-01 09:38:41.111817] NOTICE: Worker_3: no saved model parameters found in:
/home/muzikin/tmp/test_4_11/current_train_checkpoint
[2019-05-01 09:38:41.112857] NOTICE: Worker_3: training from scratch...










    






W0501 12:38:41.114001 139801407391552 deprecation_wrapper.py:119] From /home/muzikin/Repos/btgym/btgym/algorithms/worker.py:367: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.











    






[2019-05-01 09:38:42.189233] NOTICE: Worker_2: started training at step: 0
[2019-05-01 09:38:42.270320] NOTICE: Worker_1: started training at step: 0
[2019-05-01 09:38:42.632287] NOTICE: Worker_3: started training at step: 0
[2019-05-01 09:43:38.028686] NOTICE: Worker_0: env. step: 66070; cluster speed: 219 step/sec; checkpoint saved.










    






W0501 12:48:38.232499 139801407391552 deprecation.py:323] From /home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/training/saver.py:960: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.










    






[2019-05-01 09:48:38.251251] NOTICE: Worker_0: env. step: 134321; cluster speed: 227 step/sec; checkpoint saved.
[2019-05-01 09:53:38.568674] NOTICE: Worker_0: env. step: 202690; cluster speed: 227 step/sec; checkpoint saved.










    






Process BTgymServer-3:1:
Process BTgymServer-5:1:
Process BTgymDataFeedServer-2:1:
Process BTgymServer-4:1:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/muzikin/Repos/btgym/btgym/server.py", line 713, in run
    episode = cerebro.run(stdstats=True, preload=False, oldbuysell=True)[0]
  File "/home/muzikin/Repos/btgym/btgym/server.py", line 713, in run
    episode = cerebro.run(stdstats=True, preload=False, oldbuysell=True)[0]
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1298, in runstrategies
    self._runnext(runstrats)
  File "/home/muzikin/Repos/btgym/btgym/server.py", line 713, in run
    episode = cerebro.run(stdstats=True, preload=False, oldbuysell=True)[0]
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1561, in _runnext
    self._dtmaster = dmaster.num2date(dt0)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1298, in runstrategies
    self._runnext(runstrats)
  File "/home/muzikin/Repos/btgym/btgym/dataserver.py", line 136, in run
    service_input = socket.recv_pyobj()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1298, in runstrategies
    self._runnext(runstrats)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1630, in _runnext
    strat._next()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/feed.py", line 256, in num2date
    return num2date(dt, tz or self._tz, naive)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/strategy.py", line 347, in _next
    super(Strategy, self)._next()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1623, in _runnext
    self._brokernotify()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/lineiterator.py", line 263, in _next
    indicator._next()
  File "/home/muzikin/.local/lib/python3.6/site-packages/zmq/sugar/socket.py", line 626, in recv_pyobj
    msg = self.recv(flags)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 1370, in _brokernotify
    owner._addnotification(order, quicknotify=self.p.quicknotify)
  File "zmq/backend/cython/socket.pyx", line 788, in zmq.backend.cython.socket.Socket.recv
Process DrawCerebro-2:2:177:
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/utils/dateintern.py", line 189, in num2date
    dt += datetime.timedelta(microseconds=1e6 - microsecond)
  File "zmq/backend/cython/socket.pyx", line 824, in zmq.backend.cython.socket.Socket.recv
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/lineiterator.py", line 260, in _next
    clock_len = self._clk_update()
KeyboardInterrupt
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/lineiterator.py", line 289, in _clk_update
    self.forward()
  File "zmq/backend/cython/socket.pyx", line 186, in zmq.backend.cython.socket._recv_copy
Traceback (most recent call last):
  File "zmq/backend/cython/checkrc.pxd", line 12, in zmq.backend.cython.checkrc._check_rc










    






[2019-05-01 09:55:56.864915] NOTICE: LauncherShell: worker_1 has joined.










    






  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/lineseries.py", line 554, in forward
    self.lines.forward(value, size)










    






[2019-05-01 09:55:56.867504] NOTICE: LauncherShell: worker_2 has joined.










    






  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/strategy.py", line 559, in _addnotification
    comminfo=order.comminfo)










    






[2019-05-01 09:55:56.868895] NOTICE: LauncherShell: worker_3 has joined.










    






  File "/home/muzikin/Repos/btgym/btgym/rendering/plotter.py", line 76, in run
    figfilename='_tmp_btgym_render.png',
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/lineseries.py", line 254, in forward
    line.forward(value, size=size)










    






[2019-05-01 09:55:56.870204] NOTICE: LauncherShell: chief_worker_0 has joined.










    






  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/cerebro.py", line 991, in plot
    start=start, end=end, use=use)










    






[2019-05-01 09:55:56.871413] NOTICE: LauncherShell: parameter_server_0 has joined.










    






  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/linebuffer.py", line 254, in forward
    self.idx += size










    






[2019-05-01 09:55:56.872689] NOTICE: LauncherShell: Launcher closed.










    






  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/linebuffer.py", line 98, in set_idx
    self._idx = idx
KeyboardInterrupt
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/trade.py", line 213, in update
    def update(self, order, size, price, value, commission, pnl,
KeyboardInterrupt
KeyboardInterrupt
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/plot/plot.py", line 189, in plot
    self.plotind(None, ptop, subinds=self.dplotsover[ptop])
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/plot/plot.py", line 392, in plotind
    ax = masterax or self.newaxis(ind, rowspan=self.pinf.sch.rowsminor)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/backtrader/plot/plot.py", line 358, in newaxis
    rowspan=rowspan, sharex=self.pinf.sharex)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/pyplot.py", line 1292, in subplot2grid
    a = fig.add_subplot(subplotspec, **kwargs)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/figure.py", line 1021, in add_subplot
    a = subplot_class_factory(projection_class)(self, *args, **kwargs)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axes/_subplots.py", line 73, in __init__
    self._axes_class.__init__(self, fig, self.figbox, **kwargs)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 551, in __init__
    self.cla()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 984, in cla
    spine.cla()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/spines.py", line 170, in cla
    self.axis.cla()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axis.py", line 760, in cla
    self.reset_ticks()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axis.py", line 775, in reset_ticks
    self.minorTicks.extend([self._get_tick(major=False)])
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axis.py", line 1729, in _get_tick
    return XTick(self.axes, 0, '', major=major, **tick_kw)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axis.py", line 151, in __init__
    self.tick2line = self._get_tick2line()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/axis.py", line 434, in _get_tick2line
    zorder=self._zorder)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/lines.py", line 425, in __init__
    self.set_fillstyle(fillstyle)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/lines.py", line 555, in set_fillstyle
    self._marker.set_fillstyle(fs)
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/markers.py", line 237, in set_fillstyle
    self._recache()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/markers.py", line 208, in _recache
    self._marker_function()
  File "/home/muzikin/anaconda3/envs/tf/lib/python3.6/site-packages/matplotlib/markers.py", line 716, in _set_tickup
    self._transform = Affine2D().scale(1.0, 1.0)
KeyboardInterrupt

















In [ ]:



    


# Save, restore or resume:

# Use launcher.export_checkpoint() method to save most recent trained model parameters to external directory; 
# one can load it as pre-trained model for next run via cluster_gongig -> initial_ckpt_dir arg, (see above).
#
# Note: 
# 1. when loading pre-trained model, training is started at global_step=0 unlike
#    restoring from current checkpoint, when training resumes from last saved global_step value;
# 2. answering Yes to Launcher's `Override[y/n]?` affects log_dir content only;
# 3. launcher now got 'save_secs' arg, cpecifying how often checkpoints
should be written. Default value is 600;
# 4. exporting checkpoint overrides content of destination folder.
#
# Launcher starting routine:
# 1. if initial_ckpt_dir is given - try to load pre-trained model and start at step=0 if succeeded;
# 2. if failed - look for routinely saved checkpoint and if succeeded - resume training at step found in that point;
# 3. if that fails - start training from scratch.

launcher.export_checkpoint(os.path.expanduser('~/tmp/pre_trained_model/test_4_11'))



    











In [ ]:

Content source: Kismuz/btgym

Similar notebooks:

notebook.community | gallery | about