Tensorflow

Graph = Operations + Tensors

Session runs a graph on a device(s).



In [ ]:



In [ ]:

Tensorflow na GPU -

praktyczne uwagi pracy na tym samym systemie

https://www.tensorflow.org/tutorials/using_gpu

Limit memory:

- config.gpu_options.allow_growth = True
- config.gpu_options.per_process_gpu_memory_fraction = 0.01

Uwaga: GPU resources not released when session is closed https://github.com/tensorflow/tensorflow/issues/1727

Trzeba zakończyć proces pythona (w jupyter: kernel restart)

Aby sprawdzić zasoby GPU: nvidia-smi



In [9]:

    
!nvidia-smi









    



Fri Mar 17 10:00:44 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.57                 Driver Version: 367.57                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K40m          Off  | 0000:04:00.0     Off |                    0 |
| N/A   30C    P0    63W / 235W |     95MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K40m          Off  | 0000:82:00.0     Off |                    0 |
| N/A   32C    P0    63W / 235W |     73MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0    129495    C   /opt/conda/bin/python                           93MiB |
|    1    129495    C   /opt/conda/bin/python                           71MiB |
+-----------------------------------------------------------------------------+



In [6]:

    
import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
#config.gpu_options.per_process_gpu_memory_fraction = 0.01

Interactive TensorFlow Session.

t.eval() is a shortcut for calling tf.get_default_session().run(t)



In [8]:

    
sess = tf.InteractiveSession(config=config)
x = tf.Variable([1.0, 2.0])
a = tf.constant([3.0, 3.0])
x.initializer.run()

sub = tf.subtract(x, a)
print(sub.eval())

sess.close()









    



[-2. -1.]



In [ ]:



In [10]:

    
sess = tf.Session(config=config)
x = tf.Variable([1.0, 2.0])
a = tf.constant([3.0, 3.0])
sess.run(x.initializer) # różnica 1

sub = tf.subtract(x, a)
print(sess.run(sub))   # różnica 2

sess.close()









    



[-2. -1.]

context manager



In [11]:

    
with  tf.Session(config=config) as sess:
    with tf.device("/gpu:0"):
        x = tf.Variable([1.0, 2.0])
        a = tf.constant([3.0, 3.0])
        x.initializer.run()

        sub = tf.subtract(x, a)
        print(sub.eval())









    



[-2. -1.]



In [12]:

    
with  tf.Session(config=config) as sess:
    with tf.device("/gpu:0"):
        x = tf.Variable([1.0, 2.0])
        a = tf.constant([3.0, 3.0])
        sess.run(x.initializer) # różnica 1
        
        sub = tf.subtract(x, a)
        print(sess.run(sub))   # różnica 2









    



[-2. -1.]

Graphs



In [13]:

    
c = tf.constant(4.0)
c.graph is tf.get_default_graph()









    Out[13]:





True



In [18]:

    
with  tf.Session(config=config) as sess:
    with tf.device("/gpu:0"):
        x = tf.Variable([1.0, 2.0])
        a = tf.constant([3.0, 3.0],name="A")
        sub = tf.subtract(x, a)
        sess.run(x.initializer) 
        
        writer = tf.summary.FileWriter("/DATA/marcin.kostur/tf001",sess.graph)
        writer.close()
        
        sub = tf.subtract(x, a)
        print(sess.run(sub))









    



[-2. -1.]

Tensorboard

https://www.tensorflow.org/get_started/summaries_and_tensorboard

Uwaga - należy wybrać unikalny port: --port 3456



In [19]:

    
!ls -l   /DATA/marcin.kostur/tf001/*









    



-rw-r--r-- 1 marcin.kostur staff 10360 Mar 17 10:05 /DATA/marcin.kostur/tf001/events.out.tfevents.1489741503.gpu4.smcebi.us.edu.pl



In [20]:

    
!tensorboard --logdir=/DATA/marcin.kostur/tf001/ --port 4001









    



I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
Starting TensorBoard b'41' on port 4001
(You can navigate to http://155.158.128.98:4001)
^C
Traceback (most recent call last):
  File "/opt/conda/bin/tensorboard", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.5/site-packages/tensorflow/tensorboard/tensorboard.py", line 151, in main
    tb_server.serve_forever()
  File "/opt/conda/lib/python3.5/socketserver.py", line 232, in serve_forever
    ready = selector.select(poll_interval)
  File "/opt/conda/lib/python3.5/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt



In [ ]:

    
tf.



In [ ]:



In [ ]:

Constant, variables, placeholders

Constants:

tf.constant

tf.zeros
tf.zeros_like
tf.ones
tf.ones_like
tf.fill
tf.linspace
tf.range



In [21]:

    
import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
tf.reset_default_graph()



In [22]:

    
sess = tf.InteractiveSession(config=config)



In [23]:

    
a = tf.linspace(0.,11.,23)


b =  a*a
print(b.eval())









    



[   0.      0.25    1.      2.25    4.      6.25    9.     12.25   16.
   20.25   25.     30.25   36.     42.25   49.     56.25   64.     72.25
   81.     90.25  100.    110.25  121.  ]



In [24]:

    
c = tf.constant([1,2,3.])



In [25]:

    
type(a)









    Out[25]:





tensorflow.python.framework.ops.Tensor

Variables



In [28]:

    
x = tf.Variable([1,2,3.])



In [29]:

    
type(x)









    Out[29]:





tensorflow.python.ops.variables.Variable



In [34]:

    
x.initializer.run()
x.eval()









    Out[34]:





array([ 1.,  2.,  3.], dtype=float32)



In [35]:

    
(a[:3]*x)









    Out[35]:





<tf.Tensor 'mul_1:0' shape=(3,) dtype=float32>



In [36]:

    
(a[:3]*x).eval()









    Out[36]:





array([ 0.,  1.,  3.], dtype=float32)

Placeholders



In [37]:

    
y = tf.placeholder(tf.float32)



In [9]:

    
d = y*a



In [10]:

    
d.eval(feed_dict={y:1.23})









    Out[10]:





array([  0.        ,   0.61500001,   1.23000002,   1.84500003,
         2.46000004,   3.07500005,   3.69000006,   4.30500031,
         4.92000008,   5.53499985,   6.1500001 ,   6.76500034,
         7.38000011,   7.99499989,   8.61000061,   9.22500038,
         9.84000015,  10.45499992,  11.06999969,  11.68500042,
        12.30000019,  12.91499996,  13.53000069], dtype=float32)

Random numbers



In [38]:

    
n = tf.random_normal([2,2])



In [41]:

    
n.eval()









    Out[41]:





array([[-1.3262018 ,  1.82594943],
       [-2.26355791,  0.71352303]], dtype=float32)



In [24]:

    
n = tf.random_uniform([12])



In [25]:

    
n.eval()









    Out[25]:





array([ 0.40110481,  0.81999946,  0.41389871,  0.11443293,  0.18903399,
        0.08966553,  0.96734262,  0.5571183 ,  0.7045517 ,  0.1259985 ,
        0.56874275,  0.24476755], dtype=float32)



In [49]:

    
sess.close()



In [ ]:



In [ ]:



In [ ]: