notebook.community

Edit and run

Introduction to the TensorFlow computation graph

When I started with deep learning, one of the concepts that took me quite a while to wrap my head around was the use of a computation graph within code. Furthermore, that the code in tensorflow is usally packed in objects does not really help grapsing this concept. That is I wanted to give a simple explanation of the TensorFlow computation graph, what it is and how to use it. The explanation focuses on the link between the intuition, the math and the code.

Structure of notebook

  • Intuition of a computation graph
  • Why is it useful for neural networks
  • Feeding values
  • Visualize them.

Todo

  • example
  • picture
  • graph
  • feed_dict

Resources





In [2]:



    


import tensorflow as tf
assert tf.__version__=="1.2.0" # we want that version

Intuition behind a computation graph

  • A computation graph is a visual way of representing calculations.
  • A graph consists of nodes and edges.
  • The nodes represent operations, the edges represent variables.

Example:

  • Math calculation is: $c=a*b$
  • Three variables, $a,b,c$
  • One operation $*$

Visualization:

In tensorflow

Two steps:

  • need to define the graph
  • need to execute operations on the graph

Explain difference between

  • Trainable variables
  • Non-Trainable Variables
  • Placeholders
  • Constants




In [11]:



    


# define calculation graph
a = tf.Variable(1.0, dtype=tf.float32, name="a", trainable=True) # 
b = tf.Variable(2.0, dtype=tf.float32, name="b", trainable=False) # scalar
c = tf.placeholder(shape=(), dtype=tf.float32, name="c") # scalar

r_1 = tf.multiply(a, b, name="a_times_b")
r_2 = tf.add(r_1, c, name="r_1_plus_c")

# run operations on graph
session = tf.Session()
    
variable_assignment = {
    a:1.0, # assign variable a of calulation graph to 1
    b:2.0, # assign variable b of calulation graph to 2.0
}

r_2_result = session.run(
    fetches=[# which operations of the calculation graph to fetch
        r_2
    ], 
    feed_dict=variable_assignment # which variables to use for this assigmnent
)

print(r_2_result)



    


















    






---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
~/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1138     try:
-> 1139       return fn(*args)
   1140     except errors.OpError as e:

~/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
   1120                                  feed_dict, fetch_list, target_list,
-> 1121                                  status, run_metadata)
   1122 

/usr/lib/python3.5/contextlib.py in __exit__(self, type, value, traceback)
     65             try:
---> 66                 next(self.gen)
     67             except StopIteration:

~/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py in raise_exception_on_not_ok_status()
    465           compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 466           pywrap_tensorflow.TF_GetCode(status))
    467   finally:

InvalidArgumentError: You must feed a value for placeholder tensor 'c' with dtype float
	 [[Node: c = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-11-ec0d1384f797> in <module>()
     19             r_2
     20         ], 
---> 21         feed_dict=variable_assignment # which variables to use for this assigmnent
     22     )
     23 

~/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    787     try:
    788       result = self._run(None, fetches, feed_dict, options_ptr,
--> 789                          run_metadata_ptr)
    790       if run_metadata:
    791         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    995     if final_fetches or final_targets:
    996       results = self._do_run(handle, final_targets, final_fetches,
--> 997                              feed_dict_string, options, run_metadata)
    998     else:
    999       results = []

~/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
   1130     if handle is None:
   1131       return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
-> 1132                            target_list, options, run_metadata)
   1133     else:
   1134       return self._do_call(_prun_fn, self._session, handle, feed_dict,

~/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1150         except KeyError:
   1151           pass
-> 1152       raise type(e)(node_def, op, message)
   1153 
   1154   def _extend_graph(self):

InvalidArgumentError: You must feed a value for placeholder tensor 'c' with dtype float
	 [[Node: c = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Caused by op 'c', defined at:
  File "/usr/lib/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/ipykernel_launcher.py", line 16, in <module>
    app.launch_new_instance()
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/ipykernel/kernelapp.py", line 477, in start
    ioloop.IOLoop.instance().start()
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/zmq/eventloop/ioloop.py", line 177, in start
    super(ZMQIOLoop, self).start()
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tornado/ioloop.py", line 888, in start
    handler_func(fd_obj, events)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tornado/stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tornado/stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell
    handler(stream, idents, msg)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
    user_expressions, allow_stdin)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2698, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2802, in run_ast_nodes
    if self.run_code(code, result):
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-11-ec0d1384f797>", line 4, in <module>
    c = tf.placeholder(shape=(), dtype=tf.float32, name="c") # scalar
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1530, in placeholder
    return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1954, in _placeholder
    name=name)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/sthaler/Repositories/tf-spikes/vampprior/.env/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'c' with dtype float
	 [[Node: c = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Notation

  • use squares for operations
  • use circles for data

Content source: stefanthaler/tf-spikes

Similar notebooks:

notebook.community | gallery | about