TensorFlow (https://www.tensorflow.org/) is a software library, developed by Google Brain Team within Google's Machine Learning Intelligence research organization, for the purposes of conducting machine learning and deep neural network research.
TensorFlow combines the computational algebra of compilation optimization techniques, making easy the calculation of many mathematical expressions that would be difficult to calculate, instead.
Defining, optimizing, and efficiently calculating mathematical expressions involving multi-dimensional arrays (tensors).
Programming support of deep neural networks and machine learning techniques.
Transparent use of GPU computing, automating management and optimization of the same memory and the data used. You can write the same code and run it either on CPUs or GPUs. More specifically, TensorFlow will figure out which parts of the computation should be moved to the GPU.
High scalability of computation across machines and huge data sets.
TensorFlow is available with Python and C++ support, but the Python API is better supported and much easier to learn.
In [1]:
# A simple calculation in Python
x = 1
y = x + 10
print(y)
In [2]:
import tensorflow as tf
In [3]:
# The ~same simple calculation in Tensorflow
x = tf.constant(1, name='x')
y = tf.Variable(x+10, name='y')
print(y)
Meaning: "When the variable y is computed, take the value of the constant x and add 10 to it"
In [4]:
model = tf.global_variables_initializer() # model is used by convention
In [5]:
with tf.Session() as session:
session.run(model)
print(session.run(y))
multiply, add, dividetensors) on which the operations are performed.Node: In TensorFlow, each node represents the instantion of an operation.
>= 2) and outputs >= 0.Edges: In TensorFlow, there are two types of edge:
tensors), where an output of one operation (from one node) becomes the input for another operation.A and B and a dependency edge connecting A to B. This means that B will start its operation only when the operation in A ends. Operation: This represents an abstract computation, such as adding or multiplying matrices.
Kernel: This represents the concrete implementation of that operation.
add matrix operation can have a CPU implementation and a GPU one.Session: When the client program has to establish communication with the TensorFlow runtime system, a session must be created.
As soon as the session is created for a client, an initial graph is created and is empty. It has two fundamental methods:
session.extend: To be used during a computation, requesting to add more operations (nodes) and edges (data). The execution graph is then extended accordingly.
session.run: The execution graphs are executed to get the outputs (sometimes, subgraphs are executed thousands/millions of times using run invocations).
TensorBoard is a visualization tool, devoted to analyzing Data Flow Graph and also to better understand the machine learning models.
It can view different types of statistics about the parameters and details of any part of a computer graph graphically. It often happens that a graph of computation can be very complex.
Run the TensorBoard Server:
tensorboard --logdir=/tmp/tf_logs
In [6]:
a = tf.constant(5, name="a")
b = tf.constant(45, name="b")
y = tf.Variable(a+b*2, name='y')
model = tf.global_variables_initializer()
with tf.Session() as session:
# Merge all the summaries collected in the default graph.
merged = tf.summary.merge_all()
# Then we create `SummaryWriter`.
# It will write all the summaries (in this case the execution graph)
# obtained from the code's execution into the specified path”
writer = tf.summary.FileWriter("tmp/tf_logs_simple", session.graph)
session.run(model)
print(session.run(y))
In [7]:
import numpy as np
tensor_1d = np.array([1, 2.5, 4.6, 5.75, 9.7])
tf_tensor=tf.convert_to_tensor(tensor_1d,dtype=tf.float64)
In [8]:
with tf.Session() as sess:
print(sess.run(tf_tensor))
print(sess.run(tf_tensor[0]))
print(sess.run(tf_tensor[2]))
In [9]:
tensor_2d = np.arange(16).reshape(4, 4)
print(tensor_2d)
tf_tensor = tf.placeholder(tf.float32, shape=(4, 4))
with tf.Session() as sess:
print(sess.run(tf_tensor, feed_dict={tf_tensor: tensor_2d}))
In [10]:
matrix1 = np.array([(2,2,2),(2,2,2),(2,2,2)],dtype='float32')
matrix2 = np.array([(1,1,1),(1,1,1),(1,1,1)],dtype='float32')
In [11]:
tf_mat1 = tf.constant(matrix1)
tf_mat2 = tf.constant(matrix2)
In [12]:
matrix_product = tf.matmul(tf_mat1, tf_mat2)
matrix_sum = tf.add(tf_mat1, tf_mat2)
In [13]:
matrix_det = tf.matrix_determinant(matrix2)
In [14]:
with tf.Session() as sess:
prod_res = sess.run(matrix_product)
sum_res = sess.run(matrix_sum)
det_res = sess.run(matrix_det)
In [15]:
print("matrix1*matrix2 : \n", prod_res)
print("matrix1+matrix2 : \n", sum_res)
print("det(matrix2) : \n", det_res)
In [16]:
%matplotlib inline
In [17]:
import matplotlib.image as mp_image
filename = "../imgs/keras-logo-small.jpg"
input_image = mp_image.imread(filename)
In [18]:
#dimension
print('input dim = {}'.format(input_image.ndim))
#shape
print('input shape = {}'.format(input_image.shape))
In [19]:
import matplotlib.pyplot as plt
plt.imshow(input_image)
plt.show()
In [20]:
my_image = tf.placeholder("uint8",[None,None,3])
slice = tf.slice(my_image,[10,0,0],[16,-1,-1])
In [21]:
with tf.Session() as session:
result = session.run(slice,feed_dict={my_image: input_image})
print(result.shape)
In [22]:
plt.imshow(result)
plt.show()
In [23]:
x = tf.Variable(input_image,name='x')
model = tf.global_variables_initializer()
with tf.Session() as session:
x = tf.transpose(x, perm=[1,0,2])
session.run(model)
result=session.run(x)
In [24]:
plt.imshow(result)
plt.show()
In [25]:
x = tf.placeholder(tf.float32)
y = tf.log(x)
var_grad = tf.gradients(y, x)
with tf.Session() as session:
var_grad_val = session.run(var_grad, feed_dict={x:2})
print(var_grad_val)
On a typical system, there are multiple computing devices.
In TensorFlow, the supported device types are CPU and GPU.
They are represented as strings. For example:
"/cpu:0": The CPU of your machine."/gpu:0": The GPU of your machine, if you have one."/gpu:1": The second GPU of your machine, etc.If a TensorFlow operation has both CPU and GPU implementations, the GPU devices will be given priority when the operation is assigned to a device.
For example, matmul has both CPU and GPU kernels. On a system with devices cpu:0 and gpu:0, gpu:0 will be selected to run matmul.
# Creates a graph.
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print(sess.run(c))
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 760, pci bus
id: 0000:05:00.0
b: /job:localhost/replica:0/task:0/gpu:0
a: /job:localhost/replica:0/task:0/gpu:0
MatMul: /job:localhost/replica:0/task:0/gpu:0
[[ 22. 28.]
[ 49. 64.]]# Creates a graph.
c = []
for d in ['/gpu:0', '/gpu:1']:
with tf.device(d):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3])
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2])
c.append(tf.matmul(a, b))
with tf.device('/cpu:0'):
sum = tf.add_n(c)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print sess.run(sum)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 760, pci bus
id: 0000:02:00.0
/job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: GeForce GTX 760, pci bus
id: 0000:03:00.0
Const_3: /job:localhost/replica:0/task:0/gpu:0
Const_2: /job:localhost/replica:0/task:0/gpu:0
MatMul_1: /job:localhost/replica:0/task:0/gpu:0
Const_1: /job:localhost/replica:0/task:0/gpu:1
Const: /job:localhost/replica:0/task:0/gpu:1
MatMul: /job:localhost/replica:0/task:0/gpu:1
AddN: /job:localhost/replica:0/task:0/cpu:0
[[ 44. 56.]
[ 98. 128.]]