Learning Objectives:
Variable
s
In [1]:
import tensorflow as tf
You can perform many typical mathematical operations on tensors (TF API). The following code creates and manipulates two vectors (1-D tensors), each having exactly six elements:
In [2]:
with tf.Graph().as_default():
# Create a six-element vector (1-D tensor).
primes = tf.constant([2, 3, 5, 7, 11, 13], dtype=tf.int32)
# Create another six-element vector. Each element in the vector will be
# initialized to 1. The first argument is the shape of the tensor (more on shapes below).
ones = tf.ones([6], dtype=tf.int32)
# Add the two vectors. The resulting tensor is a six-element vector.
primes_plus_one = tf.add(primes, ones)
# Create a session to run the default graph.
with tf.Session() as sess:
print(primes_plus_one.eval())
Shapes are used to characterize the size and number of dimensions of a tensor. The shape of a tensor is expressed as list
, with the i
th element representing the size along dimension i
. The length of the list then indicates the rank of the tensor (i.e., the number of dimensions).
For more information, see the TensorFlow documentation.
A few basic examples:
In [3]:
with tf.Graph().as_default():
# A scalar (0-D tensor).
scalar = tf.zeros([])
# A vector with 3 elements.
vector = tf.zeros([3])
# A matrix with 2 rows and 3 columns
matrix = tf.zeros([2, 3])
with tf.Session() as sess:
print("Scalar shape: {0} and the value:\n {1}".format(scalar.get_shape(), scalar.eval()))
print("Vector shape: {0} and the value:\n {1}".format(vector.get_shape(), vector.eval()))
print("Matrix shape: {0} and the value:\n {1}".format(matrix.get_shape(), matrix.eval()))
In mathematics, you can only perform element-wise operations (e.g. add and equals) on tensors of the same shape. In TensorFlow, however, you may perform operations on tensors that would traditionally have been incompatible. TensorFlow supports broadcasting (a concept borrowed from numpy), where the smaller array in an element-wise operation is enlarged to have the same shape as the larger array. For example, via broadcasting:
[6]
tensor, a size [1]
or a size []
tensor can serve as an operand.[4, 6]
tensor, any of the following sizes can serve as an operand:[1, 6]
[6]
[]
If an operation requires a size [3, 5, 6]
tensor, any of the following sizes can serve as an operand:
[1, 5, 6]
[3, 1, 6]
[3, 5, 1]
[1, 1, 1]
[5, 6]
[1, 6]
[6]
[1]
[]
NOTE: When a tensor is broadcast, its entries are conceptually copied. (They are not actually copied for performance reasons. Broadcasting was invented as a performance optimization.)
The full broadcasting ruleset is well described in the easy-to-read numpy broadcasting documentation.
The following code performs the same tensor addition as before, but using broadcasting:
In [4]:
with tf.Graph().as_default():
# Create a six-element vector (1-D tensor).
primes = tf.constant([2, 3, 5, 7, 11, 13], dtype=tf.int32)
# Create a constant scalar with value 1.
ones = tf.constant(1, dtype=tf.int32)
# Add the two tensors. The resulting tensor is a six-element vector.
primes_plus_one = tf.add(primes, ones)
with tf.Session() as sess:
print(primes_plus_one.eval())
In linear algebra, when multiplying two matrices, the number of columns of the first matrix must equal the number of rows in the second matrix.
3x4
matrix by a 4x2
matrix. This will result in a 3x2
matrix.4x2
matrix by a 3x4
matrix.
In [5]:
with tf.Graph().as_default():
# Create a matrix (2-d tensor) with 3 rows and 4 columns.
# | 5, 2, 4, 3 |
# | 5, 1, 6, -2 |
# | -1, 3, -1, -2 |
x = tf.constant([[5, 2, 4, 3], [5, 1, 6, -2], [-1, 3, -1, -2]], dtype=tf.int32)
# Create a matrix with 4 rows and 2 columns.
# | -2, 3 |
# | 2, 6 |
# | -1, 1 |
# | 0, 8 |
y = tf.constant([[-2, 3], [2, 6], [-1, 1], [0, 8]], dtype=tf.int32)
# Multiply `x` by `y`. The resulting matrix will have 3 rows and 2 columns.
matrix_mul_result = tf.matmul(x, y)
with tf.Session() as sess:
print(matrix_mul_result.eval())
In [6]:
with tf.Graph().as_default():
# Create an 8x2 matrix (2-D tensor).
x = tf.constant([[1,2], [3,4], [5,6], [7,8], [9,10], [11,12], [13, 14], [15,16]], dtype=tf.int32)
# Reshape the 8x2 matrix to 2x8 matrix.
x_reshaped_2x8 = tf.reshape(x, [2, 8])
# Reshape the 8x2 matrix to 4x4 matrix.
x_reshaped_4x4 = tf.reshape(x, [4, 4])
with tf.Session() as sess:
print("Original matrix (8x2):\n{0}".format(x.eval()))
print("Reshaped matrix (2x8):\n{0}".format(x_reshaped_2x8.eval()))
print("Reshaped matrix (4x4):\n{0}".format(x_reshaped_4x4.eval()))
tf.reshape
to change the number of dimensions (the "rank") of the tensor.
For example, you could reshape that 8x2 tensor into a 3-D 2x2x4 tensor or a 1-D 16-element tensor.
In [7]:
with tf.Graph().as_default():
# Create an 8x2 matrix (2-D tensor).
x = tf.constant([[1,2], [3,4], [5,6], [7,8], [9,10], [11,12], [13, 14], [15,16]], dtype=tf.int32)
# Reshape the 8x2 matrix to 2x2x4 (3-D tensor).
x_reshaped_2x2x4 = tf.reshape(x, [2, 2, 4])
# Reshape the 8x2 matrix to 16-element (1-D tensor).
x_reshaped_16 = tf.reshape(x, [16])
with tf.Session() as sess:
print("Original matrix (8x2):\n{0}".format(x.eval()))
print("Reshaped 3-D tensor (2x2x4):\n{0}".format(x_reshaped_2x2x4.eval()))
print("Reshaped 1-D vector (16):\n{0}".format(x_reshaped_16.eval()))
The following two vectors are incompatible for matrix multiplication:
a = tf.constant([5, 3, 2, 7, 1, 4])
b = tf.constant([4, 6, 3])
Reshape these vectors into compatible operands for matrix multiplication. Then, invoke a matrix multiplication operation on the reshaped tensors.
In [8]:
with tf.Graph().as_default():
# initialize given matrices
a = tf.constant([5, 3, 2, 7, 1, 4])
b = tf.constant([4, 6, 3])
# reshape a to 2x3 and b to 3x1
reshaped_a = tf.reshape(a, [2, 3])
reshaped_b = tf.reshape(b, [3, 1])
# result of multiplication of 2x3 and 3x1 matrices will be a 2x1 matrix
mul_a_b = tf.matmul(reshaped_a, reshaped_b)
with tf.Session() as sess:
print("Multiplication of a and b:\n{0}".format(mul_a_b.eval()))
So far, all the operations we performed were on static values (tf.constant
); calling eval()
always returned the same result. TensorFlow allows you to define Variable
objects, whose values can be changed.
When creating a variable, you can set an initial value explicitly, or you can use an initializer (like a distribution):
In [9]:
g = tf.Graph()
with g.as_default():
# Create a variable with the initial value of 3
v = tf.Variable([3])
# Create a variable of shape [1], with a random initial value,
# sampled from a normal distribution with mean 1 and standard deviation 0.35.
w = tf.Variable(tf.random_normal([1], mean=1, stddev=0.35))
In [10]:
with g.as_default():
with tf.Session() as sess:
try:
v.eval()
except tf.errors.FailedPreconditionError as e:
print("Caught expected error: {0}".format(e))
global_variables_initializer
. Note the use of Session.run()
, which is roughly equivalent to eval()
.
In [11]:
with g.as_default():
with tf.Session() as sess:
initialization = tf.global_variables_initializer()
sess.run(initialization)
# Now, variables can be accessed normally, and have values assigned to them.
print(v.eval())
print(w.eval())
In [12]:
with g.as_default():
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# These three prints will print the same value.
print(w.eval())
print(w.eval())
print(w.eval())
assign
op. Note that simply creating the assign
op will not have any effect. As with initialization, you have to run
the assignment op to update the variable value:
In [13]:
with g.as_default():
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# Print the current value of v
print(v.eval())
assignment = tf.assign(v, [7])
# The variable has not changed yet!
print(v.eval())
# Execute the assignment op.
sess.run(assignment)
# Now the variable is updated.
print(v.eval())
Create a dice simulation, which generates a 10x3
2-D tensor in which:
1
and 2
each hold one throw of one die.3
holds the sum of Columns 1
and 2
on the same row.For example, the first row might have the following values:
1
holds 4
2
holds 3
3
holds 7
You'll need to explore the TensorFlow documentation to solve this task.
In [15]:
with tf.Graph().as_default():
# Creating a 10x2 dices matrix with min value of 1 and max value of 6
dices = tf.Variable(tf.random_uniform([10, 2], minval=1, maxval=7, dtype=tf.int32))
# Calculating the sum of rows in dices matrix
dices_sum = tf.reduce_sum(dices, 1)
# Reshaping the sum matrix to 10x1 from 1x10 concat with dices matrix
dices_sum = tf.reshape(dices_sum, [10, 1])
# Concat two matrices
dices = tf.concat([dices, dices_sum], 1)
with tf.Session() as sess:
initialization = tf.global_variables_initializer()
sess.run(initialization)
print(dices.eval())