In [1]:
import tensorflow as tf
h = tf.constant('Hello')
w = tf.constant(' world!')
hw = h + w
with tf.Session() as s:
ans = s.run(hw)
In [2]:
hw
Out[2]:
In [3]:
ans
Out[3]:
In [4]:
a = tf.constant(5)
b = tf.constant(2)
c = tf.constant(3)
d = tf.multiply(a, b)
e = tf.add(b, c)
f = tf.subtract(d, e)
with tf.Session() as s:
fetches = [a, b, c, d, e, f]
ans = s.run(fetches)
ans
Out[4]:
In [5]:
a = tf.constant(5)
b = tf.constant(2)
c = tf.constant(3)
d = a * b
e = b + c
f = d - e
with tf.Session() as s:
fetrches = [a,b,c,d,e,f]
ans = s.run(fetches)
ans
Out[5]:
In [6]:
import numpy as np
In [7]:
x_data = np.random.randn(5, 10)
w_data = np.random.randn(10, 1)
x = tf.placeholder('float32', (5, 10))
w = tf.placeholder('float32', (10, 1))
b = tf.fill((5,1), -1.0)
xwb = tf.matmul(x, w) + b
v = tf.reduce_max(xwb)
with tf.Session() as s:
ans = s.run(v, feed_dict={x: x_data, w: w_data})
ans
Out[7]:
In [8]:
n, p = 1000, 3
α = -1.0
β = np.reshape([0.5, 0.2, 1.0], (3,1))
x_data = np.random.randn(n, p)
y = α + x_data @ β + np.random.randn(n, 1)
In [ ]:
x = tf.placeholder('float32', [None, p])
y_true = tf.placeholder('float32', [None, 1])
a = tf.Variable(0.0, dtype='float32')
b = tf.Variable(np.zeros((3,1), dtype='float32'))
y_pred = a + tf.matmul(x, b)
ϵ = 0.5
loss = tf.reduce_mean(tf.square(y_true - y_pred))
optimizer = tf.train.GradientDescentOptimizer(learning_rate=ϵ)
train = optimizer.minimize(loss)
init = tf.global_variables_initializer()
In [9]:
steps = 5
with tf.Session() as session:
session.run(init)
for i in range(1, steps):
session.run(train, feed_dict={y_true: y, x: x_data})
if i% 1 == 0:
a_, b_ = session.run([a, b])
print('iter={}'.format(i))
print('a = {}'.format(a_))
print('b = {}'.format(b_.ravel()))
print()
Collection of $28 \times 28$ pixel images of hand-written digits. Objective is to classify image into one of ten possile classes. State of the art DNN methods can achieve accuracy of approxmately 99.8% accuracy.
input_data from tutorials.mnist
In [ ]:
from tensorflow.examples.tutorials.mnist import input_data
n, p = 784, 10
steps = 1000
batch_size = 100
alpha = 0.5
data_dir = '/tmp/data'
data = input_data.read_data_sets(data_dir, one_hot=True)
In [ ]:
x = tf.placeholder(tf.float32, [None, n])
W = tf.Variable(tf.zeros([n, p]))
y_true = tf.placeholder(tf.float32, [None, 10])
y_pred = tf.matmul(x, W)
loss = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits(logits=y_pred, labels=y_true))
gd = tf.train.GradientDescentOptimizer(alpha).minimize(loss)
correct_mask = tf.equal(tf.arg_max(y_pred, 1), tf.arg_max(y_true, 1))
accuracy = tf.reduce_mean(tf.cast(correct_mask, tf.float32))
In [10]:
with tf.Session() as s:
s.run(tf.global_variables_initializer())
# train
for i in range(steps):
batch_xs, batch_ys = data.train.next_batch(batch_size)
s.run(gd, feed_dict={x: batch_xs, y_true: batch_ys})
# test
ans = s.run(accuracy, feed_dict={x: data.test.images, y_true: data.test.labels})
In [12]:
ans
Out[12]:
In [13]:
! pip install --quiet tflearn
In [14]:
import tflearn
import tflearn.datasets.mnist as mnist
X, Y, validX, validY = mnist.load_data(one_hot=True)
# Building our neural network
input_layer = tflearn.input_data(shape=[None, 784])
output_layer = tflearn.fully_connected(input_layer, 10, activation='softmax')
# Optimization
sgd = tflearn.SGD(learning_rate=0.5)
net = tflearn.regression(output_layer, optimizer=sgd)
# Training
model = tflearn.DNN(net, tensorboard_verbose=3)
model.fit(X, Y, validation_set=(validX, validY), n_epoch=3)
In [15]:
model.evaluate(validX, validY)
Out[15]:
In [16]:
import edward as ed
from edward.models import Normal
In [17]:
n, p = 1000, 3
α = -1.0
β = np.reshape([0.5, 0.2, 1.0], (3,1))
# data for training
x_train = np.random.randn(n, p)
y_train = α + x_train @ β + np.random.normal(0, 1, (n,1))
y_train = y_train.ravel()
# data for testing
x_test = np.random.randn(n, p)
y_test = α + x_test @ β + np.random.normal(0, 1, (n,1))
y_test= y_test.ravel()
Given data $(x, y)$,
$$ p(w) = \mathcal{N}(w, 0, 1) \\ p(b) = \mathcal{N}(b, 0, 1) \\ p(y \mid w, b, x) = \prod_{i=1}^{n} \mathcal{N}(y_i \mid x_i^T w + b, 1) $$Note that we label the intercept $\alpha$ as the bias $b$ and the coefficeints $\beta$ as weights $w$ following neural network conventions.
In [18]:
X = tf.placeholder(tf.float32, [n, p])
w = Normal(mu=tf.zeros(p), sigma=tf.ones(p))
b = Normal(mu=tf.zeros(1), sigma=tf.ones(1))
y = Normal(mu=ed.dot(X, w) + b, sigma=tf.ones(n))
In [19]:
qw = Normal(mu=tf.Variable(tf.random_normal([p])),
sigma=tf.nn.softplus(tf.Variable(tf.random_normal([p]))))
qb = Normal(mu=tf.Variable(tf.random_normal([1])),
sigma=tf.nn.softplus(tf.Variable(tf.random_normal([1]))))
In [20]:
inference = ed.KLqp({w: qw, b: qb}, data={X: x_train, y: y_train})
inference.run()
Find the posterior predictive distrbution.
In [21]:
y_post = ed.copy(y, {w: qw, b: qb})
# This is equivalent to
# y_post = Normal(mu=ed.dot(X, qw) + qb, sigma=tf.ones(N))
Calculate evalution metrics.
In [22]:
print("Mean squared error on test data:")
print(ed.evaluate('mean_squared_error', data={X: x_test, y_post: y_test}))
print("Mean absolute error on test data:")
print(ed.evaluate('mean_absolute_error', data={X: x_test, y_post: y_test}))
Check parameters (true, prior, posterior)
In [23]:
list(zip(β, w.eval(), qw.eval()))
Out[23]:
In [24]:
α, b.eval(), qb.eval()
Out[24]:
See tutorials