First Neurons



In [1]:

    
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

Set Random Seeds for results reproducibility



In [2]:

    
np.random.seed(101)
tf.set_random_seed(101)

Data Setup

Setting Up some Random Data for Demonstration Purposes



In [3]:

    
rand_a = np.random.uniform(low = 0,
                           high = 100, 
                           size = (5,5))
rand_a









    Out[3]:





array([[51.63986277, 57.06675869,  2.84742265, 17.15216562, 68.52769817],
       [83.38968626, 30.69662197, 89.36130797, 72.15438618, 18.99389542],
       [55.42275911, 35.2131954 , 18.18924027, 78.56017619, 96.54832224],
       [23.23536618,  8.35614337, 60.35484223, 72.89927573, 27.62388285],
       [68.53063288, 51.78674742,  4.84845374, 13.78692376, 18.69674261]])



In [4]:

    
rand_b = np.random.uniform(low = 0,
                           high = 100, 
                           size = (5,1))
rand_b









    Out[4]:





array([[99.43179012],
       [52.06653967],
       [57.87895355],
       [73.48190583],
       [54.19617722]])

Placeholders



In [5]:

    
# Initialize placeholders as of type float32
a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)

Operations



In [6]:

    
add_op = a + b # tf.add(a,b)
mult_op = a * b #tf.multiply(a,b)

Running Sessions to create Graphs with Feed Dictionaries



In [7]:

    
with tf.Session() as sess:
    add_result = sess.run(add_op, 
                          feed_dict={a : rand_a,
                                     b : rand_b})
    print(add_result)
    
    print('\n')
    
    mult_result = sess.run(mult_op,
                           feed_dict={a : rand_a,
                                      b : rand_b})
    print(mult_result)









    



[[151.07166  156.49855  102.27921  116.58396  167.95949 ]
 [135.45622   82.76316  141.42784  124.22093   71.06043 ]
 [113.30171   93.09215   76.06819  136.43912  154.42728 ]
 [ 96.71727   81.83804  133.83675  146.38118  101.10579 ]
 [122.72681  105.982925  59.044632  67.9831    72.89292 ]]


[[5134.644   5674.25     283.12433 1705.4707  6813.8315 ]
 [4341.8125  1598.267   4652.734   3756.8293   988.94635]
 [3207.8113  2038.1029  1052.7742  4546.9805  5588.1157 ]
 [1707.379    614.02527 4434.989   5356.7773  2029.8555 ]
 [3714.0984  2806.6438   262.76764  747.19855 1013.292  ]]

Example Neural Network



In [8]:

    
n_features = 10
n_dense_neurons = 3



In [9]:

    
# Placeholder for x
x = tf.placeholder(tf.float32, (None, n_features))



In [10]:

    
# Variables for w and b
b = tf.Variable(tf.zeros([n_dense_neurons]))

W = tf.Variable(tf.random_normal([n_features, n_dense_neurons]))



In [11]:

    
b.get_shape()









    Out[11]:





TensorShape([Dimension(3)])



In [12]:

    
W.get_shape()









    Out[12]:





TensorShape([Dimension(10), Dimension(3)])

Operation Activation Function



In [13]:

    
xW = tf.matmul(x, W)



In [14]:

    
z = tf.add(xW,b)



In [15]:

    
# tf.nn.relu() or tf.tanh()
a = tf.sigmoid(z)

Variable Intializer!



In [16]:

    
init = tf.global_variables_initializer()



In [17]:

    
with tf.Session() as sess:
    # Run session with the initializer
    sess.run(init)
    # Result
    layer_out = sess.run(a,feed_dict={x : np.random.random([1, n_features])})



In [18]:

    
print(layer_out)









    



[[0.63504547 0.4896275  0.98692095]]

We still need to finish off this process with optimization! Let's learn how to do this next.

Full Network Example

Let's work on a regression example, we are trying to solve a very simple equation:

y = mx + b

y will be the y_labels and x is the x_data. We are trying to figure out the slope and the intercept for the line that best fits our data!

Artifical Data (Some Made Up Regression Data)



In [19]:

    
x_data = np.linspace(0, 10, 10) + np.random.uniform(-1.5, 1.5, 10)



In [20]:

    
x_data









    Out[20]:





array([-1.20856056, -0.08034641,  2.82674411,  4.50477294,  3.42312535,
        4.88227319,  7.18414126,  6.77068715,  9.4930023 ,  9.96290567])



In [21]:

    
y_label = np.linspace(0, 10, 10) + np.random.uniform(-1.5, 1.5, 10)



In [22]:

    
%matplotlib inline
plt.figure(figsize = (10, 10))
plt.plot(x_data,y_label,'*')









    Out[22]:





[<matplotlib.lines.Line2D at 0x1d273d44240>]

Variables



In [23]:

    
np.random.rand(2)









    Out[23]:





array([0.44236813, 0.87758732])



In [24]:

    
m = tf.Variable(0.442)
b = tf.Variable(0.877)

Cost Function



In [25]:

    
error = 0
for x, y in zip(x_data, y_label):
    # Predicted value
    y_hat = m * x + b  
    # Mean Squared Error (MSE)
    error += (y - y_hat) ** 2

Optimizer



In [26]:

    
optimizer = tf.train.GradientDescentOptimizer(learning_rate = 0.001)
train = optimizer.minimize(error)

Initialize Variables



In [27]:

    
init = tf.global_variables_initializer()

Create Session and Run!



In [28]:

    
epochs = 1000



In [29]:

    
with tf.Session() as sess:
    sess.run(init)
    for i in range(epochs):
        sess.run(train)
        # Print the error for every 50th epoch.
        if (i % 50 == 0):
            print("Epoch {}: {}".format(i, sess.run(error)))
    # Get the m and b values.
    final_slope , final_intercept = sess.run([m, b])









    



Epoch 0: 13.684274673461914
Epoch 50: 11.007654190063477
Epoch 100: 10.98678970336914
Epoch 150: 10.97636890411377
Epoch 200: 10.971160888671875
Epoch 250: 10.9685640335083
Epoch 300: 10.967263221740723
Epoch 350: 10.96661376953125
Epoch 400: 10.966290473937988
Epoch 450: 10.966130256652832
Epoch 500: 10.966045379638672
Epoch 550: 10.966008186340332
Epoch 600: 10.965985298156738
Epoch 650: 10.96597671508789
Epoch 700: 10.965972900390625
Epoch 750: 10.96596908569336
Epoch 800: 10.965967178344727
Epoch 850: 10.965965270996094
Epoch 900: 10.96596908569336
Epoch 950: 10.965965270996094



In [30]:

    
final_slope









    Out[30]:





0.76363987



In [31]:

    
final_intercept









    Out[31]:





0.7633267

Evaluate Results



In [32]:

    
x_test = np.linspace(-1, 11, 10)
y_pred_plot = final_slope * x_test + final_intercept

plt.plot(x_test,y_pred_plot,'r')

plt.plot(x_data,y_label,'*')









    Out[32]:





[<matplotlib.lines.Line2D at 0x1d27d6a72b0>]