Write a program using tensorflow to build a stochastic gradient descent model for linear regression.

Previous Work

Co-efficients $\theta_1$ & $\theta_0$ for

are calculated for given values from previous assignment.

Stochastic gradient descent requires two parameters:

Learning Rate: Used to limit the amount each coefficient is corrected each time it is updated.

Epochs: The number of times to run through the training data while updating the coefficients.

There are 3 loops we need to perform in the function:

a. Loop over each epoch.
b. Loop over each row in the training data for an epoch.
c. Loop over each coefficient and update it for a row in an epoch.

Part 1

// for Trial - dataset = [[1, 1], [2, 3], [4, 3], [3, 2], [5, 5]]

1. Start with 0 (zero ) value for m & c .
2. Using cost function $$J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2$$ where $(h_\theta(x^{(i)})$ is prediction for present iteration

$$\theta_1(t+1) = \theta_1(t) - learning\_rate * cost(t) * x(t)$$
$$\theta_0(t+1) = \theta_0(t) - learning\_rate * cost(t) $$

1. Find the values of $\theta_1$ & $\theta_0$ by updating the values with learning rate of 0.005, 0.0005 etc and epoch value 100,500,1000
2. Stop the iterations when error falls below threshold

Part 2

1. Display the graph of decreasing cost function wrt learning rate and epoch
2. Compare time complexity of tensorflow api and user defined function
3. Start with random values of $\theta_1$ & $\theta_0$ . Analyse best approach for initial values of $\theta_1$ & $\theta_0$.

Additional Cost Function :

A1 = Root Mean Square Error A2 = B1 = B2 = C1 = C2 = D1 = D2 = Sum of Square Error