The following challenges build on what you’ve learned in this notebook. You can find solutions in the Solutions notebook.
The gradient descent method is used to find the minimum value of a function. Similar to the gradient ascent method, the gradient descent method is an iterative method: we start with an initial value of the variable and gradually get closer to the variable value that corresponds to the minimum value of the function. The step that gets us closer is the equation
$$x_{new} = x_{old} - \lambda \frac{df}{dx}$$where $\lambda$ is the step size and
$$\frac{df}{dx}$$is the result of differentiating the function. Thus, the only difference from the gradient ascent method is how we obtain the value of x_new from x_old.
Your challenge is to implement a generic program using the gradient descent algorithm to find the minimum value of a single-variable function specified as input by the user. The program should also create a graph of the function and show all the intermediate values it found before finding the minimum.
In [1]:
import numpy as np
import sympy
import matplotlib.pyplot as plt
%matplotlib inline
In [2]:
def gradient_descent(func, var, initial_value, lamb = 1e-3, eps = 1e-6):
'''
Calculates gradient descent for the function func in the variable var starting from initial_value
with a step of lamb. Stops if the difference in the calculated solution between two consecutive steps
is below eps.
Returns sol = calculated solution and steps = list of successive solutions found
'''
x = sympy.Symbol(var)
df = sympy.Derivative(f, x).doit()
xold = initial_value
xnew = xold - lamb*df.subs({x:xold})
steps = [xold]
while abs(xold - xnew) > eps:
steps.append(xnew)
xold = xnew
xnew = xold - lamb*df.subs({x:xold})
return xnew, steps
In [3]:
f = 'x**2'
sol, steps = gradient_descent(f, 'x', 2)
In [5]:
print(sol)
print(steps[0:3])
print(steps[-1])
In [7]:
def find_min():
'''
Asks input for a function, the independent variable and an initial value and applies gradient descent
to find the mininum. Plots a graph of the various steps of gradient descent
'''
func = input('Function: ')
var = input('Variable: ')
initial_value = float(input('Initial value: '))
sol, steps = gradient_descent(func, var, initial_value)
print('The minimum is {}'.format(sol))
x = sympy.Symbol(var)
t = np.linspace(float(np.min(steps))-0.2, float(np.max(steps))+0.2, 1000)
fig = plt.figure(figsize=(8, 8))
ax = plt.axes()
ax.plot(t, [sympy.sympify(func).subs({x:p}).evalf() for p in t])
ax.plot(steps, [sympy.sympify(func).subs({x:p}).evalf() for p in steps], 'ro', alpha=.5, markerfacecolor='None')
ax.axes.set_xlim(float(np.min(steps))-0.2, float(np.max(steps))+0.2)
In [8]:
find_min()
We learned that the integral
$$\int_{a}^{b} f(x) dx$$expresses the area enclosed by the function $f(x)$, with the x-axis between $x = a$ and $x = b$. The area between two curves is thus expressed as the integral
$$\int_{a}^{b} (f(x)-g(x)) dx$$where $a$ and $b$ are the points of intersection of the two curves with $a < b$. The function $f(x)$ is referred to as the upper function and $g(x)$ as the lower function. Figure 1-9 illustrates this, assuming $f(x) = x$ and $g(x) = x^2$, with a = 0 and b = 1.
Your challenge here is to write a program that will allow the user to input any two single-variable functions of $x$ and print the enclosed area between the two. The program should make it clear that the first function entered should be the upper function, and it should also ask for the values of $x$ between which to find the area.
Figure 1-9: The functions f(x) = x and g(x) = x2 enclose an area between x = 0 and x = 1.0.
In [7]:
upperf = 'x'
lowerf = 'x**2'
a = 0
b = 1
x = sympy.Symbol('x')
# Calculate the area between curves
sympy.Integral(sympy.sympify(upperf)-sympy.sympify(lowerf), (x, a, b)).doit().evalf()
Out[7]:
In [8]:
def area_between():
'''
Asks input for the upper function, the lower function, the independent variable and upper and lower bounds for the area.
Prints the area between the curves between in the interval.
'''
upperf = input('Enter the upper function: ')
lowerf = input('Enter the lower function: ')
var = input('Enter the variable: ')
a = input('Enter the lower bound: ')
b = input('Enter the upper bound: ')
x = sympy.Symbol(var)
print('The area between the two curves is', sympy.Integral(sympy.sympify(upperf)-sympy.sympify(lowerf), (x, a, b)).doit().evalf())
In [9]:
area_between()
Let’s say you just completed cycling along a road that looks roughly like Figure 1-10. Because you didn’t have an odometer, you want to know whether there’s a mathematical way to determine the distance you cycled. First, we’ll need to find an equation—even an approximation will do—that describes this path.
Figure 1-10: An approximation of the cycling path
Notice how it looks very similar to the quadratic functions you learned about in algebra? In fact, for this challenge, let’s assume that the equation is $y=f(x)=2x^2 +3x + 1$ and that you cycled from point $A (−5, 36)$ to point $B (10, 231)$. To find the length of this arc—that is, the distance you cycled—we’ll need to calculate the integral
$$\int_{a}^{b}\sqrt{1+\left(\frac{dy}{dx}\right)^2} dx,$$where $y$ describes the preceding function. Your challenge here is to write a program that will calculate the length of the arc, $AB$. You may also want to generalize your solution so that it allows you to find the length of the arc between any two points for any arbitrary function, $f(x)$.
In [10]:
x = sympy.Symbol('x')
f = 2*x**2 + 3*x + 1
a = -5.36
b = 10.231
# Calculate the integrand
g = sympy.sqrt(1 + (sympy.Derivative(f, x).doit())**2)
# Calculate the length of the4 curve
sympy.Integral(g, (x, a, b)).doit().evalf()
Out[10]:
In [11]:
def arc_length():
'''
Asks input for the curve function, the independent variable and start and end points for the path.
Prints the length of the arc of curve between start and end point.
'''
f = input('Enter a function: ')
var = input('Enter the variable: ')
a = input('Enter the starting point: ')
b = input('Enter the end point: ')
x = sympy.Symbol(var)
g = sympy.sqrt(1 + (sympy.Derivative(sympy.sympify(f), x).doit())**2)
print('The length is', sympy.Integral(g, (x, a, b)).doit().evalf())
In [12]:
arc_length()