Automatic differentiation with autograd



In [1]:

    
import mxnet as mx



In [2]:

    
x = mx.nd.array([[1, 2], [3, 4]])

If we want to compute a function f with respect to x, we need a place to store it. Communicating that we want plan to store a gradient is done by invoking 'attack_grad()' function.



In [3]:

    
x.attach_grad()

MXNet will build the graph only when it is explicitly said so in order to minimize the computational requirements. Recoding the gradients it done as follow:



In [4]:

    
with mx.autograd.record():
    y = x * 2
    z = y * x 
    # Thus z = 2 * x * x
    v = y / x



In [5]:

    
z.backward(retain_graph=True)



In [6]:

    
print(z)









    



[[ 2.  8.]
 [18. 32.]]
<NDArray 2x2 @cpu(0)>



In [7]:

    
print(x.grad)









    



[[ 4.  8.]
 [12. 16.]]
<NDArray 2x2 @cpu(0)>



In [8]:

    
print(y)









    



[[2. 4.]
 [6. 8.]]
<NDArray 2x2 @cpu(0)>



In [9]:

    
y.backward(retain_graph=True)



In [10]:

    
x.grad









    Out[10]:





[[2. 2.]
 [2. 2.]]
<NDArray 2x2 @cpu(0)>



In [11]:

    
v.backward(retain_graph=True)



In [12]:

    
x.grad









    Out[12]:





[[0. 0.]
 [0. 0.]]
<NDArray 2x2 @cpu(0)>