

In [1]:

    
import torch
import torch.nn as nn
from torch.autograd import Variable
from torch import optim
import torch.nn.functional as F

Basic Operations

Tensor Creation

PyTorch has API designed in Numpy Style for making the life of beginners easy



In [2]:

    
torch.rand(5,3)









    Out[2]:





 0.1118  0.0959  0.9752
 0.8469  0.3039  0.6482
 0.9310  0.7859  0.1154
 0.5732  0.0528  0.9782
 0.7521  0.1328  0.8212
[torch.FloatTensor of size 5x3]



In [3]:

    
torch.Tensor(5, 3)









    Out[3]:





1.00000e-37 *
 -2.7429  0.0005  4.6774
  0.0000  0.0000  0.0000
  0.0000  0.0000  0.0000
  0.0000  0.0000  0.0000
  0.0000  0.0000  0.0000
[torch.FloatTensor of size 5x3]



In [4]:

    
x = torch.rand(5, 3)
y = torch.rand(5, 3)

Numpy Like Ops



In [5]:

    
x.size()









    Out[5]:





torch.Size([5, 3])



In [6]:

    
x.size()[0]









    Out[6]:





5



In [7]:

    
x + y









    Out[7]:





 1.4569  0.7753  1.1837
 1.5705  0.1422  0.7490
 0.6570  0.2623  0.7471
 1.4993  1.3699  0.6686
 0.3614  1.2243  1.8354
[torch.FloatTensor of size 5x3]



In [8]:

    
y[:, :2]









    Out[8]:





 0.5091  0.4318
 0.9586  0.1386
 0.5518  0.1113
 0.8788  0.9859
 0.0940  0.4532
[torch.FloatTensor of size 5x2]

Numpy Bridge

If you are coming from numpy background, you can still code in numpy and where you think you need a support of a framework or GPU, migrate your data to PyTorch and take it back when you need it there



In [9]:

    
x.numpy()









    Out[9]:





array([[ 0.94783938,  0.34351015,  0.41133446],
       [ 0.61181039,  0.00358355,  0.01855606],
       [ 0.10520189,  0.1510013 ,  0.05143904],
       [ 0.62046349,  0.38401294,  0.14046863],
       [ 0.26745191,  0.77107751,  0.99292314]], dtype=float32)



In [10]:

    
numpy_x = x.numpy()
type(numpy_x)









    Out[10]:





numpy.ndarray



In [11]:

    
torch_x = torch.from_numpy(numpy_x)
print(type(torch_x), type(x))









    



<class 'torch.FloatTensor'> <class 'torch.FloatTensor'>

On GPU



In [12]:

    
x + y









    Out[12]:





 1.4569  0.7753  1.1837
 1.5705  0.1422  0.7490
 0.6570  0.2623  0.7471
 1.4993  1.3699  0.6686
 0.3614  1.2243  1.8354
[torch.FloatTensor of size 5x3]



In [13]:

    
x = x.cuda()
y = y.cuda()
x + y









    Out[13]:





 1.4569  0.7753  1.1837
 1.5705  0.1422  0.7490
 0.6570  0.2623  0.7471
 1.4993  1.3699  0.6686
 0.3614  1.2243  1.8354
[torch.cuda.FloatTensor of size 5x3 (GPU 0)]

Create GPU tensors instead of moving to GPU



In [14]:

    
x = torch.cuda.FloatTensor(5,3)

Variable, Function and AutoGrad

These are the core components of Pytorch. If you get em right, you are PRO, from day 1.

Variables keep the track of your graph
Functions are the OPS which communicate to other variables and keep your graph acyclic
Backward() function (or autograd) do the backward pass following the history saved in the variables and creates the grad for you. Don't worry, PyTorch follows python philosophy, It won't update your parameters implicitly. It will store the gradients in .grad variable.



In [15]:

    
x = torch.Tensor([12.3])



In [16]:

    
x = Variable(x, requires_grad=True)
print('x.grad: ', x.grad)
print('x.data: ', x.data)









    



x.grad:  None
x.data:  
 12.3000
[torch.FloatTensor of size 1]



In [17]:

    
y = (x ** x) * (x - 2)
z = F.tanh(y)



In [18]:

    
z.backward()



In [19]:

    
print('x.grad: ', x.grad)
print('x.data: ', x.data)









    



x.grad:  Variable containing:
 0
[torch.FloatTensor of size 1]

x.data:  
 12.3000
[torch.FloatTensor of size 1]

Creator

Creators keep the track of your graph for autograd to do backpropagation



In [20]:

    
print('creator of z: ', z.creator)
print('creator of creator of z: ', z.creator.previous_functions[0][0])
print('creator of creator of creator of z: ', z.creator.previous_functions[0][0].previous_functions[0][0])









    



creator of z:  <torch.nn._functions.thnn.auto.Tanh object at 0x7fa01a8809e8>
creator of creator of z:  <torch.autograd._functions.basic_ops.Mul object at 0x7fa01a880c88>
creator of creator of creator of z:  <torch.autograd._functions.basic_ops.Pow object at 0x7fa07b22dba8>



In [ ]: