Gautham Santhosh

PyTorch 101

Torch is a Deep Learning framework which was written in Lua Programming Language. Inspired by this amazing library a couple of python ethusisats wrote PyTorch based on its principles. It's supported by Facebook while TensorFlow is by Google.

PyTorch was rewritten and tailored to be fast and feel native in Python. PyTorch is not a Python binding into a monolothic C++ framework.

Benefits

Impertive Programming

This key feature allows you to change computation as you type it.

Computation is defined runtime

Like most python code,while TensorFlow other framework requires the user to define the model first and then compute it. TensorFlow was made for Engineers while PyTorch was for researchrs.

Meaning PyTorch is more flexible with the prize of a little drop in efficency.

{% image https://media.giphy.com/media/fQZX2aoRC1Tqw/giphy.gif %}

Graphs are created on the fly

alt text

Since computaion graphs are built at run time they are more efficent for using RNN's. Which hence makes debugging very easy.

Examples

Basic example

#  1. Basic autograd example 1

# Create tensors.
a = Variable(torch.Tensor([1,2]), requires_grad=True)
b = Variable(torch.Tensor([0,2]), requires_grad=True)

# Build a computational graph.
x = a @ b 

print(x)

Prints 4 : Taking a dot product of x with w using the new Python 3.5 syntax

Once you are done, all you need to do is call #backward() on the result. This will calculate the gradients and you will be able to access them for Variables that were created with requires_grad = True.

# Compute gradients.
x.backward()

# Print out the gradients.
print(a.grad)

Prints 0 2

Single Layer Neural Network


# Neural Network Model (1 hidden layer)
class Net(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size) 
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, num_classes)  
    
    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        return out
    
net = Net(input_size, hidden_size, num_classes)

    
# Loss and Optimizer
criterion = nn.CrossEntropyLoss()  
optimizer = torch.optim.Adam(net.parameters(), lr=learning_rate)

Looks pretty simple right.

Tensor Flow vs Pytorch

PyTorch is still a young framework which is getting momentum fast. You may find it a good fit if you:

  • Do research or your production non-functional requirements are not very demanding
  • Want better development and debugging experience
  • Love all things Pythonic

TensorFlow is a good option if you:

  • Develop models for production
  • Develop models which need to be deployed on mobile platforms
  • Want good community support and comprehensive documentation
  • Want rich learning resources in various forms (TensorFlow has an an entire MOOC)
  • Want or need to use Tensorboard
  • Need to use large-scale distributed model training

But if you are still very new to Deeplearning and would just like to know what it means, keras is the way to go.

Learn More

If you are a developer and wants to learn pytorch and deeplearning, i suggest this course Fast.ai

← Nybles →