A First Shot at Deep Learning with PyTorch
Create a hello world for deep learning using PyTorch.
About
In this notebook, we are going to take a baby step into the world of deep learning using PyTorch. There are a ton of notebooks out there that teach you the fundamentals of deep learning and PyTorch, so here the idea is to give you some basic introduction to deep learning and PyTorch at a very high level. Therefore, this notebook is targeting beginners but it can also serve as a review for more experienced developers.
After completion of this notebook, you are expected to know the basic components of training a basic neural network with PyTorch. I have also left a couple of exercises towards the end with the intention of encouraging more research and practise of your deep learning skills.
Author: Elvis Saravia - Twitter | LinkedIn
Complete Code Walkthrough: Blog post
!pip3 install torch torchvision
## The usual imports
import torch
import torch.nn as nn
## print out the pytorch version used
print(torch.__version__)
The Neural Network
Before building and training a neural network the first step is to process and prepare the data. In this notebook, we are going to use syntethic data (i.e., fake data) so we won't be using any real world data.
For the sake of simplicity, we are going to use the following input and output pairs converted to tensors, which is how data is typically represented in the world of deep learning. The x values represent the input of dimension (6,1) and the y values represent the output of similar dimension. The example is taken from this tutorial.
The objective of the neural network model that we are going to build and train is to automatically learn patterns that better characterize the relationship between the x and y values. Essentially, the model learns the relationship that exists between inputs and outputs which can then be used to predict the corresponding y value for any given input x.
## our data in tensor form
x = torch.tensor([[-1.0], [0.0], [1.0], [2.0], [3.0], [4.0]], dtype=torch.float)
y = torch.tensor([[-3.0], [-1.0], [1.0], [3.0], [5.0], [7.0]], dtype=torch.float)
## print size of the input tensor
x.size()
The Neural Network Components
As said earlier, we are going to first define and build out the components of our neural network before training the model.
Model
Typically, when building a neural network model, we define the layers and weights which form the basic components of the model. Below we show an example of how to define a hidden layer named layer1 with size (1, 1). For the purpose of this tutorial, we won't explicitly define the weights and allow the built-in functions provided by PyTorch to handle that part for us. By the way, the nn.Linear(...) function applies a linear transformation ($y = xA^T + b$) to the data that was provided as its input. We ignore the bias for now by setting bias=False.
## Neural network with 1 hidden layer
layer1 = nn.Linear(1,1, bias=False)
model = nn.Sequential(layer1)
## loss function
criterion = nn.MSELoss()
## optimizer algorithm
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
Training the Neural Network Model
We have all the components we need to train our model. Below is the code used to train our model.
In simple terms, we train the model by feeding it the input and output pairs for a couple of rounds (i.e., epoch). After a series of forward and backward steps, the model somewhat learns the relationship between x and y values. This is notable by the decrease in the computed loss. For a more detailed explanation of this code check out this tutorial.
## training
for i in range(150):
model = model.train()
## forward
output = model(x)
loss = criterion(output, y)
optimizer.zero_grad()
## backward + update model params
loss.backward()
optimizer.step()
model.eval()
print('Epoch: %d | Loss: %.4f' %(i, loss.detach().item()))
Testing the Model
After training the model we have the ability to test the model predictive capability by passing it an input. Below is a simple example of how you could achieve this with our model. The result we obtained aligns with the results obtained in this notebook, which inspired this entire tutorial.
## test the model
sample = torch.tensor([10.0], dtype=torch.float)
predicted = model(sample)
print(predicted.detach().item())
Final Words
Congratulations! In this tutorial you learned how to train a simple neural network using PyTorch. You also learned about the basic components that make up a neural network model such as the linear transformation layer, optimizer, and loss function. We then trained the model and tested its predictive capabilities. You are well on your way to become more knowledgeable about deep learning and PyTorch. I have provided a bunch of references below if you are interested in practising and learning more.
I would like to thank Laurence Moroney for his excellent tutorial which I used as an inspiration for this tutorial.
Exercises
- Add more examples in the input and output tensors. In addition, try to change the dimensions of the data, say by adding an extra value in each array. What needs to be changed to successfully train the network with the new data?
- The model converged really fast, which means it learned the relationship between x and y values after a couple of iterations. Do you think it makes sense to continue training? How would you automate the process of stopping the training after the model loss doesn't subtantially change?
- In our example, we used a single hidden layer. Try to take a look at the PyTorch documentation to figure out what you need to do to get a model with more layers. What happens if you add more hidden layers?
- We did not discuss the learning rate (
lr-0.001) and the optimizer in great detail. Check out the PyTorch documentation to learn more about what other optimizers you can use.