Machine Learning - Mobilenet implementation
3/3/22, 5:38 PM mobilenet - Jupyter Notebook localhost:8888/notebooks/mobilenet.ipynb 1/12 CS354 Assignment 2: PyTorch 101 Before we start, please put your name in following format: FirstName Lastname Self Grading How many cells you got their outputs similar to what expected. (Same Expected output cell/#totoal number of cell) x 100 = Setup Code Before getting started we need to run some boilerplate code to set up our environment. You'll need to rerun this setup code each time you start the notebook. First, run this cell load the autoreload (https://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html?highlight=autoreload) extension. This allows us to edit .py source files, and re-import them into the notebook for a seamless editing and debugging experience. In [1]: Google Colab Setup Next we need to run a few commands to set up our environment on Google Colab. If you are running this notebook on a local machine you can skip this section. Run the following cell to mount your Google Drive. Follow the link, sign in to your Google account (the same account you used to store this notebook!) and copy the authorization code into the text box that appears below. In [2]: Now recall the path in your Google Drive where you uploaded this notebook, fill it in below. If everything is working correctly then running the folowing cell should print the filenames in the assignment directory Mounted at /content/drive/ %load_ext autoreload %autoreload 2 from google.colab import drive drive.mount('/content/drive/') https://ipython.readthedocs.io/en/stable/config/extensions/autoreload.html?highlight=autoreload 3/3/22, 5:38 PM mobilenet - Jupyter Notebook localhost:8888/notebooks/mobilenet.ipynb 2/12 In [3]: Let us start the assignment - Introduction (Highly recommeneded) Briefly read Google's 2017 paper mobilenets: Efficient revolutionary neural networks for mobile vision applications to experience Depthwise convolution and Pointwise convolution. (https://arxiv.org/pdf/1704.04861.pdf) The MobileNet model is based on depthwise separable convolutions which is a form of factorized convolutions which factorize a standard convolution into a depthwise convolution and a 1×1 convolution called a pointwise convolution. For MobileNets the depthwise convolution applies a single filter to each input channel. The pointwise convolution then applies a 1×1 convolution to combine the outputs the depthwise convolution. A standard convolution both filters and combines inputs into a new set of outputs in one step. The depthwise separable convolution splits this into two layers, a separate layer for filtering and a separate layer for combining. This factorization has the effect of drastically reducing computation and model size. Figure below shows how a standard convolution (a) is factorized into a depthwise convolution (b) and a 1 × 1 pointwise convolution (c). Figure 1. shows standard, depthwise and pointwise convolutional filters. picture Figure (1) A standard convolutional layer takes as input a × × M feature map F and produces a × × N. feature map G where is the spatial width and height of a square input feature map, M is the number of input channels (input depth), is the spatial width and height of a square output feature map and N is the number of output channel (output depth). Depthwise Separable convolution are made up of two layers: depthwise convolutions and pointwise convolutions. Depthwise convolutions to apply a single filter per each input channel (input depth). Pointwise convolution, a simple 1×1 convolution, is then used to create a linear combination of the output of the depthwise layers. MobileNets use both batchnorm and ReLU nonlinearities for both layers. ?? ?? ?? ?? ?? ?? ['images', '__pycache__', 'mobilenet_solutions.ipynb', 'mobilenet.ipynb'] import os import sys # TODO: Fill in the Google Drive path where you uploaded the assignment, it s GOOGLE_DRIVE_PATH_AFTER_MYDRIVE = 'CS354-Assignments-2022/swshah-A2' # change GOOGLE_DRIVE_PATH = os.path.join('drive', 'My Drive', GOOGLE_DRIVE_PATH_AFTER print(os.listdir(GOOGLE_DRIVE_PATH)) sys.path.append(GOOGLE_DRIVE_PATH) import time, os os.environ["TZ"] = "US/Eastern" time.tzset() https://arxiv.org/pdf/1704.04861.pdf 3/3/22, 5:38 PM mobilenet - Jupyter Notebook localhost:8888/notebooks/mobilenet.ipynb 3/12 Figure (2): Left: shows standard convolutional layer with batchnorm and ReLU. Right: Depthwise Separable convolutions with Depthwise and Pointwise layers followed by batchnorm and ReLU. In [4]: #MobileNet separable convolution In this cell, you are asked to implement both Mobilenet and Standerd block. showed in the image below. 1.10.0+cu111 #check torch version import torch print(torch.__version__) 3/3/22, 5:38 PM mobilenet - Jupyter Notebook localhost:8888/notebooks/mobilenet.ipynb 4/12 Remember that 3x3 Depthwise Conv generate same number of feature maps equal to the input channels (depth). Each of these feature maps is a result of a convolution operation over each input channel separately. The 1x1 convolution is the typical 1x1 convolution. Note that doing it this way it will kind of equivalent to the standard convolution but with less number of parameters and calculations. To illustrate this: Suppose that you have the 3x3 standard colv on WxHxD volume and the output is WxHxD. The number of parameters will be: 3x3xDxD The number of conv operations: WxHxD each at 3x3xD For the mobilenet to implement equivalent 3x3 to the standard convolutions on same input and output: The number of parameters will be: 3x3xD + 1x1xDxD The number of conv operations: WxHxD each at 3x3 + WxHxD each at 1x1xD. 3/3/22, 5:38 PM mobilenet - Jupyter Notebook localhost:8888/notebooks/mobilenet.ipynb 5/12 In [ ]: #Create DataLoader Now, you are asked to implement dataloaders that will be used during the training and testing to load the data. To implement the dataloaders you need to implement the following: We will work on CIFAR dataset. I am providing few lines to start from. transform_train and transform_test: Used to transoform and normalize your data. remember that you need to normalize the data. Do you need to use transformation in the test set ? Justify your answer. trainloader, test loader: use batch size of 128. import torch import torch.nn as nn import torch.nn.functional as F import torchvision import torchvision.transforms as transforms import matplotlib.pyplot as plt import numpy as np import torch.optim as optim class MobileNetBlock(nn.Module): def __init__(self, in_depth, out_depth, stride=1): super(MobileNetBlock, self).__init__() # add your code here ... # hint: For Depthwise convolution, use the Conv2d with groups equal t def forward(self, x): # add your code here return out class StandardBlock(nn.Module): def __init__(self, in_depth, out_depth, stride=1): super(StandardBlock, self).__init__() # add your code here ... def forward(self, x): # add your code here return out 3/3/22, 5:38 PM mobilenet - Jupyter Notebook localhost:8888/notebooks/mobilenet.ipynb 6/12 In [6]: #Create MobileNetV1 network Now it is the time to create MobileNet network that will classify CIFAR dataset. You will be asked to implement the following architecture. All the convolutions should use the MobileNetBlock except the first convolutional layer which should be standard one (We are trying to follow same steps in the MobileNet paper by starting with standard convolution). 32×32×3 ==> (Standerd convoltion) 32×32×32 ==> (MobileNet convoltion) 32×32×64 ==> (MobileNet convoltion) 16×16×128 ==> (MobileNet convoltion) 16×16×128 ==>(MobileNet convoltion) 8×8×256 ==>(MobileNet convoltion) 8×8×256 ==>(MobileNet convoltion) 4×4×512 ==>(MobileNet convoltion) 4×4×512 ==>(MobileNet convoltion) 2×2×1024 ==>(MobileNet convoltion) 2×2×1024 Mean pooling ==> 1 × 1 × 1024 Finally, it is fully connected to 10 output nodes Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz (http s://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz) to ./data/cifar-10-pyt hon.tar.gz 0%| | 0/170498071 [00:00 "change runtime type" device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") # Put the network on the GPU stNet = # add your code here for StandardNet, remeber to move the network for MobileNet = # add your code here criterion = # add your code here to define the loss function optimizer = # add your code here to define the optimizer. MobileNetoptimizer = # add your code here to define the optimizer. 3/3/22, 5:38 PM mobilenet - Jupyter Notebook localhost:8888/notebooks/mobilenet.ipynb 8/12 In [17]: #Model test Epoch: 1 Minibatch: 1 loss: 2.344 Epoch: 1 Minibatch: 101 loss: 1.870 Epoch: 1 Minibatch: 201 loss: 1.661 Epoch: 1 Minibatch: 301 loss: 1.362 Epoch: 2 Minibatch: 1 loss: 1.329 Epoch: 2 Minibatch: 101 loss: 1.426 Epoch: 2 Minibatch: 201 loss: 1.151 Epoch: 2 Minibatch: 301 loss: 1.229 Epoch: 3 Minibatch: 1 loss: 1.206 Epoch: 3 Minibatch: 101 loss: 1.020 Epoch: