1-3 李宏毅2021春季機器學習教程-Google Colab教學-助教許湛然介紹了Colab的使用，這篇文章是助教許湛然關于PyTorch框架的簡要講解，

更多操作查看: https://pytorch.org/docs/stable/tensors.html

目錄

Prerequisites-準備作業

What is PyTorch?-什么是pytorch?

PyTorch v.s. TensorFlow

Overview of the DNN Training Procedure

Tensor

Tensor -- Data Type

Tensor -- Shape of Tensors

Tensor -- Constructor

??Tensor -- Operators

Tensor -- PyTorch v.s. NumPy

Tensor -- Device

Tensor -- Device(GPU)?

How to Calculate Gradient?

Load Data

Dataset & Dataloader

Define Neural Network

torch.nn -- Neural Network Layers

??torch.nn -- Activation Functions

Loss Function

torch.nn -- Loss Functions

torch.nn -- Build your own neural network

Optimizer

torch.optim

Neural Network Training

前期準備

多次epoch

Neural Network Evaluation (Validation Set)

Neural Network Evaluation (Testing Set)

Save/Load a Neural Network

More About PyTorch

Reference

Prerequisites-準備作業

?

熟悉python3的有關知識：if-else, loop等；熟悉numpy，了解陣列等操作，

What is PyTorch?-什么是pytorch?

?

• 開源的機器學習框架
• 提供兩個高水平特征的python庫

PyTorch v.s. TensorFlow

?

Overview of the DNN Training Procedure

?

Tensor

?

Tensor -- Data Type

?

ref: torch.Tensor — PyTorch 1.9.1 documentation

Tensor -- Shape of Tensors

?

Tensor -- Constructor

?

Tensor -- Operators

squeeze()：主要對資料的維度的進行壓縮，去掉維數為1的維度，例如一行或者一列這種，維度為（1,3）的一行三列去掉第一個維數為一的維度之后就變成（3）行，有三種形式：①squeeze(a)就是將a中所有為1的維度刪掉，不為1的維度沒有影響，②a.squeeze(N) 就是去掉a中指定的維數為一的維度，③還有一種形式就是b=torch.squeeze(a，N)，去掉a中指定的定的維數為一的維度，

unsqueeze()：主要對資料維度進行填充，給指定位置加上維數為一的維度，例如有個三行的資料（3），在0的位置加了一維就變成一行三列（1,3），

transpose()：交換矩陣的兩個維度，transpose(dim0, dim1) → Tensor，其和torch.transpose()函式作用一樣，

cat()：拼接函式，在給定維度上對輸入的張量序列seq 進行連接操作，torch.cat()可以看做 torch.split() 和 torch.chunk()的反操作，

?

?

?

?

?

?

more operators: torch.Tensor — PyTorch 1.9.1 documentation

Tensor -- PyTorch v.s. NumPy

?

?

ref: https://github.com/wkentaro/pytorch-for-numpy-users

Tensor -- Device

?

Tensor -- Device(GPU)???????

上圖的鏈接如下：

https://towardsdatascience.com/what-is-a-gpu-and-do-you-need-one-in-deep-learning-718b9597aa0d

How to Calculate Gradient?

?

Load Data

?

Dataset & Dataloader

?

注意shuffle引數，在train時為True，在test時為False：?

Define Neural Network

?

torch.nn -- Neural Network Layers

?

矩陣與向量表示：?

代碼如下：

?torch.nn -- Activation Functions

?

Loss Function

?

torch.nn -- Loss Functions

• Mean Squared Error (for linear regression)回歸

nn.MSELoss()

• Cross Entropy (for classification)分類

nn.CrossEntropyLoss()

torch.nn -- Build your own neural network

?

代碼：

import torch.nn as nn 
class MyModel(nn.Module): 
    def __init__(self): 
        super(MyModel, self).__init__() 
        self.net = nn.Sequential( 
            nn.Linear(10, 32), 
            nn.Sigmoid(), 
            nn.Linear(32, 1) 
        ) 
    def forward(self, x): 
        return self.net(x)

Optimizer

?

torch.optim

?

代碼：

torch.optim.SGD(params,lr,momentum = 0)

Neural Network Training

?

前期準備

?

代碼如下：

dataset = MyDataset(file) 
tr_set = DataLoader(dataset, 16, shuffle=True) 
model = MyModel().to(device) 
criterion = nn.MSELoss() 
optimizer = torch.optim.SGD(model.parameters(), 0.1)

多次epoch

?

代碼如下：

for epoch in range(n_epochs): 
    model.train() 
    for x, y in tr_set: 
        optimizer.zero_grad() 
        x, y = x.to(device), y.to(device) 
        pred = model(x) 
        loss = criterion(pred, y) 
        loss.backward() 
        optimizer.step()

Neural Network Evaluation (Validation Set)

?

代碼如下：

model.eval() 
total_loss = 0 
for x, y in dv_set: 
    x, y = x.to(device), y.to(device) 
    with torch.no_grad():#不希望進行梯度計算 
        pred = model(x) 
        loss = criterion(pred, y) 
        total_loss += loss.cpu().item() * len(x) 
        avg_loss = total_loss / len(dv_set.dataset)

Neural Network Evaluation (Testing Set)

?

代碼如下：

model.eval() 
preds = [] 
for x in tt_set: 
    x = x.to(device) 
    with torch.no_grad(): 
    pred = model(x) 
    preds.append(pred.cpu())

Save/Load a Neural Network

?

代碼如下：

#Save 
torch.save(model.state_dict(), path) 
# Load 
ckpt = torch.load(path) 
model.load_state_dict(ckpt)

More About PyTorch

• torchaudio

speech/audio processing

• torchtext

natural language processing

• torchvision

computer vision

• skorch

scikit-learn + pyTorch

• Useful github repositories using PyTorch
  • Huggingface Transformers (transformer models: BERT, GPT, ...)
  • Fairseq (sequence modeling for NLP & speech)
  • ESPnet (speech recognition, translation, synthesis, ...)
  • Many implementation of papers
  • ...

Reference

PyTorch

GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration

GitHub - wkentaro/pytorch-for-numpy-users: PyTorch for Numpy users. https://pytorch-for-numpy-users.wkentaro.com

Pytorch vs. TensorFlow: What You Need to Know | Udacity

https://www.tensorflow.org/

NumPy

說明：記錄學習筆記，如果錯誤歡迎指正！寫文章不易，轉載請聯系我，