深度學習—從入門到放棄(一)pytorch

Tensor

類似于numpy的array，pandas的dataframe；在pytorch里的資料結構是tensor，即張量

tensor簡單操作

1.Flatten and reshape

###
Original z: 
 tensor([[ 0,  1],
        [ 2,  3],
        [ 4,  5],
        [ 6,  7],
        [ 8,  9],
        [10, 11]])
Flattened z: 
 tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
Reshaped (3x4) z: 
 tensor([[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]])
###

2.Squeezing tensors

當我們處理類似于x.shape=[1,10]或[256，1,3]這樣的高維資料時，單純輸入下x[0]可能無法輸出對應的點資料，所以我們需要用torch.squeeze()提取某一個具體維度

x = torch.randn(1, 10)
x = x.squeeze(0)#取到了第一行的x的資料
print(x.shape)
print(f"x[0]: {x[0]}")
###
torch.Size([10])
x[0]: -0.7390837073326111
###

3.permute

torch.permute()可以用來重新排列維度之間的順序

x = torch.rand(3, 48, 64)
x = x.permute(1, 2, 0)
###
torch.Size([48, 64, 3])
###

4.Concatenation

tensor和tensor之間按維度的拼接

x = torch.arange(12, dtype=torch.float32).reshape((3, 4))
y = torch.tensor([[2.0, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])
#行連接
cat_rows = torch.cat((x, y), dim=0)
#列連接
cat_cols = torch.cat((x, y), dim=1)
###
行連接: shape[6, 4] 
 tensor([[ 0.,  1.,  2.,  3.],
        [ 4.,  5.,  6.,  7.],
        [ 8.,  9., 10., 11.],
        [ 2.,  1.,  4.,  3.],
        [ 1.,  2.,  3.,  4.],
        [ 4.,  3.,  2.,  1.]])
列連接: shape[3, 8]  
 tensor([[ 0.,  1.,  2.,  3.,  2.,  1.,  4.,  3.],
        [ 4.,  5.,  6.,  7.,  1.,  2.,  3.,  4.],
        [ 8.,  9., 10., 11.,  4.,  3.,  2.,  1.]])
###

GPU vs CPU

在處理大規模與高速資料時，CPU很難滿足需要，而深度學習往往就需要處理大規模的資料，所以我們需要靈活的選擇CPU或GPU

def set_device():
  device = "cuda" if torch.cuda.is_available() else "cpu"
  if device != "cuda":
    print("GPU is not enabled in this notebook. \n"
          "If you want to enable it, in the menu under `Runtime` -> \n"
          "`Hardware accelerator.` and select `GPU` from the dropdown menu")
  else:
    print("GPU is enabled in this notebook. \n"
          "If you want to disable it, in the menu under `Runtime` -> \n"
          "`Hardware accelerator.` and select `None` from the dropdown menu")

  return device
  
 DEVICE = set_device()

簡單神經網路

Pytorch有一個 nn.Module類專門用于構建深度學習網路，我們需要從 nn.Module中繼承并實作一些重要的方法：

1. init
   在該__init__方法中，我們需要定義網路的結構，在這里，我們將指定網路由哪些層組成，將使用哪些激活函式等，
2. forward
   所有神經網路模塊都需要實作該forward方法，它指定了當資料通過網路時網路需要進行的計算，
3. predict
   這不是神經網路模塊的強制性方法，但可用于快速從網路中獲得最可能的標簽
4. train
   這也不是強制性方法，但可用于訓練網路中的引數

# Inherit from nn.Module - the base class for neural network modules provided by Pytorch
class NaiveNet(nn.Module):

  # Define the structure of your network
  def __init__(self):
    super(NaiveNet, self).__init__()

    # The network is defined as a sequence of operations
    self.layers = nn.Sequential(
        nn.Linear(2, 16),  # Transformation from the input to the hidden layer
        nn.ReLU(),         # Activation function (ReLU) is a non-linearity which is widely used because it reduces computation. The function returns 0 if it receives any
                           # negative input, but for any positive value x, it returns that value back.
        nn.Linear(16, 2),  # Transformation from the hidden to the output layer
    )

  # Specify the computations performed on the data
  def forward(self, x):
    # Pass the data through the layers
    return self.layers(x)

  # Choose the most likely label predicted by the network
  def predict(self, x):
    # Pass the data through the networks
    output = self.forward(x)

    # Choose the label with the highest score
    return torch.argmax(output, 1)

 # Implement the train function given a training dataset X and correcsponding labels y
def train(model, X, y):
  # The Cross Entropy Loss is suitable for classification problems
  loss_function = nn.CrossEntropyLoss()

  # Create an optimizer (Stochastic Gradient Descent) that will be used to train the network
  learning_rate = 1e-2
  optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

  # Number of epochs
  epochs = 15000

  # List of losses for visualization
  losses = []

  for i in range(epochs):
    # Pass the data through the network and compute the loss
    # We'll use the whole dataset during the training instead of using batches
    # in to order to keep the code simple for now.
    y_logits = model.forward(X)
    loss = loss_function(y_logits, y)

    # Clear the previous gradients and compute the new ones
    optimizer.zero_grad()
    loss.backward()

    # Adapt the weights of the network
    optimizer.step()

    # Store the loss
    losses.append(loss.item())

    # Print the results at every 1000th epoch
    if i % 1000 == 0:
      print(f"Epoch {i} loss is {loss.item()}")

      plot_decision_boundary(model, X, y, DEVICE)
      plt.savefig('frames/{:05d}.png'.format(i))

  return losses


# Create a new network instance a train it
model = NaiveNet().to(DEVICE)
losses = train(model, X, y)

以上為一個簡單神經網路應用于分類的實體，整個網路的結構如下：1 個大小為 2 的輸入層+1 個大小為 16 的隱藏層（ReLU為激活函式）+1 個大小為 2 的輸出層

NaiveNet(
(layers): Sequential(
(0): Linear(in_features=2, out_features=16, bias=True)
(1): ReLU()
(2): Linear(in_features=16, out_features=2, bias=True)
)
)

今天大家只需對神經網路的基本結構有一個了解，明天將會系統學習簡單線性神經網路的詳細結構，也歡迎大家關注公眾號奇趣多多一塊交流！

深度學習—從入門到放棄（二）簡單線性神經網路