我有一個神經網路，當輸入興奮時會產生一個值。我需要使用網路回傳的這個值來閾值另一個陣列。該閾值操作的結果用于計算損失函式（閾值的值事先不知道，需要通過訓練得出）。以下是 MWE
import torch
x = torch.randn(10, 1) # Say this is the output of the network (10 is my batch size)
data_array = torch.randn(10, 2) # This is the data I need to threshold
ground_truth = torch.randn(10, 2) # This is the ground truth
mse_loss = torch.nn.MSELoss() # Loss function
# Threshold
thresholded_vals = data_array * (data_array >= x) # Returns zero in all places where the value is less than the threshold, the value itself otherwise
# Compute loss and gradients
loss = mse_loss(thresholded_vals, ground_truth)
loss.backward() # Throws error here
由于閾值操作回傳一個沒有任何梯度的張量陣列，因此該backward()操作會引發錯誤。
在這種情況下如何訓練網路？
uj5u.com熱心網友回復：
您的閾值函式在閾值中不可微，因此torch不計算閾值的梯度，這就是您的示例不起作用的原因。
import torch
x = torch.randn(10, 1, requires_grad=True) # Say this is the output of the network (10 is my batch size)
data_array = torch.randn(10, 2, requires_grad=True) # This is the data I need to threshold
ground_truth = torch.randn(10, 2) # This is the ground truth
mse_loss = torch.nn.MSELoss() # Loss function
# Threshold
thresholded_vals = data_array * (data_array >= x) # Returns zero in all places where the value is less than the threshold, the value itself otherwise
# Compute loss and gradients
loss = mse_loss(thresholded_vals, ground_truth)
loss.backward() # Throws error here
print(x.grad)
print(data_array.grad)
輸出：
None #<- for the threshold x
tensor([[ 0.1088, -0.0617], #<- for the data_array
[ 0.1011, 0.0000],
[ 0.0000, 0.0000],
[-0.0000, -0.0000],
[ 0.2047, 0.0973],
[-0.0000, 0.2197],
[-0.0000, 0.0929],
[ 0.1106, 0.2579],
[ 0.0743, 0.0880],
[ 0.0000, 0.1112]])
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/442579.html