GradientTape：獲取nan的梯度

我正在嘗試計算 tensorflow 中的梯度，但是回傳None. 我已經將型別調整為tensorflow.python.framework.ops.EagerTensor，但是 htis 并沒有解決問題。

這是到目前為止的代碼：

accuracy = tf.keras.metrics.CategoricalAccuracy('accuracy')
loss = tf.keras.metrics.CategoricalCrossentropy('loss')
  
for epoch in range(epochs):
    accuracy.reset_states()
    loss.reset_states()
    
    for batch in iterate_minibatches(X_train, y_train, batch_size):
        imgs = batch[0]
        labels = batch[1]
        with tf.GradientTape() as tape:
            preds = model(imgs)
            
            labels = tf.convert_to_tensor(labels, dtype=tf.float32)
            
            #print(loss(labels,preds))
            # Loss is crossentropy loss with regularization term for each parameter
            total_loss = loss(labels, preds) # l2_penalty(model, theta_A) 

        grads = tape.gradient(total_loss, model.trainable_variables)
        model.optimizer.apply_gradients(zip(grads, model.trainable_variables))
       
        accuracy.update_state(labels, preds)
        loss.update_state(labels, preds)
        print("\rEpoch: {}, Batch: {}, Loss: {:.3f}, Accuracy: {:.3f}".format(
            epoch 1, batch 1, loss.result().numpy(), accuracy.result().numpy()), flush=True, end='')
        print("")
   
print("Task B accuracy after training trained model on Task B: {}".format(model.evaluate(task_B_test)))
print("Task A accuracy after training trained model on Task B: {}".format(model.evaluate(task_A_test)))

有誰知道為什么它不轉或我如何解決這個問題？

編輯：我的錯誤訊息如下所示：

AttributeError Traceback（最近一次呼叫最后一次）C:\Users\DC5DE~1.ALB\AppData\Local\Temp/ipykernel_13300/818221091.py in 34 grads = tape.gradient(total_loss, model.trainable_variables) 35 ---> 36 model.optimizer.apply_gradients(zip(grads, model.trainable_variables)) 37 38accuracy.update_state(labels, preds)

AttributeError: 'NoneType' 物件沒有屬性 'apply_gradients'

由于我不確定這是否與我將影像資料傳遞給 GradientTape 的方式有關，這里是我的 minibatch 函式：

def iterate_minibatches(inputs, targets, batchsize, shuffle=False):
    assert inputs.shape[0] == targets.shape[0]
    if shuffle:
        indices = np.arange(inputs.shape[0])
        np.random.shuffle(indices)
    for start_idx in range(0, inputs.shape[0] - batchsize   1, batchsize):
        if shuffle:
            excerpt = indices[start_idx:start_idx   batchsize]
        else:
            excerpt = slice(start_idx, start_idx   batchsize)
        yield inputs[excerpt], targets[excerpt]

另外：這里提到了一個類似的問題，但是沒有任何有效的解決方案。

uj5u.com熱心網友回復：

你把一些事情搞混了。您需要呼叫model.compile或定義自己的優化器并使用它。此外，您不應將指標與損失函式混淆。這是一個作業示例：

import tensorflow as tf

accuracy = tf.keras.metrics.CategoricalAccuracy('accuracy')
metric = tf.keras.metrics.CategoricalCrossentropy('metric_ categorical_crossentropy')
loss = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
epochs = 2
model = tf.keras.Sequential([
    tf.keras.layers.Dense(units=3, input_shape=(1,))
]) 
optimizer = tf.keras.optimizers.Adam()

dataset = tf.data.Dataset.from_tensor_slices((tf.random.normal((50, 1)), tf.random.normal((50, 3)))).batch(5)
for epoch in range(epochs):
    accuracy.reset_states()
    metric.reset_states()
    
    for i, batch in enumerate(dataset):
        imgs = batch[0]
        labels = batch[1]
        print(imgs.shape, labels.shape)
        with tf.GradientTape() as tape:
            preds = model(imgs)
                      
            #print(loss(labels,preds))
            # Loss is crossentropy loss with regularization term for each parameter
            total_loss = loss(labels, preds) # l2_penalty(model, theta_A) 

        grads = tape.gradient(total_loss, model.trainable_variables)
        optimizer.apply_gradients(zip(grads, model.trainable_variables))
       
        accuracy.update_state(labels, preds)
        metric.update_state(labels, preds)
        print("\rEpoch: {}, Batch: {}, Loss: {:.3f}, Accuracy: {:.3f}".format(
            epoch 1, i 1, metric.result().numpy(), accuracy.result().numpy()), flush=True, end='')
        print("")

Epoch: 1, Batch: 1, Loss: 4.209, Accuracy: 0.200
Epoch: 1, Batch: 2, Loss: 1.641, Accuracy: 0.400
Epoch: 1, Batch: 3, Loss: 1.294, Accuracy: 0.333
Epoch: 1, Batch: 4, Loss: 1.025, Accuracy: 0.300
Epoch: 1, Batch: 5, Loss: -0.110, Accuracy: 0.320
Epoch: 1, Batch: 6, Loss: 0.316, Accuracy: 0.267
Epoch: 1, Batch: 7, Loss: -0.118, Accuracy: 0.257
Epoch: 1, Batch: 8, Loss: -0.284, Accuracy: 0.225
Epoch: 1, Batch: 9, Loss: -0.249, Accuracy: 0.244
Epoch: 1, Batch: 10, Loss: -0.464, Accuracy: 0.260
Epoch: 2, Batch: 1, Loss: 4.468, Accuracy: 0.200
Epoch: 2, Batch: 2, Loss: 1.578, Accuracy: 0.400
Epoch: 2, Batch: 3, Loss: 1.012, Accuracy: 0.400
Epoch: 2, Batch: 4, Loss: 0.836, Accuracy: 0.350
Epoch: 2, Batch: 5, Loss: -0.294, Accuracy: 0.360
Epoch: 2, Batch: 6, Loss: 0.168, Accuracy: 0.300
Epoch: 2, Batch: 7, Loss: -0.201, Accuracy: 0.286
Epoch: 2, Batch: 8, Loss: -0.634, Accuracy: 0.250
Epoch: 2, Batch: 9, Loss: -0.552, Accuracy: 0.267
Epoch: 2, Batch: 10, Loss: -0.920, Accuracy: 0.280

uj5u.com熱心網友回復：

您需要tf.keras.losses.CategoricalCrossentropy用于損失計算，而不是tf.keras.metrics.CategoricalCrossentropy其作業方式不同并且會停止梯度傳播。

標籤：Python 张量流 凯拉斯

上一篇：關于基于秩的計算的自動微分

下一篇：jqueryclick函式宣告在運行時評估變數