假設我有一個分類問題，它有 30 個潛在的二元標簽。這些標簽并不相互排斥。標簽往往是稀疏的——平均而言，每 30 個標簽有 1 個正標簽，但有時不止 1 個。在下面的代碼中，我如何通過預測全零來懲罰模型？準確率會很高，但召回率會很糟糕！
import numpy as np
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
OUTPUT_NODES = 30
np.random.seed(0)
def get_dataset():
"""
Get a dataset of X and y. This is a learnable problem as there is some signal in the features. 10% of the time, a
positive-output's index will also have a positive feature for that index
:return: X and y data for training
"""
n_observations = 30000
y = np.random.rand(n_observations, OUTPUT_NODES)
y = (y <= (1 / OUTPUT_NODES)).astype(int) # Makes a sparse output where there is roughly 1 positive label: ((1 / OUTPUT_NODES) * OUTPUT_NODES ≈ 1)
X = np.zeros((n_observations, OUTPUT_NODES))
for i in range(len(y)):
for j, feature in enumerate(y[i]):
if feature == 1:
X[i][j] = 1 if np.random.rand(1) > 0.9 else 0 # Makes the input features more noisy
# X[i][j] = 1 # Using this instead will make the model perform very well
return X, y
def create_model():
input_layer = Input(shape=(OUTPUT_NODES, ))
dense1 = Dense(100, activation='relu')(input_layer)
dense2 = Dense(100, activation='relu')(dense1)
output_layer = Dense(30, activation='sigmoid')(dense2)
model = Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['Recall'])
return model
def main():
X, y = get_dataset()
model = create_model()
model.fit(X, y, epochs=10, batch_size=10)
X_pred = np.random.randint(0, 2, (100, OUTPUT_NODES))
y_pred = model.predict(X_pred)
print(X_pred)
print(y_pred.round(1))
if __name__ == '__main__':
main()
我相信我在這里讀到我可以使用：
weighted_cross_entropy_with_logits
來解決這個問題。這將如何影響我最終輸出層的激活函式？我必須要有激活函式嗎？如何指定對真正類的錯誤分類的懲罰？
uj5u.com熱心網友回復：
好的，這是一個有趣的問題
首先你需要定義一個加權交叉熵損失包裝器：
def wce_logits(positive_class_weight=1.):
def mylossw(y_true, logits):
cross_entropy = tf.reduce_mean(tf.nn.weighted_cross_entropy_with_logits(logits=logits, labels=tf.cast(y_true, dtype=tf.float32), pos_weight=positive_class_weight))
return cross_entropy
return mylossw
所述positive_class_weight被施加到正類的資料。您需要tf.nn.weighted_cross_entropy_with_logits 的這個包裝器來獲得一個將 y_true 和 y_pred （僅）作為輸入的損失函式。請注意，您必須將 y_true 轉換為 float32。
其次，您不能使用預定義的 Recall，因為它不適用于 logits。我在這個討論中找到了一個解決方法
class Recall(tf.keras.metrics.Recall):
def __init__(self, from_logits=False, *args, **kwargs):
super().__init__(*args, **kwargs)
self._from_logits = from_logits
def update_state(self, y_true, y_pred, sample_weight=None):
if self._from_logits:
super(Recall, self).update_state(y_true, tf.nn.sigmoid(y_pred), sample_weight)
else:
super(Recall, self).update_state(y_true, y_pred, sample_weight)
最后，您需要在使用 logits 時從最后一層洗掉 sigmoid 激活
def create_model():
input_layer = Input(shape=(OUTPUT_NODES, ))
dense1 = Dense(100, activation='relu')(input_layer)
dense2 = Dense(100, activation='relu')(dense1)
output_layer = Dense(30)(dense2)
model = Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss=wce_logits(positive_class_weight=27.), metrics=[Recall(from_logits=True)])
return model
請注意，這里的正權重設定為 27。您可以閱讀有關如何正確計算重量的討論
轉載請註明出處，本文鏈接：https://www.uj5u.com/caozuo/351802.html