ValueError：使用MultiLabelBinarizer后形狀不一致

我正在嘗試為我的 CRF 模型創建一個性能評估結果，它描述了這個詞所屬的詞性。我創建了一個函式來以更“資料集”的格式轉換資料。此函式將資料作為兩個串列回傳，一個是特征字典，另一個是標簽。

def transform_to_dataset(tagged_sentences):
    X, y = [], []
    for sentence, tags in tagged_sentences:
        sent_word_features, sent_tags = [], []
        for index in range(len(sentence)):
            sent_word_features.append(extract_features(sentence, index)),
            sent_tags.append(tags[index])
        X.append(sent_word_features)
        y.append(sent_tags)
    return X, y

然后我將編碼前的集合劃分為訓練/測驗集中的完整句子。

penn_train_size = int(0.8*len(penn_treebank))
penn_training = penn_treebank[:penn_train_size]
penn_testing = penn_treebank[penn_train_size:]
X_penn_train, y_penn_train = transform_to_dataset(penn_training)
X_penn_test, y_penn_test = transform_to_dataset(penn_testing)

然后我加載模型來訓練和測驗我的資料

penn_crf = CRF(
    algorithm='lbfgs',
    c1=0.01,
    c2=0.1,
    max_iterations=100,
    all_possible_transitions=True
)
#The fit method is the default name used by Machine Learning algorithms to start training.
print("Started training on Penn Treebank corpus!")
penn_crf.fit(X_penn_train, y_penn_train)
print("Finished training on Penn Treebank corpus!")

然后我用

y_penn_pred=penn_crf.predict(X_penn_test)

但是當我嘗試

from sklearn.metrics import accuracy_score

print("Accuracy: ", accuracy_score(y_penn_test, y_penn_pred))

它發出一個錯誤：

ValueError：您似乎正在使用舊的多標簽資料表示。不再支持序列序列；改用二進制陣列或稀疏矩陣 - MultiLabelBinarizer 轉換器可以轉換為這種格式。

但是當我嘗試使用 MultiLabelBinarizer 時；

from sklearn.preprocessing import MultiLabelBinarizer

bin_y_penn_test = MultiLabelBinarizer().fit_transform(y_penn_test)
bin_y_penn_pred = MultiLabelBinarizer().fit_transform(y_penn_pred)

它給了我一個錯誤：

ValueError：形狀不一致

這是完整的追溯

-------------------------------------------------- ------------------------- ValueError Traceback（最近一次呼叫最后）/tmp/ipykernel_5694/856179584.py in 1 from sklearn.metrics import accuracy_score 2 ----> 3 print("精度：", accuracy_score(bin_y_penn_test, bin_y_penn_pred))

~/.local/lib/python3.8/site-packages/sklearn/utils/validation.py in inner_f(*args, **kwargs) 61 extra_args = len(args) - len(all_args) 62 如果 extra_args <= 0 : ---> 63 return f(*args, **kwargs) 64 65 # extra_args > 0

~/.local/lib/python3.8/site-packages/sklearn/metrics/_classification.py in accuracy_score(y_true, y_pred, normalize, sample_weight) 203 check_consistent_length(y_true, y_pred, sample_weight) 204 if y_type.startswith('multilabel '): --> 205 different_labels = count_nonzero(y_true - y_pred, axis=1) 206 score = different_labels == 0 207 else:

~/.local/lib/python3.8/site-packages/scipy/sparse/base.py in sub (self, other) 431 elif isspmatrix(other): 432 if other.shape != self.shape: --> 433 raise ValueError("inconsistent shapes") 434 return self._sub_sparse(other) 435 elif isdense(other):

ValueError：形狀不一致

我應該怎么做才能產生模型的混淆矩陣？

uj5u.com熱心網友回復：

嘗試做

penn_train_size = int(0.7*len(penn_treebank))

然后檢查形狀是否仍然不一致

bin_y_penn_test.shape
bin_y_penn_pred.shape
if bin_y_penn_test.shape == bin_y_penn_pred.shape:
    print('Consistent Shape')
else:
    print('Inconsistent Shape')

如果它發出一致的形狀，做

from sklearn.metrics import multilabel_confusion_matrix
multilabel_confusion_matrix(bin_y_penn_test, bin_y_penn_pred)

但是，如果仍然不一致，請嘗試修改您的資料。

標籤：Python 机器学习 scikit-学习 混淆矩阵 crf

上一篇：K-Fold交叉驗證的應用和部署

下一篇：TypeError：傳遞給引數“輸入”的值具有DataTypebool不在允許值串列中，將檔案中的資料傳遞給模型