如何對AttributeSelectedClassifier模型進行交叉驗證？-有解無憂

我做了一個這樣的模型：

base = Classifier(classname="weka.classifiers.trees.ADTree", 
                  options=["-B", "10", "-E", "-3", "-S", "1"])

CostS_cls = SingleClassifierEnhancer(classname="weka.classifiers.meta.CostSensitiveClassifier", 
                                options =["-cost-matrix", "[0.0 1.0; 1.0 0.0]", "-S", "1"])
CostS_cls.classifier = base
smote = Filter(classname="weka.filters.supervised.instance.SMOTE", 
               options=["-C", "0", "-K", "3", "-P", "250.0", "-S", "1"])
fc = FilteredClassifier(options=["-S","1"])
fc.filter = smote
fc.classifier = CostS_cls
bagging_cls = SingleClassifierEnhancer(classname="weka.classifiers.meta.Bagging",
                         options=["-P", "100", "-S", "1", "-num-slots", "1", "-I", "100"])
bagging_cls.classifier = fc
multisearch_cls = MultiSearch(options = ["-S", "1"])
multisearch_cls.evaluation = "FM"
multisearch_cls.search = ["-sample-size", "100", "-initial-folds", "2", "-subsequent-folds", "10",
                          "-initial-test-set", ".", "-subsequent-test-set", ".", "-num-slots", "1"]                        
mparam = MathParameter()
mparam.prop = "numOfBoostingIterations"
mparam.minimum = 5.0
mparam.maximum = 50.0
mparam.step = 1.0
mparam.base = 10.0
mparam.expression = "I"
multisearch_cls.parameters = [mparam]
multisearch_cls.classifier = bagging_cls
AttS_cls = AttributeSelectedClassifier()
AttS_cls.search = from_commandline('weka.attributeSelection.GreedyStepwise -B -T -1.7976931348623157E308 -N -1 -num-slots 1', classname=get_classname(ASSearch))
AttS_cls.evaluation = from_commandline('weka.attributeSelection.CfsSubsetEval -P 1 -E 1', classname=get_classname(ASEvaluation))
AttS_cls.classifier = multisearch_cls
train, test = data_modelos_1_2.train_test_split(70.0, Random(1))
AttS_cls.build_classifier(train)

我正在嘗試通過交叉驗證來驗證它，但是當我這樣做時：

train, test = data_modelos_1_2.train_test_split(70.0, Random(1))
AttS_cls.build_classifier(train)
evl = Evaluation(test)
evl.crossvalidate_model(AttS_cls, test, 10, Random(1))

我收到此錯誤：

---------------------------------------------------------------------------
JavaException                             Traceback (most recent call last)
/tmp/ipykernel_50548/1197040560.py in <module>
     47 print(AttS_cls.to_commandline())
     48 evl = Evaluation(test)
---> 49 evl.crossvalidate_model(AttS_cls, test, 10, Random(1))
     50 print(AttS_cls)
     51 print("----------------------------------------------------------------------------")

/usr/local/lib/python3.8/dist-packages/weka/classifiers.py in crossvalidate_model(self, classifier, data, num_folds, rnd, output)
   1289         else:
   1290             generator = [output.jobject]
-> 1291         javabridge.call(
   1292             self.jobject, "crossValidateModel",
   1293             "(Lweka/classifiers/Classifier;Lweka/core/Instances;ILjava/util/Random;[Ljava/lang/Object;)V",

~/.local/lib/python3.8/site-packages/javabridge/jutil.py in call(o, method_name, sig, *args)
    890     ret_sig = sig[sig.find(')') 1:]
    891     nice_args = get_nice_args(args, args_sig)
--> 892     result = fn(*nice_args)
    893     x = env.exception_occurred()
    894     if x is not None:

~/.local/lib/python3.8/site-packages/javabridge/jutil.py in fn(*args)
    857             x = env.exception_occurred()
    858             if x is not None:
--> 859                 raise JavaException(x)
    860             return result
    861     else:

JavaException: Thread-based execution of evaluation tasks failed!

所以我不知道我做錯了什么，因為我知道使用 weka 你可以交叉驗證這種型別的模型，但我正在嘗試 pyweka 并遇到了這個問題。

uj5u.com熱心網友回復：

我已經將您的代碼片段變成了一個帶有匯入的代碼片段，并修復了 Bagging 的 MultiSearch 設定（mparam.prop = "numIterations"而不是mparam.prop = "numOfBoostingIterations"），允許執行它。

由于我無權訪問您的資料，因此我只使用了 UCI 資料集vote.arff。

您的代碼有點奇怪，因為它進行了 70/30 的訓練/測驗拆分，訓練了分類器，然后對測驗資料進行了交叉驗證。對于交叉驗證，您不需要訓練分類器，因為這發生在內部交叉驗證回圈中（該回圈內的每個訓練分類器都會被丟棄，因為交叉驗證僅用于收集統計資料）。

因此，下面的代碼包含三個部分：

您的原始評估代碼，但已注釋掉
執行適當的交叉驗證
執行訓練/測驗評估

我不使用 Jupyter 筆記本并在我的 Linux Mint 上的常規虛擬環境中成功測驗了代碼：

Python：3.8.10

輸出pip freeze：

numpy==1.22.3
packaging==21.3
pyparsing==3.0.7
python-javabridge==4.0.3
python-weka-wrapper3==0.2.7

修改后的代碼本身：

import weka.core.jvm as jvm
from weka.core.converters import load_any_file
from weka.classifiers import Classifier, SingleClassifierEnhancer, FilteredClassifier, MultiSearch, AttributeSelectedClassifier, Evaluation
from weka.core.classes import MathParameter, from_commandline, Random, get_classname
from weka.filters import Filter
from weka.attribute_selection import ASEvaluation, ASSearch

jvm.start(packages=True)

# the dataset/path needs adjusting
data_modelos_1_2 = load_any_file("/some/where/vote.arff")
data_modelos_1_2.class_is_last()

base = Classifier(classname="weka.classifiers.trees.ADTree",
                  options=["-B", "10", "-E", "-3", "-S", "1"])

CostS_cls = SingleClassifierEnhancer(classname="weka.classifiers.meta.CostSensitiveClassifier",
                                     options=["-cost-matrix", "[0.0 1.0; 1.0 0.0]", "-S", "1"])
CostS_cls.classifier = base
smote = Filter(classname="weka.filters.supervised.instance.SMOTE",
               options=["-C", "0", "-K", "3", "-P", "250.0", "-S", "1"])
fc = FilteredClassifier(options=["-S", "1"])
fc.filter = smote
fc.classifier = CostS_cls
bagging_cls = SingleClassifierEnhancer(classname="weka.classifiers.meta.Bagging",
                                       options=["-P", "100", "-S", "1", "-num-slots", "1", "-I", "100"])
bagging_cls.classifier = fc
multisearch_cls = MultiSearch(options=["-S", "1"])
multisearch_cls.evaluation = "FM"
multisearch_cls.search = ["-sample-size", "100", "-initial-folds", "2", "-subsequent-folds", "10",
                          "-initial-test-set", ".", "-subsequent-test-set", ".", "-num-slots", "1"]
mparam = MathParameter()
mparam.prop = "numIterations"
mparam.minimum = 5.0
mparam.maximum = 50.0
mparam.step = 1.0
mparam.base = 10.0
mparam.expression = "I"
multisearch_cls.parameters = [mparam]
multisearch_cls.classifier = bagging_cls

AttS_cls = AttributeSelectedClassifier()
AttS_cls.search = from_commandline('weka.attributeSelection.GreedyStepwise -B -T -1.7976931348623157E308 -N -1 -num-slots 1', classname=get_classname(ASSearch))
AttS_cls.evaluation = from_commandline('weka.attributeSelection.CfsSubsetEval -P 1 -E 1', classname=get_classname(ASEvaluation))
AttS_cls.classifier = multisearch_cls

# original
# train, test = data_modelos_1_2.train_test_split(70.0, Random(1))
# AttS_cls.build_classifier(train)
# evl = Evaluation(test)
# evl.crossvalidate_model(AttS_cls, test, 10, Random(1))
# print(evl.summary())

# cross-validation
print("\ncross-validation\n")
evl = Evaluation(data_modelos_1_2)
evl.crossvalidate_model(AttS_cls, data_modelos_1_2, 10, Random(1))
print(evl.summary())

# train/test split
print("\ntrain/test split\n")
train, test = data_modelos_1_2.train_test_split(70.0, Random(1))
AttS_cls.build_classifier(train)
evl = Evaluation(test)
evl.test_model(AttS_cls, test)
print(evl.summary())

jvm.stop()

這產生了以下輸出：

cross-validation


Correctly Classified Instances         416               95.6322 %
Incorrectly Classified Instances        19                4.3678 %
Kappa statistic                          0.9094
Mean absolute error                      0.0737
Root mean squared error                  0.1778
Relative absolute error                 15.5353 %
Root relative squared error             36.5084 %
Total Number of Instances              435     


train/test split


Correctly Classified Instances         126               96.1832 %
Incorrectly Classified Instances         5                3.8168 %
Kappa statistic                          0.9216
Mean absolute error                      0.0735
Root mean squared error                  0.1649
Relative absolute error                 15.3354 %
Root relative squared error             33.6949 %
Total Number of Instances              131

轉載請註明出處，本文鏈接：https://www.uj5u.com/ruanti/445402.html

標籤：Python 机器学习分类威卡交叉验证

上一篇：什么是`Prism'sa`但帶有背景關系`ctx`？

下一篇：keras圖層類中缺少特定選項