我正在使用這個Notebook，其中Apply DocumentClassifier部分更改如下。
Jupyter 實驗室，內核：conda_mxnet_latest_p37.
我了解錯誤意味著我正在傳遞str而不是int. 但是，這應該不是問題，因為它適用于原始 Notebook 中的其他 .pdf/.txt 檔案。
代碼單元：
doc_dir = "GRIs/" # contains 2 .pdfs
with open('filt_gri.txt', 'r') as filehandle:
tags = [current_place.rstrip() for current_place in filehandle.readlines()]
doc_classifier = TransformersDocumentClassifier(model_name_or_path="cross-encoder/nli-distilroberta-base",
task="zero-shot-classification",
labels=tags,
batch_size=2)
# convert to Document using a fieldmap for custom content fields the classification should run on
docs_to_classify = [Document.from_dict(d) for d in docs_sliding_window]
# classify using gpu, batch_size makes sure we do not run out of memory
classified_docs = doc_classifier.predict(docs_to_classify)
# let's see how it looks: there should be a classification result in the meta entry containing labels and scores.
print(classified_docs[0].to_dict())
all_docs = convert_files_to_dicts(dir_path=doc_dir)
preprocessor_sliding_window = PreProcessor(split_overlap=3,
split_length=10,
split_respect_sentence_boundary=False,
split_by='passage')
輸出錯誤：
INFO - haystack.modeling.utils - Using devices: CUDA
INFO - haystack.modeling.utils - Number of GPUs: 1
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-11-82b54cd162ff> in <module>
14
15 # classify using gpu, batch_size makes sure we do not run out of memory
---> 16 classified_docs = doc_classifier.predict(docs_to_classify)
17
18 # let's see how it looks: there should be a classification result in the meta entry containing labels and scores.
~/anaconda3/envs/mxnet_latest_p37/lib/python3.7/site-packages/haystack/nodes/document_classifier/transformers.py in predict(self, documents)
144 for prediction, doc in zip(predictions, documents):
145 if self.task == 'zero-shot-classification':
--> 146 prediction["label"] = prediction["labels"][0]
147 doc.meta["classification"] = prediction
148
TypeError: string indices must be integers
如果還有什么我應該添加到帖子/澄清中，請告訴我。
uj5u.com熱心網友回復：
我docs_sliding_window用my_dsw.
my_dsw只保留帶有<= 1000長度字符的行。這有助于我的資料形狀更好地適應。
my_dsw = []
for dsw in range(0, len(docs_sliding_window)-1):
if len(docs_sliding_window[dsw]['content']) <= 1000:
my_dsw.append(docs_sliding_window[dsw])
將它交換出來docs_to_classify：
# convert to Document using a fieldmap for custom content fields the classification should run on
docs_to_classify = [Document.from_dict(d) for d in docs_sliding_window]
誠然，我不確定這與錯誤有何具體關系；但它確實有助于資料更好地擬合；現在我可以增加batch_size=4。
轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/383828.html
標籤：蟒蛇-3.x 弹性搜索 机器学习 类型错误 jupyter 实验室