將值附加到CSV中的新列

我有兩個 CSV，一個是主資料，另一個是組件資料，主資料有兩行和兩列，其中組件資料有 5 行和兩列。

我試圖在標記化、詞干化和詞形還原之后找到它們之間的余弦相似度，然后將相似度索引附加到新列，我無法將相應的值附加到資料幀中的列中進一步需要轉換為 CSV。

我的方法：

import re
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer,WordNetLemmatizer
from collections import Counter
import pandas as pd

portStemmer=PorterStemmer()
wordNetLemmatizer = WordNetLemmatizer()
fields = ['Sentences']
cosineSimilarityList = []

def fetchLemmantizedWords():
    eliminatePunctuation = re.sub('[^a-zA-Z]', ' ',value)
    convertLowerCase = eliminatePunctuation.lower()
    tokenizeData = convertLowerCase.split()
    eliminateStopWords = [word for word in tokenizeData if not word in set(stopwords.words('english'))]
    stemWords= list(set([portStemmer.stem(value) for value in eliminateStopWords]))
    wordLemmatization = [wordNetLemmatizer.lemmatize(x) for x in stemWords]
    return wordLemmatization

def fetchCosine(eachMasterData,eachComponentData):
    masterDataValues = Counter(eachMasterData)
    componentDataValues = Counter(eachComponentData)
    bagOfWords  = list(masterDataValues.keys() | componentDataValues.keys())
    masterDataVector = [masterDataValues.get(bagOfWords, 0) for bagOfWords in bagOfWords]    
    componentDataVector = [componentDataValues.get(bagOfWords, 0) for bagOfWords in bagOfWords]          
    masterDataLength  = sum(contractElement*contractElement for contractElement in masterDataVector) ** 0.5                
    componentDataLength  = sum(questionElement*questionElement for questionElement in componentDataVector) ** 0.5         
    dotProduct    = sum(contractElement*questionElement for contractElement,questionElement in zip(masterDataVector, componentDataVector))      
    cosine = int((dotProduct / (masterDataLength * componentDataLength))*100) 
    return cosine

masterData = pd.read_csv('C:\\Similarity\\MasterData.csv', skipinitialspace=True)
componentData =  pd.read_csv('C:\\Similarity\\ComponentData.csv', skipinitialspace=True)
for value in masterData['Sentences']:
    eachMasterData = fetchLemmantizedWords()
    for value in componentData['Sentences']:
        eachComponentData = fetchLemmantizedWords()
        cosineSimilarity = fetchCosine(eachMasterData,eachComponentData)
        cosineSimilarityList.append(cosineSimilarity)
    for value in cosineSimilarityList:
        componentData = componentData.append(pd.DataFrame(cosineSimilarityList, columns=['Cosine Similarity']), ignore_index=True)
        #componentData['Cosine Similarity'] = value

將 df 轉換為 CSV 后的預期輸出，

在將值附加到資料框時面臨問題，請幫助我解決此問題。謝謝。

uj5u.com熱心網友回復：

這是我想出的：

樣品設定

csv_master_data = \
"""
SI.No;Sentences
1;Emma is writing a letter.
2;We wake up early in the morning.
"""

csv_component_data = \
"""
SI.No;Sentences
1;Emma is writing a letter.
2;We wake up early in the morning.
3;Did Emma Write a letter?
4;We sleep early at night.
5;Emma wrote a letter.
"""

import pandas as pd
from io import StringIO

df_md = pd.read_csv(StringIO(csv_master_data), delimiter=';')
df_cd = pd.read_csv(StringIO(csv_component_data), delimiter=';')

我們最終得到 2 個資料幀（顯示df_cd）：

我用以下虛擬函式替換了您使用的 2 個函式：

import random

def fetchLemmantizedWords(words):
    return [random.randint(1,30) for x in  words]

def fetchCosine(lem_md, lem_cd):
    return 100 if len(lem_md) == len(lem_cd) else random.randint(0,100)

處理資料

首先，我們fetchLemmantizedWords在每個資料幀上應用該函式。句子的正則運算式替換、小寫和拆分由 Pandas 完成，而不是在函式本身中完成。

通過首先使句子小寫，我們可以將正則運算式簡化為只考慮小寫字母。

for df in (df_md, df_cd):
    df['lem'] = df.apply(lambda x: fetchLemmantizedWords(x.Sentences
                                                         .lower()
                                                         .replace(r'[^a-z]', ' ')
                                                         .split()), 
                         result_type='reduce', 
                         axis=1)

結果df_cd：

接下來，我們使用交叉連接來創建一個包含所有可能的資料組合的資料md框cd。

df_merged = pd.merge(df_md[['SI.No', 'lem']], 
                     df_cd[['SI.No', 'lem']], 
                     how='cross', 
                     suffixes=('_md','_cd')
                    )

df_merged內容：

Next, we calculate the cosine value:

df_merged['cosine'] = df_merged.apply(lambda x: fetchCosine(x.lem_md, 
                                                            x.lem_cd), 
                                      axis=1)

In the last step, we pivot the data and merge the original df_cd with the calculated results :

pd.merge(df_cd.drop(columns='lem').set_index('SI.No'),
         df_merged.pivot_table(index='SI.No_cd', 
                               columns='SI.No_md').droplevel(0, axis=1),
         how='inner',
         left_index=True, 
         right_index=True)

Result (again, these are dummy calculations):

標籤：Python python-3.x 列表 数据框 CSV

上一篇：將XML決議為CSV的多個for回圈不起作用

下一篇：如何用python在csv中追加列