在多處理中,我想更新manager.dict(),它正在更新……但是更新時跳過了一些資料?可以做什么?這是類似的東西......
from multiprocessing import Process, Manager
manager = Manager()
a = manager.dict()
a['url_info'] = manager.list()
def parse_link(link):
# parse link, pared_info returns dict
pared_info = link_parser(link)
a['url_info'].append(pared_info)
# Links contains a lot of url that needs to be parsed.
links = ["https://url.com/1","https://url.com/2", "https://url.com/3"]
processes = []
for link in links:
p = Process(target=parse_link, args=link,))
p.start()
processes.append(p)
for process in processes:
process.join()
link_parser()是一個回傳字典的函式,其中包含有關抓取/決議的網頁的資訊。
> print(list(a['url_info']))
> ['#info_1', '#info_3']
在這里,多處理程式跳過#info_2了串列(又名陣列)中的更新。請幫幫我
uj5u.com熱心網友回復:
這是一些代碼,演示了您正在嘗試執行的改進結構。
顯然它沒有你的link_parser()的細節,但你會明白的。
from concurrent.futures import ProcessPoolExecutor
from multiprocessing import Manager
from functools import partial
LINKS = ['abc', 'def', 'ghi']
KEY = 'url_info'
def parse_link(a, link):
a[KEY].append(link)
def main():
with Manager() as manager:
a = manager.dict()
a[KEY] = manager.list()
with ProcessPoolExecutor() as executor:
executor.map(partial(parse_link, a), LINKS)
print(a[KEY])
if __name__ == '__main__':
main()
輸出:
['abc', 'def', 'ghi']
轉載請註明出處,本文鏈接:https://www.uj5u.com/yidong/436880.html
標籤:Python python-3.x 字典 多处理
上一篇:如何在這里列印鍵的值?
