執行緒化NNTP，如何？（這里是新手）

我無法理解如何將我的代碼重寫為多執行緒。

我正在撰寫的代碼用于自動歸檔現有新聞組串列中的每篇文章，但我希望能夠利用我的新聞組計劃并使其最多 20 個執行緒。我以前從未撰寫過執行緒代碼，而且我的嘗試很順利。

這是我的代碼，不包括用戶名和密碼（但如果你真的想在https://my.xsusenet.com獲得一個最多 5 個執行緒的免費帳戶）

請不要過分評價我:(

import nntplib
import sys
import datetime
import os
basetime = datetime.datetime.today()
#daysback = int(sys.argv[1])
#date_list = [basetime - datetime.timedelta(days=x) for x in range(daysback)]
s = nntplib.NNTP('free.xsusenet.com', user='USERNAME', password='PASSWORD') # I am only allowed 5 connections at a time, so try for 4.
groups = []
resp, groups_list_tuple = s.list()


def remove_non_ascii_2(string):
    return string.encode('ascii', errors='ignore').decode()


for g_tuple in groups_list_tuple:
    #print(g_tuple) # DEBUG_LINE
    # Parse group_list info
    group = g_tuple[0]
    last = g_tuple[1]
    first = g_tuple[2]
    flag = g_tuple[3]

    # Parse newsgroup info
    resp, count, first, last, name = s.group(group)
    for message_id in range(first, last):
        resp, number, mes_id = s.next()
        resp, info = s.article(mes_id)
        if os.path.exists('.\\'   group):
            pass
        else:
            os.mkdir('.\\'   group)
        print(f"Downloading: {message_id}")
        outfile = open('.\\'   group   '\\'   str(message_id), 'a', encoding="utf-8")
        for line in info.lines:
            outfile.write(remove_non_ascii_2(str(line))   '\n')
        outfile.close()

嘗試使用 ThreadPoolExecutor 進行執行緒化，使其使用 20 個執行緒，但失敗了，導致它對相同的訊息 id 重復相同的程序。預期的結果是一次下載 20 條不同的訊息。

這是我嘗試使用執行緒的代碼，請注意，我確實喜歡它的 6-8 種變體來嘗試讓它作業，這是我放棄在這里詢問之前的最后一個。

import nntplib
import sys
import datetime
import os
import concurrent.futures
basetime = datetime.datetime.today()
#daysback = int(sys.argv[1])
#date_list = [basetime - datetime.timedelta(days=x) for x in range(daysback)]
s = nntplib.NNTP('free.xsusenet.com', user='USERNAME', password='PASSWORD') # I am only allowed 5 connections at a time, so try for 4.
groups = []
resp, groups_list_tuple = s.list()


def remove_non_ascii_2(string):
    return string.encode('ascii', errors='ignore').decode()

def download_nntp_file(mess_id):
    resp, count, first, last, name = s.group(group)
    message_id = range(first, last)

    resp, number, mes_id = s.next()
    resp, info = s.article(mes_id)
    if os.path.exists('.\\'   group):
        pass
    else:
        os.mkdir('.\\'   group)
    print(f"Downloading: {mess_id}")
    outfile = open('.\\'   group   '\\'   str(mess_id), 'a', encoding="utf-8")
    for line in info.lines:
        outfile.write(remove_non_ascii_2(str(line))   '\n')
    outfile.close()


for g_tuple in groups_list_tuple:
    #print(g_tuple) # DEBUG_LINE
    # Parse group_list info
    group = g_tuple[0]
    last = g_tuple[1]
    first = g_tuple[2]
    flag = g_tuple[3]

    # Parse newsgroup info
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        futures = executor.submit(download_nntp_file)

uj5u.com熱心網友回復：

我不能用XSUseNet.

我不會使用全域變數，因為當行程同時作業時，它們可能會從這些變數中獲得相同的值。

您應該將值作為引數發送給函式。

像這樣的東西：

def download_nntp_file(g_tuple):
    # ... code which uses `g_tuple` instead of global variables ...

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    for g_tuple in groups_list_tuple:
        executor.submit(download_nntp_file, g_tuple)

但我會更簡單地使用map()而不是submit()因為它獲取帶有引數的串列并且它不需要for-loop

def download_nntp_file(g_tuple):
    # ... code which uses `g_tuple` instead of global variables ...

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    executor.map(download_nntp_file, groups_list_tuple)

標籤：Python多线程ntp

上一篇：彼得森的解決方案只使用一個變數

下一篇：ConcurrentBag<T>與list<T>的用法